Test­ing

It is interesting to have a different approach to the structure of an proposal. It is also helpful that [...] you have to reflect: Where do I want to go and what do I need and what is our societal impact?

In order to develop the SynSICRIS monitoring tool in the best possible way for its use in research and funding processes, extensive testing was carried out with the central user group of researchers (11/2021 to 03/2022). The focus was on identifying problems and difficulties in using the tool. Two methodological approaches were chosen for testing: Usability tests and accompanied testing.

A total of 29 researchers from publicly funded research projects of the funding bodies BLE and DLR took part in usability tests and tested the user-friendliness of the monitoring tool in approx. 80 test sessions of 75 minutes. In parallel, eight researchers further used the SynSICRIS monitoring tool to describe a currently approved project overall.

When researchers were approached as test subjects, a high level of interest in a tool to support research and funding processes and impact orientation was noted. The high level of commitment and motivation of the test subjects clearly contributed to the development of the SynSICRIS monitoring tool.

The testing has shown that the monitoring tool is already easy to use in the current beta version (see Figure 1). Almost two thirds of the researchers involved would recommend the monitoring tool (see Figure 2). In particular, the range of functions was rated very positively (see Figure 3).

Fig­ure 1 and 2: Pos­it­ive eval­u­ation in the us­ab­il­ity tests

ToolVerwendungsempfehlung-Erprobende
Own representation; final survey of researchers in usability tests. N=29.
Tool-Funktionsumfang
Own representation; final survey of researchers in usability tests. N=29.
Toolempfehlung aus Erprobung
Own representation; final survey of researchers in usability tests. N=29.

I read the help fields, but once I got a feel for it, it was actually quite clear to me and the help was very useful.

The use of the Impact Pathway from application to reporting was assessed as helpful, as was the support in project management, e.g. through the working plan. Working in the Impact Pathway was initially new and unfamiliar for the test persons. Overall, the Impact Pathway was assessed as useful for the structured presentation of the research work and planned impacts.

The testing also revealed technical errors and provided many valuable tips as well as suggestions for improved user-friendliness. During the joint use, it became obvious, among other things, that a comment function on the processing status was needed. The greatest need for improvement was revealed with regard to better orientation in the tool.

All results are now being incorporated into the agile process of further developing the content and technology of the monitoring tool in the project. Further tests are planned from autumn 2022, also with the user group of funding bodies.

Very cool that in one place after the end of the project it can be shown what is developing.