Testing
Participant usability test
It is interesting to have a different approach to the structure of an proposal. It is also helpful that [...] you have to reflect: Where do I want to go and what do I need and what is our societal impact?
In order to develop the SynSICRIS monitoring tool in the best possible way for its use in research and funding processes, extensive testing was carried out with the central user group of researchers (11/2021 to 03/2022). The focus was on identifying problems and difficulties in use. Two methodological approaches were chosen for testing: Usability tests and accompanied tests. Further tests were carried out in the spring of 2023 with the user groups researchers, funding bodies and knowledge transfer.
Participant usability test
The fact that the proposal data for work packages, milestones, etc. are stored and that all partners have access would save a lot of time and work in the case of project changes and interim reports.
When researchers were addressed as test subjects, a high level of interest in a tool to support research and funding processes and impact orientation was observed. The high level of commitment and motivation of the test subjects have contributed significantly to the development of the SynSICRIS monitoring tool.
The usability tests showed that the monitoring tool can already be easily used in the current beta version by researchers and staff of funding bodies (see Figure 1). Two thirds of the respondents would recommend the monitoring tool (see Figure 2). In particular, the range of functions was rated very positively by the researchers (see Figure 3).
Figure 1 to 3: Evaluation in the usability tests from 2021-2023.
Participant usability test
I read the help fields, but once I got a feel for it, it was actually quite clear to me and the help was very useful.
The use from proposal submission to reporting was assessed as helpful, as was the support in project management, e.g. through the work plan. Working in the impact pathway was initially new and unfamiliar for the test persons. Overall, the impact pathway was assessed as useful for the structured presentation of the research work and planned impacts.
The testing also revealed technical errors and many valuable tips as well as suggestions for improved usability were given. The greatest need for improvement was found with regard to better orientation in the tool.
All results are integrated into the agile process of further developing the content and technology of the monitoring tool in the project. Further tests are also planned for the future.
Participant usability test
The approach is good. However, improvements are still needed to make it more user-friendly and intuitive.
In 2021/2022, 29 researchers from publicly funded research projects of the funding bodies BLE and DLR participated in usability tests and assessed the monitoring tool in approx. 80 test sessions of 75 minutes duration each. In parallel, a further eight researchers used the SynSICRIS monitoring tool to describe a currently funded project (4 consortia) as a whole and to test the tool, accompanied by the SynSICRIS team.
In 2023, 14 researchers from publicly funded research projects of the funding bodies BLE and DLR participated in usability tests in 25 test sessions of 75 minutes each, including 3 people who had already participated in the previous year. The assessments of 9 staff members of the funding bodies BLE and DLR were included in 19 test sessions of 75 minutes duration. In addition, nine researchers entered data from four consortia and tested the SynSICRIS monitoring tool overall.
Participant usability test
Very cool that in one place after the end of the project it can be shown what is developing.
Participant usability test
Very helpful to work on and monitor projects, provides good structure and overview, simplifies processes such as the interim report for a project.