Practical guide 6: Evaluating research findings and impacts

In line with CHILD's intentions regarding inclusive research practices, this guide on inclusive research aims to provide researchers with suggestions on how to engage children and families in the evaluation of research findings and their impact. Evaluating research findings is an important but sometimes forgotten stage of the research process.

This process involves comparing the results achieved in the project related to the predetermined objectives. It also involves drawing conclusions and evaluating the progression of the activity, what it can mean for children and families as well as other participants. The process of measuring and evaluating different elements of a research project should be ongoing and provide information that allows the research team to evaluate, modify, improve and report on the value of their efforts in the short and longer term.

It is beneficial to develop an evaluation plan from the beginning of the research project, in order to document progress. This plan can be reviewed at regular research team meetings. At a later stage, the impact of the research can be evaluated in relation to everyday practices, policies, processes or services.

Evaluation plans

These plans (1) describe how project results will be measured, (2) document findings to enable transparency and accountability, (3) guide implementation, and (4) store data. When developing an evaluation plan, it should be useful (assist with decision-making, resource allocation), feasible (realistic and practical), ethically sound.

Key points to discuss:

• What are the expected outcomes of the research project?
• What specific issues are expected to lead to change?
• What are the indicators of change and how are they measured?
• What does a successful project look like?
• How will evaluation data be collected and analyzed?
• What impact will the research have on those it affects?

Guide to creating an evaluation plan

1. Engaging the research beneficiaries and other stakeholders

Continuous contact and follow-up with participants are important steps. It is important to understand different perspectives and integrate them into the plan. The authentic needs of those affected by the research and how they will be affected must be articulated. What will be improved? Will it change? And how?

It is important to clarify the expectations, involvement and responsibilities of the children; if necessary, a family member or other caregiver can support the child in this.

2. Creating an evaluation plan

It is important to identify what resources and assets are available to contribute to the expected outcome of the project.

Project resources refer to the assets that team members and partners bring to the research project, such as funding, (e.g. access to databases, different activities, such as schools, health clinics, associations, etc.

Research activities are the activities carried out within the research project, including training, communication and networking.

Outputs of the project can be products, such as popular science publications, training packages, knowledge passed on through, for example, PhD students or trained teachers.

The usefulness of the research refers, for example, to how many students were trained, professional development of teachers and health professionals. Who and how many 'used' products produced by the project, such as training materials?

Research application and outcomes refer to what happened as a result of project outputs and activities. These can include changes in behavior, attitudes, policies, school practices and funding systems.

It can refer to the value and benefits of the research. These may include health gains, productivity gains, increased school or employment rates, and the economic and social value/benefit of these outcomes. Each project will have its own way of specifying likely research impacts.

Consensus should be reached among the research team based on input from advisory groups related to the objectives of the project, what is expected to change and what impact this will have.

3. Evaluate the methods used

Research participants can provide researchers with important perspectives and critiques on the measurement of research results and their usefulness, from the perspective of lived experience. Research participants can also provide different perspectives on methods of data collection that are appropriate within the group of children being studied and interpretation of findings that one is trying to understand.

4. Assigning responsibilities for implementation

The roles and responsibilities of the different actors must be described for accountability and reporting.

5. Setting objectives

Objectives set with the participants in the project allow everyone to understand what the plan is expected to achieve. These need to be realistic, and flexible in terms of changes in direction over time. It may be necessary to react to new events, opportunities and problems over time.

6. Fefine reporting, dissemination and exploitation of results

Those involved in the children's project must ensure the exploitation of the results of the research project. Preliminary results can be presented at strategically timed user meetings/workshops, so that they are tailored to the interests and needs of specific stakeholders. It is important to identify appropriate venues for information and dissemination methods, taking into account the views of the groups with whom the research is conducted and the co-researchers.

Measuring the impact and benefits of research

Examples of this are indicators that show:

• Change and increased understanding of the research question at different stages of life
• Improvement in the health and well-being of the children involved in the study
• Contribution to the knowledge base for the benefit of children and their families, professionals in different settings.
• Change in clinical, diagnostic, educational and research practice
• Implementation of policy and legislative changes in areas such as education, employment, health, disability.
• Economic benefits
• Scope of interdisciplinary research
• Patterns of collaboration
• Scientific impact
• Research productivity
• Publication practices
• Citation impact (number of times researchers are cited in other publications).

Visibility of project outcomes in different activities

• Build relationships and networks with groups with similar characteristics to those the research concerns and is conducted with.
• Supporting individuals with knowledge of the subject and promoting their ideas, networks and relationships.
• Ensure that communication channels for advocacy are active before, during and after a research project.
• Develop expertise and reliability in the research area available.
• Address areas of policy interest.
• Get policy makers involved in research.
• Join relevant committees/networks and make sure your findings are included when decisions are made.
• Strive to publish research results as early as possible.
• Present research results.