Some questions to ask
- What is the framework or pathway by which our activities will lead to improvement in desired outcomes?
- What questions do we hope to answer with the evaluation?
- What methods will we use to answer the evaluation questions, and to monitor and evaluate the effort?
- How will we make sense of the data? How will stakeholders, including those most affected, be involved in reviewing the data and contributing to what it means?
- How will we use the evaluation information for celebration and improvement?
- How will we use an equity lens to see whether the effects of the effort benefit all people, including vulnerable groups that have had worse outcomes?
Some recommended actions
- __ Describe the program or initiative’s framework or logic model (pathway by which our activities will lead to intended outcomes). Include information about:
- Purpose or mission (i.e., the problem or goal to which the program, effort, or initiative is addressed).
- Context or conditions (i.e., the situation in which the effort will take place; factors that may affect outcomes)
- Inputs: Resources and barriers (i.e., resources may include time, talent, equipment, information, money, etc.; barriers may include history of conflict, environmental factors, economic conditions, etc.)
- Activities or interventions (i.e., what the initiative will do to effect change and improvement) (e.g., provide information and enhancing skills; enhance services and support; modify access, barriers and opportunities; change the consequences; modify policies and broader systems)
- Outputs (i.e., direct evidence of having performed the activities) (e.g., targeted actions to those most affected; number of services provided)
- Intended effects or outcomes
- Short-term (e.g., community and systems changes)
- Intermediate (e.g., changes in behavior)
- Longer-term (e.g., improvement in outcomes; reduced inequities)
- __ Focus the evaluation questions and methods. Include:
- Evaluation questions – What information about the effort is important to stakeholders? Some illustrative evaluation questions:
- How well was the program or initiative planned and implemented?
- How well has the program met its stated objectives?
- How much/what kind of difference has the effort made for those intended to benefit?
- How much/what kind of difference has the program or initiative made in the community as a whole? For those groups experiencing poorer outcomes?
- Methods – What methods will be used to evaluate the effects of the program or initiative? Match the methods to the evaluation questions. Some methods include:
- Surveys about satisfaction and importance of the initiative
- Goal attainment reports
- Behavioral surveys
- Interviews with key participants
- Archival records
- Observations
- Self-reporting, logs, or diaries
- Monitoring system and analysis of contribution of the initiative
- Community-level indicators of impact (e.g., rates of obesity)
- Case studies and experiments
- __ Indicators of success. What indicators will be used to judge the success of the program or initiative? Match the indicators to the evaluation questions. Potential indicators include:
- Program outputs
- Participation rates
- Levels of satisfaction
- Changes in behavior
- Community or system changes
- Improvements in community-level indicators
- Evaluation questions – What information about the effort is important to stakeholders? Some illustrative evaluation questions:
- __ Outline and implement the evaluation plan.
- Indicate how you will involve all stakeholders (i.e., including those most affected) in identifying indicators of success, documenting evidence of progress, and sense making about the overall initiative and how it can be improved.
- Indicate how you will track implementation of the intervention.
- Indicate how you will assess ongoing changes in specific objectives (e.g., changes in behavior).
- Indicate how you will assess ongoing changes in specific outcomes (e.g., community-level indicators; inequities).
- Consider the ethical implications of the initiative.
- __ Make sense of the data and justify conclusions. This includes engaging stakeholders (including those most affected) in:
- Sensemaking and interpretation – How will we engage those responsible, and those most affected, in making sense of the data (i.e., what we are seeing, what it means)? How will we use the information to help answer the evaluation questions?
- Judgments – statements of worth or merit of the initiative. How will we communicate what the findings suggest about the value added by the effort?
- Recommendations – How will we identify recommendations based on the results of the evaluation?
- __ Use the information from the evaluation, including to:
- Celebrate accomplishments
- Make adjustments in activities and interventions
- Communicate lessons learned to stakeholders and relevant audiences
Examples of evaluation
- Participatory Evaluation of State and Local Efforts to Prevent Sexual and Domestic Violence in Kansas
- Monitoring and evaluating the Covid-19 response and recovery in Lawrence-Douglas County
- Documenting the collective impact of a local coalition in Sedgwick County
Resources for monitoring and evaluation
Tools from the Community Tool Box
Toolkit: Evaluating the Initiative
Some Methods for Evaluating Comprehensive Community Initiatives
Tools from KDHE and Other Sources
CDC Planning, Implementing, and Evaluating an Intervention
Prevention Institute – Measurement and Evaluation for Health Equity