Develop an Evaluation Plan
The project evaluation flows directly from the strategic plan. To gauge your project’s effectiveness, you should have already worked with your partners to add reasonable measurements to the strategic plan’s activities and goals. (See Devise a Written Plan in step 3 of this guide.)
Although there are numerous methods for developing a project evaluation, the three pilot sites used a simple output- and outcome-based plan to collect information on the following:
Outputs are simply a number or count of services provided. Each site tracked and reported the following information:
- Numbers of training sessions and presentations conducted quarterly.
- Numbers of individuals with disabilities and professionals attending the training sessions and presentations.
- Types of groups trained or educated:
- Own agency staff/volunteers.
- Criminal justice professionals.
- Disability service providers.
- Victim assistance providers.
- Persons with disabilities.
- Number of unduplicated crime victims with disabilities the agency served each quarter. (Note: This will lead to an unduplicated count through the course of the entire project.)
- Number of crime victims by type of disability the agency served each quarter (indicate individuals who self-identify as having multiple disabilities by documenting them in multiple categories):
- Physical disability.
- Cognitive disability.
- Deaf or hard of hearing.
- Blind or low vision.
- Acquired disability.
- Mental illness label.
- Age-related disability.
- Other disability.
- Number of crimes against persons with disabilities reported to project partners each quarter.
Outcomes measure the actual impact that the program makes. Outcomes tend to be expressed as percentages. Types of outcomes may include changed attitudes or values, changed behaviors, new knowledge, or new skills. Sites reported on the following outcomes:
- Percentage of individuals with disabilities who were educated on the issue and who demonstrated an increased understanding of the issue through pre- and post-tests.
- Percentage of service providers and other stakeholders who were trained on the issue who reported an increased understanding of the issue through pre- and post-tests.
For this project, 85.5 percent of persons with disabilities and 85 percent of professionals reported or demonstrated an increased understanding of the issues through these tests, which were given immediately before and after each educational session.
In collecting data on your outputs and outcomes—
- Identify the types of information you need to gather. You may want to know the number of disability service providers you trained, the number of crime victims with disabilities who are receiving services because they were reached through your outreach efforts, the number of victims reporting satisfaction with services, and so forth.
- Determine the method of data collection that will be used to measure the effectiveness of outcome activities (e.g., pre- and post-class surveys, class evaluation form).
- Identify the staff person who will be collecting and compiling the data and indicate how often data will be collected.
- Decide whether the information will be reviewed monthly or quarterly.
- Identify who will review data and who will be involved in making changes to services based on information from the evaluation.