Analyzing quantitative data from program beneficiaries
Abstract
To satisfy performance measurement requirements, programs typically use quantitative data to gauge the degree to which output, intermediate outcome, and end outcome targets have been met. When logs, surveys, tests, or structured interviews are used to collect responses and other information from program beneficiaries to assess program outcomes, and responses are gathered using “check box” options, these responses should be changed to numerical values. This effective practice, submitted by Project STAR in June 2006, helps programs learn to convert and examine this important information.Issue
Performance measurement is useful both as a program management tool and a means by which to communicate program impact to funders, but gathering data and transforming it into meaningful language can be challenging.Action
The following six steps can help programs effectively analyze the quantitative data obtained from program beneficiaries and report on specified targets.
- Start with a plan: Determine what type of data should be reported.
- A count is the number of people responding.
- A frequency is the number of people who responded "a, b, or c."
- A percent distribution is the proportion of people who answered "a, b, or c."
- A mean is an average score for those who answered.
- A change in scores is the difference between pre-test and post-test.
- Collect and clean data.
Keep all of the forms completed and received for each survey, test, or structured interview in one place, removing forms that are substantially incomplete or when responses do not make any sense. Keep a count of the forms removed and reasons for removal, in case you want to report this. Assign an ID number to each form to help keep track of forms that have been processed.
- Decide on a coding system (by hand, on a spreadsheet, or in a database).
Beneficiary responses can be tallied directly onto a blank copy of the survey if fewer than 50 have been completed and returned. For more than 50, consider using a spreadsheet program. "Check box" responses should be converted to a number. Example: Very Satisfied = 4, Satisfied = 3, Unsatisfied = 2, Very Unsatisfied = 1.
- Tabulate the responses to each question by hand or computer.
After tallying responses for each item on completed surveys, the number of times each response was selected should be counted, either by hand or computer. If the anticipated target describes change in terms of a percentage of beneficiaries, then compute the percentage of responses in each category (e.g., 80 percent of beneficiaries reported that they were very satisfied with the services provided).
- Transfer the information to a copy of the original instrument.
Record the frequencies or percentages for each item.
For example: "How satisfied were you with the services provided?"
- 80 percent, Very Satisfied
- 15 percent, Satisfied
- 3 percent, Unsatisfied
- 2 percent, Very Unsatisfied
Having all the results of the analysis on one instrument will help when reporting on whether the anticipated targets set in the performance measure have been met.
- Check the data analysis plan created in Step 1 to see if the planned analysis was followed.
Context
Performance measurement is conducted by all types of programs funded by the Corporation for National and Community Service (AmeriCorps State and National, AmeriCorps*VISTA, Learn and Serve America, and Senior Corps). It can be used to measure outcomes for beneficiaries and communities served, participants and volunteers, and organizations providing the service. Instruments used to measure outcomes in performance measurement include (but are not limited to) logs, questionnaires, knowledge and skills tests, and observational record forms.Outcome
Effective data analysis as a part of performance measurement provides information that is useful for managing programs. It allows programs to track progress toward larger goals, and to identify program strengths and possible areas for improvement. If program staff/members are actively involved in performance measurement activities, tracking of outputs and outcomes helps to ensure program accountability, and helps to improve services and beneficiary outcomes.Evidence
The results of the quantitative analysis should be checked against the anticipated targets set in the performance measure. Results can be reported in Corporation progress reports and to other stakeholders in a variety of other formats (for example, newsletters or presentations). Do the data collected provide evidence that the intermediate outcome and end outcome targets have been achieved? If the targets have been met, the results should provide evidence that the services contributed to achieving the desired results. If targets have not been met, the results should help program staff raise questions directed toward improving the program.Posted On
June 14, 2006For More Information