Providing outcome indicators for reporting on environmental programs
Abstract
The Corporation for National and Community Service uses information derived from various sources to evaluate and report on its performance with the objective of improving the lives of recipients of assistance provided by members, volunteers, and sub-grantees. The Urban Institute Report on Performance Measurement (July 2002) for the Corporation for National and Community Service provides a critique of current measurements and recommends ways to enhance reporting. This effective practice, excerpted with permission from the Urban Institute's Report, shares standardized outcome indicators for environmental service agencies and organizations. Read the report, Outcome Indicators and Outcome Management.Issue
Using standardized outcome indicators for program management and reporting to enhance credibility, frequency, and value.Action
- The Urban Institute Report on Performance Measurement (2002) for the Corporation for National and Community Service provides a critique of current program measurements and recommends ways to enhance reporting. Although the various divisions within AmeriCorps programs have different specifications for their service, they all follow the same general service goals. Consequently, these outcome indicators are common across programs (AmeriCorps, National Civilian Community Corps, AmeriCorps*VISTA) despite differences in activities and in the means by which they are accomplished.
- The outcome indicators have been selected to be of value to both project sponsors (for internal management purposes) and the Corporation for National and Community Service (for policy development and aggregate reporting purposes).
- The following represent recommended outcome indicators for the major issue area of "Environment and neighborhood/community enhancement" in which grants from the Corporation for National and Community Service are primarily used:
1. Number of units (i.e. miles, acres, feet, tons) of environmental or community (a) restoration, (b) refurbishment, (c) clean-up, (d) or renewal accomplished through projects in which AmeriCorps members or volunteers provided a significant amount of service. [Intermediate outcome]
Source(s) for future collection of data for this indicator include:Aggregation of counts of unit of measure -- appropriate to the particular activity -- prepared and submitted at the project level. Appropriate units of measure might include land, trails, irrigation systems, streambeds, riverbeds, riverbanks, shoreline, footbridges, toxic vegetation, trees, parks, gardens, fencing, wildlife habitats, historical and community buildings that are restored, refurbished, cleaned up, or renewed (including trees moved or planted).
2. Percent of beneficiaries who can be identified as receiving environmental, community, or disaster assistance, or environmental educational programs in which AmeriCorps members or volunteers provided a significant amount of service, who report quality of life benefits as a consequence of this service during the reporting period.[End outcome]
Source(s) for future collection of data for this indicator include:Provider-administered survey of all, or a sample of those benefiting from environmental, community, or disaster assistance programs, conducted at the time assistance is provided or for some (to be determined) days following such assistance.
Note: For this outcome indicator, as well as for others that may not involve direct, person-to-person service, providers need to develop explicit definitions and procedures for identifying beneficiaries and report these along with the results.
As AmeriCorps programs transition from outcome evaluation to performance measurement, the Corporation has selected terminology to describe this new method to determine program results. Other funding agencies may use different terms to describe these same performance measurement concepts.
Additionally, as the Corporation works to ensure measurement standardization, program directors might review outcome indicators used by other Federal agencies. Since the public service efforts of AmeriCorps members and volunteers are often similar to beneficiary assistance provided by agencies such as the U.S. Departments of Education, Health and Human Services, Housing and Urban Development, and Labor, their outcome indicators might help programs gauge their own effectiveness.
Context
A major objective of AmeriCorps is to improve the lives of the direct recipients or beneficiaries of the assistance provided by members, volunteers, and sub-grantees. The Corporation for National and Community Service uses information derived from various sources to evaluate and report on its performance with respect to this objective, including:- Project/program accomplishments data assembled and submitted by sponsoring agencies/non-profit organizations
- Surveys of sponsoring agencies'/non-profit organizations' ratings of project success
- Surveys of corporation members' ratings of project success
- Surveys of "community representatives" ratings of project success
- Reports of ad-hoc program evaluation studies
Although these methods provide useful information, they can be limited with respect to quality and credibility. Given the limitations of the indicators currently being used to assess the improvement in the lives of beneficiaries, and the need to enhance their credibility, frequency, and value, the Urban Institute has recommended a standardized set of outcome indicators.
The Urban Institute is a nonpartisan economic and social policy research organization, concentrating on issues important to the nation. Quantitative modeling is a core competency of the Urban Institute, as is revamping research methodologies to broaden their application. Frequently the Institute collaborates with outside researchers and other institutions or works in partnership with federal, state, and local agencies.
Citation
Hatry, Harry P., et al. Outcome Indicators and Outcome Management. Washington, DC: The Urban Institute, Final Report, July 15, 2002.
Submitted to the Corporation for National and Community Service.
Contract No. GS-23F-8198HUI No. 07112-004-00
The Urban Institute is a nonprofit, nonpartisan policy research and educational organization that examines the social, economic, and governance problems facing the nation. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
Outcome
According to the Urban Institute, if outcome indicators were at least roughly standardized over many, if not all, projects, the annual findings could serve both local project management purposes, such as identifying if, and where, problems exist; and be converted into index scores for reporting to the Corporation for National and Community Service and aggregating across projects.
The more such indicators are standardized across projects, the more useful they will be for purposes of comparison and aggregation.
Posted On
November 4, 2002For More Information
Resources
From The Resource Center library:
Outcome Indicators and Outcome Management
Item number: 1819
Source Documents
Related Practices
Related sites
The Corporation for National and Community Service