Abstract

Effective evaluation not only satisfies the requirements of funders, but is an essential practice that is integrated into daily program implementation and management. Public Allies models this type of evaluation approach, resulting in successful outcomes for members, service recipients, and partners. This effective practice is excerpted from the resource, Web-Based Evaluation Tools: Tracking Multiple Impacts, shared at the AmeriCorps Direct Best Practices Conference in Nashville, Tennessee in April 2005.

Back to top

Issue

Creating performance measurement and evaluation that is meaningful to both program participants and funders.

Back to top

Action

According to the Public Allies' document, Web-Based Evaluation Tools: Tracking Multiple Impacts, effective evaluation practices include:

Continuous learning processes:

• Individual Development Plans and Coaching: Each month members set personal and professional goals. (Helps answer the questions: "How can I push myself towards greater performance? How do I wish to be supported?")
• 360-degree Evaluations: Performed by fellow Allies, Partner Organization supervisors, and Public Allies Program Managers followed by an interactive group feedback circle. ("How do others view my performance and leadership? In what areas do others see me needing focus?")
• Critical Reflection Exercises: Allies reflect and share their experiences, lessons and connections to larger social issues. ("Why am I called to serve anyway? How does my service connect with my belief in change?")
• Presentations of Learning: These happen at the end of the year, demonstrating service and leadership outcomes to peers, supervisors and supporters. ("What did my service experience teach me?" Where did I make the most strides?")

The Personal Impact and Service Documentation (PISD) is a personalized tool that prompts members to track the impact people are having on a bi-weekly basis. Both the Partner Organization Supervisor and the PA Program Manager approve the PISD. It tracks:

• Profiles and Retention: Ally demographics, Partner Organization demographics, and Ally retention information
• Outcomes and Objectives: Tracks progress and achievement of up to two measurable service outcomes and three service objectives for each Ally.
• Measuring Outputs: Numbers of people served, volunteers engaged, target populations served, linkages and collaborations among organizations
• 360 Tool: Compiles self, peer and supervisor reviews on Allies' practice of leadership values
• Stories: Captures stories of success and learning from Allies

Web-based surveys: There are many web-based survey tools on the market that can be used to easily gather data and information for a program. Surveys are used to gather Allies, Partner Organizations and Alumni satisfaction:

• Allies: Satisfaction with the program, learning and development, civic engagement and social capital, and lessons for improving the program.
• Partner Organizations: Satisfaction with service, capacity building gains, sustainability of service, and lessons for Public Allies.
• Alumni: Satisfaction with experience, current education and employment status, civic engagement and social capital, and lessons for Public Allies.

Back to top

Context

Public Allies evaluates for:

• Program and Organizational Learning and Improvement
• Member and Partner Learning
• Grant Reporting Requirements
• Communicating the Impact of Their Mission

Evaluation Approach:

• Evaluation is not an external process, but an internal practice that is integrated into the daily work of Allies and staff.
• Emerging leaders should develop a habit of mind that is inquisitive about impact and change and they should hold themselves accountable for results. Allies should know how to set and measure outcomes, track outputs, and report on their results.
• Public Allies internal staff management system models the Ally evaluation tools to track goal progress, support continuous learning, and promote leadership development for staff.

Back to top

Citation

McKinney, David and Paul Schmitz. Web-Based Evaluation Tools: Tracking Multiple Impacts. Public Allies. April 18, 2005.

Back to top

Outcome

Program results are measured based on program standards and program outcomes. The results are used to capture lessons learned, identify emerging innovation, and provide sound technical support.Information is:

• Collected in real time
• Automatically tabulated and aggregated
• More detailed and higher quality
• Easier to search, compare and analyze
• Woven into weekly activities of staff and participants

Back to top

Evidence

Public Allies reports statistics on members, service, partners and return on investment:

Members:

• Allies graduate: 85 to 90 percent of Allies graduate each year.
• Allies Gain Important Skills: The skills Allies report gaining the most are interpersonal communication, teamwork, practicing inclusion and diversity, project planning and evaluation.
Eighty-four percent of alumni report that they are adequately prepared for future educational and career goals as a result of participation in the program.
• Allies Remain in Public Life: Over eighty percent continue careers in the nonprofit and public sectors.
• Allies are Active Citizens: They are more than twice as likely as peers to volunteer (78 percent), mentor a young person (64 percent), serve on a board (36 percent), attend political meetings (68 percent), protest (56 percent), and engage in other civic activities.

Service:

• Allies Achieve Outcomes: Allies achieved 92 percent of measurable outcomes. Ninety-six percent of Partner Organizations reported that Allies met or exceeded expectations.
• Allies Serve Thousands: 202 Allies served the educational, economic security, health and other needs of 166,893 people, about two-thirds children and youth (2005 statistic).
• Allies Generate Sustainable Volunteers: 202 Allies recruited 8,505 volunteers who recruited 70,255 hours of volunteer time (41 FTEs) benefiting 59,437 people. Sixty-one percent of volunteers came back four or more times to serve.
• Allies Service is Sustainable: Eighty-seven percent of Partner Organizations reported that they would sustain the services enhanced and expanded by Allies.

Partners:

• Seventy-four percent measurably increased their overall service capacity
• Seventy-three percent benefited from collaborations
• Sixty-seven percent from increased volunteerism
• Sixty-one percent from more diverse perspectives
• Fifty-eight percent improved supervisory skills
• Thirty-four percent improved evaluation tools and processes
• Seventy-nine percent reported that these capacity benefits will be sustained

Return on Investment:

• For each dollar invested by Partner Organizations that contribute two-third of each Ally's stipend, they received $3.41 in service value.
• Organizations save $27,000 each, on average, from the cost of recruiting, hiring and training staff, totaling over $5.4 million a year.

Back to top

Posted On

June 15, 2005

Back to top

For More Information

Back to top

Source Documents

Related Practices

• Developing a program evaluation system
• Assigning importance to member evaluation

Back to top

Related sites

Topic Areas

• Evaluation
• Reflection
• Retention
• Supervision
• Capacity building
• Nonprofit management
• Program evaluation
• Technology
• AmeriCorps
• Program management
• POL and HON

Back to top