Program Evaluation

Program Evaluation. Lecture Overview. Program evaluation and program development Logic of program evaluation (Program theory) Four-Step Model Comprehensive Quality Programming Empowerment Evaluation Group Activity. Program Development.

Share Presentation

Embed Code

Link

Download Presentation

jcoombs

jcoombs + Follow

Download Presentation

Program Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

Presentation Transcript

Program Evaluation
Lecture Overview • Program evaluation and program development • Logic of program evaluation (Program theory) • Four-Step Model • Comprehensive Quality Programming • Empowerment Evaluation • Group Activity
Program Development • refers to the planning and implementation of a program determining its purposes and setting goals and outcomes, evaluating program process, and evaluating attainment of desired program outcomes and wider impacts.
History of Program Evaluation • 1930’s human service field grew (post depression) • WWII programs were launched to meet the needs for urban development and housing, technological and cultural education, occupation training, and preventive health activities. • By the end of the 1950’s program evaluation was commonplace • 1960’s increased papers on evaluation research • 1970’s, evaluation was a distinct field • 1980’s benefits and costs of programs • 1990’s new social problems and booming economy
Program Evaluation • Refers to the collection of evidence to judge the effectiveness of a community programs or other community interventions (com psych definition) • The use of social research procedures to systematically investigate the effectiveness of social intervention programs (research definition).
Project Drug Abuse Resistance Education D.A.R.E. • Most popular school based intervention • Little evidence on its effectiveness • Limited positive influence on psychological factors (self-esteem) • How can this lack of measurable outcomes in a popular intervention be explained?
Reasons Why Programs May Be Unsuccessful • Theory Failure: concerns program theory which is the rationale for why a particular intervention is considered appropriate for a particular problem with a specific target population • Implementation Failure: concerns quality of program implementation (ie: program may not be generalizable)
Effects do not occur when……. • The underlying assumptions of program theory are incorrect. • The program, even if implemented well, doesn’t affect the variables specified by program theory. • Or the activity or program is not implemented adequately
Logic Model • Graphic representation of the program. • Shows the logical connections between the conditions that contribute to the need for a program in a community, the activities aimed at addressing these conditions, and the outcomes and impacts expected to result from the activities.
Evaluation “Terminology” • Stakeholders • Fiduciary • Funding sources • Need Assessment • Evaluation Instruments
Four Step Model of Program Evaluation (Lindsay & Wandersman, 1991) • Step 1: Identify Goals and Desired Outcomes • Step 2: Process Evaluation • Step 3 Outcome Evaluation • Step 4: Impact Evaluation:
STEP 1) Identifying Goals and Desired Outcomes • What is your project striving for? • Goals set a framework for specific outcomes. Program Developers MUST describe: • The primary goals of the program • Target population • Outcomes desired
STEP 2) Process Evaluation • Monitoring program activities helps organize program efforts • Provides information to help manage the program accountability that the program is conducting what was done and who was reached • After an evaluation of outcomes and impacts , the evaluation can provide information about why the program worked or did not work • Can help decide whether or not you are ready to assess the effects of your program • What were the intended and actual activities of the program/ Did you learn anything.
STEP 3) Outcome/Impact Evaluation • Assess the immediate effects of a program “the bottom line” • Looks at the desired outcomes defined in Step 1 and seeks evidence regarding the extent to which those outcomes were achieved • Should be closely linked to goals, but more specific
STEP 4) Impact Evaluation • Concerned with the ultimate effects of the program • Impacts are ultimate or longer-term effects of the program
Comprehensive Quality Programming: 9 Strategies • Why is the intervention or program needed? • What are the program’s goals, target populations and desired outcomes? • How does your program use the scientific knowledge and best practice of what works? • How will this new program fit in with existing programs • How will you carry out the program • How well was the program carried out • How did the program work? • What can you do to improve the program the next time you do it? • If the program or parts of the program were effective, what are you doing to continue institutionalize the program
Why is the intervention or program needed? What are the program’s goals, target populations and desired outcomes? How does your program use the scientific knowledge and best practice of what works? How will this new program fit in with existing programs How will you carry out the program How well was the program carried out How did the program work? What can you do to improve the program the next time you do it? If the program or parts of the program were effective, what are you doing to continue institutionalize the program Needs and assets assessments Identifying goals and desired outcomes Review research literature and best practices Community feedback and planning Planning, implementation Process evaluation Outcome and impact evaluations Continuous program improvement Institutionalization strategies Accountability Questions & Corresponding CQP Strategies
Empowerment Evaluation • Program planners and developers learn the basics of program evaluation in ongoing program improvement. • Help program developers and staff to achieve their program goals, by providing the with the tools for assessing and improving the planning, implementation, and results of their program.
Limitations of Evaluation • Use of an outside evaluator • Evaluation feedback is provided at the end of the program • Evaluation findings are perceived as being too complex
Conclusions • Program evaluation is necessary to find out whether a program is working • Evaluation can be used for program planning and program implementation • As society becomes more concerned about accountability and results , evaluation will become more necessary.
Group Activity • Apply 4 step model in terms of your group project. • Answer CQP questions

Related
More by User

Program Evaluation

Program Evaluation. It’s Not Just for OMB Anymore…. Questions. Think about your state What has your public investment been in workforce development in your state in the last five years? How many people participated in what services?

589 views • 31 slides

Program Evaluation

Program Evaluation. A systematic effort to describe the status of a program Extent to which program objectives achieved. Uses of Health Program Evaluation. Insight - Needs - Barriers - Activities Improvement - Social mobilization - Inter- sectoral Coordination - Implementation

682 views • 45 slides

Program Evaluation

Program Evaluation. How Do I Show This Works? Paul F. Cook, PhD UCDHSC School of Nursing. Why Evaluate?. We have to (state, federal, or contract reg.s) In order to compete (JCAHO, NCQA, URAC) It helps us manage staff It helps us manage programs It helps us maintain high-quality programs

709 views • 46 slides

Program evaluation

Program evaluation. Sebastian Uijtdehaage bas@mednet.ucla.edu 310.794.9009. Why do you evaluate?. Five reasons of evaluating. FORMATIVE informs program development SUMMATIVE demonstrate program’s success and impact UNDERSTANDING how and why did program work?

754 views • 26 slides

Program Evaluation

Program Evaluation. Using qualitative & qualitative methods . Program evaluations measure:. Program effectiveness, efficiency, quality, and participant satisfaction with the program. Program evaluation can also measure: . How or why a program is effective or is not effective.

464 views • 21 slides

Program Evaluation

2008 NCLB School Choice Leadership Summit. Program Evaluation. Cristy Davidson, Joe Sterensis, and Anke Toth Wednesday, June 25, 2008. Agenda. Legislative Requirements Data Reporting Ensure Quality of Data SES Evaluation Ensure Program Effectiveness . Legislative Requirements.

494 views • 36 slides

Program Evaluation

Program Evaluation. Just enough knowledge to be dangerous Belinda De La Rosa and Megan Mustafoff Off. of the Dean of Students and Counseling Ctr. What to expect. Definitions Know your Audience & Population Goals Methodologies Report Writing

344 views • 26 slides

Program Evaluation

Program Evaluation. Mark Spolidoro , University of Massachusetts Boston. What is Evaluation?. Program evaluation is the systematic assessment of the processes and outcomes of a program with the intent of furthering its development and improvement. *

618 views • 15 slides

Program Evaluation

Program Evaluation. 1. Clinically relevant? Do the tools give meaningful results? 2. Reproducible? Do the tools produce the same results when administered repeatedly for stable characteristics? Do the tools produce the same results when administered by different staff members?.

142 views • 2 slides

Program Evaluation

Program Evaluation. Maria Lisak Management Advisor. Economic Governance of the Commons. Ostrom identifies eight "design principles" of stable local common pool resource management: [4] Clearly defined boundaries (effective exclusion of external unentitled parties);

152 views • 3 slides

Program Evaluation

Program Evaluation. Evaluation Defined. Green and Kreuter ( 1999) broad definition: “ comparison of an object of interest against a standard of acceptability ” Weiss (1998) more targeted:

442 views • 33 slides

Program Evaluation

Program Evaluation. Amanda Valbuena Evaluator, CFEP. What is Program Evaluation?. What words come to mind when you think of Program Evaluation? What does Evaluation mean?. Defining Program Evaluation.

294 views • 8 slides

PROGRAM EVALUATION

PROGRAM EVALUATION. Produced by Dr. James J. Kirk Professor of HRD Western Carolina University. What You Will Learn. The definition of “program evaluation” Various terms used used by program evaluators Common reasons for conducting program evaluations Selected types of evaluations.

877 views • 53 slides

Program Evaluation

Program Evaluation. Michelle Mohr Carney, Ph.D. Institute for Nonprofit Organizations University of Georgia. Purpose/Objectives. Increase your knowledge of processes involved in program evaluation Provide information and resources to help you design and conduct your own program evaluation.

3.24k views • 65 slides

Program Evaluation

Program Evaluation. Program evaluation Methodological techniques of the social sciences social policy public welfare administration. Evaluation Formative – help form the program Ongoing assessment to improve implementation Outcome – after the fact. Needs Assessment.

753 views • 46 slides

Program Evaluation

Program Evaluation. Debra Spielmaker, PhD Utah State University School of Applied Sciences, Technology & Education - Graduate Program Advisor USDA-NIFA, Agriculture in the Classroom - Project Director.

328 views • 18 slides

PROGRAM EVALUATION

PROGRAM EVALUATION. Carol Davis. Purpose of Evaluation. The purpose of evaluation is to understand whether or not the program is achieving the intended outcomes. Helps identify what works and does not work. Today’s Evaluation Discussion. PEEC Outcomes (Indigenous Evaluation)

765 views • 38 slides

Evaluation Program

Evaluation Program. Customer Pre-market Program. Program Purpose Customer experience vs. print and media Build awareness Demonstrated performance Hearsay Customer testimonial A satisfied customer is our best advertisement. Target Customer Profile. Industry influencer

316 views • 13 slides

Program Evaluation

Program Evaluation. August 29, 2013. Essential Questions. What is the connection between formative program evaluation and summative goal evaluation? How often should I use formative assessment? What process should I use for program evaluation?

298 views • 10 slides

Program Evaluation

954 views • 21 slides

Program Evaluation

330 views • 26 slides

Program Evaluation

Program Evaluation. Presentation by: Kathleen Tebb , PhD Assistant Professor of Pediatrics Division of Adolescent Medicine University of California, San Francisco 11.12.09. Introduction. Eval. Research Examples : Influence of formula gift packs on breastfeeding cessation

490 views • 40 slides