Performance Evaluation: Proven Approaches for Improving Program and Organizational Performance

Section 1: Introduction to Evaluation

Chapter 1

Introduction to Evaluation

Key Points

Decisions, Decisions…

A Brief Overview of Evaluation History

Evaluation: Purpose and Definition

Performance Improvement: A Conceptual Framework

Making Evaluation Happen:  Ensuring Stakeholders Buy-in

The Evaluator: A job or a role?

The Relationship to Other Investigative Processes: Evaluation, Needs Assessment, and Scientific Research

When Does Evaluation Occur?

General Evaluation Orientations

Challenges Faced By Evaluators

Ensuring Commitment

Benefits of Evaluation

Basic Definitions

Chapter 2 Principles of Performance-Based Evaluation

Key Points

What are we seeking?

Principle 1: Evaluation—good evaluation—is based on asking the right questions

Evaluation Question Strings

Developing Questions that Guide Valid and Useful Evaluation

Principle 2: Evaluation of process (means) is a function of obtained results (ends)

Getting Ends and Means Identified and Linked

Principle 3: Goals and objectives of organizations should be based on valid needs

Principle 4: Derive valid needs using a top-down approach: rolling down from external impact and value added to the results and methods used within the organization

Principle 5: Every organization should aim for best that society can attain. Evaluation standards must be linked to a useful and real strategic vision

Principle 6: The set of evaluation questions drives the evaluation study

Section 2 Models of Evaluation

Chapter 3 Overview of Existing Evaluation Mode

Key Points

Overview of Classic Evaluation Models

Selected Evaluation Models

Which model should you select?

Conceptualizing a Useful Evaluation That Fits Your Situation

Reflection Questions

Chapter 4 Kirkpatrick’s Four Levels of Evaluation

Key Points

Overview of the Model and Relevant Research

Description of Kirkpatrick’s Levels

Criticisms of the Model

Strengths and limitations

Application Sample

Additional Application

Reflection Questions

Chapter 5 Phillips Return on Investment

Key Points

Overview of the Model

Description of Phillip’s ROI Process

Application Example

Additional case applications

Reflection Questions

Chapter 6 Brinkerhoff Success Case Method

Key Points103

Overview of the Model

Description of the SCM Process

Conclusion

Application Example

Reflection Questions

Chapter 7 The Impact Evaluation Process

Key Points

Overview of the Model

A note on the nature of inquiry of the Impact Evaluation Process

The Impact Evaluation Process: Seven Aligned and Reiterative Steps

Conclusion

Application Example

Reflection Questions

Chapter 8 The CIPP Model

Key Points

Overview of the Model

Description of Stufflebeam’s Four Types of Evaluation

Articulating Core Values of Programs and Solutions

Methods Used in CIPP Evaluations

Application Example

Additional case applications

Reflection Questions

Chapter 9 Evaluating Evaluations

Key Points

Can we trust Evaluation Findings?

Definition of Metaevaluation

Evaluation Standards

The American Evaluation Association Principles for Evaluators

Application Example

Reflection Questions

Section 3 Tools and Techniques of Evaluation

Chapter 10 Data

Key Points

Data Defined

Characteristics of Data

Defining Required Data from Performance Objectives

Deriving Measurable Indicators

Finding Data Sources

Chapter 11 Data Collection

Key Points

Introduction to Data Collection Methods

Observation Methodology and the Purpose of Measurement

Problems with Classic Experimental Studies in Applied Settings

Time Series Studies

Document-Centered Methods

Reflection Questions

Chapter 12 Analysis of Evaluation Data: Tools and Techniques

Key points

Methods of Qualitative Analysis

Analysis of Models and Patterns

Analysis using Structured Discussion

Imposing Structure on Emerging Issues

Relevancy

Controls on Structured Discussion

Methods of Quantitative Analysis

Statistics

Two types of statistics

Measures of Central Tendency

Measures of Dispersion (Variability)

Graphical Representations of Data27

Measures of Relationsh

Inferential Statistics:  Parametric and Non-Parametric

Interpretation

Reflection Questions

Chapter 13 Communicating the Findings

Key Points

Recommendations

Considerations for Implementing Recommendations

Developing the Report

The Evaluator’s Role After the Report

Reflection Questions

Section 4 Continual Improvement

Chapter 14 Common Errors in Evaluation

Errors of System Mapping

Errors of Logic

Errors of procedure

Reflection Questions

Chapter 15 Continual Improvement

Key Points

What is Continual Improvement?

Monitoring Performance

Adjusting Performance

The Role of Leadership

Reflection Questions

Chapter 16 Contracting for Evaluation Services

Key Points

The Nature of Contracts

Fixed Price

Cost Plus Fixed Fee

Time and Materials

Consulting Retaine

Contracting Controls

Contract Elements

Other Contract Clauses

Sample Statement of Work (SOW)

General Information

Contract Award Meeting

General Requirements

Specific Mandatory Tasks and Associated Deliverables

Description of Tasks and Associated Deliverables:

Schedule for Deliverables

Changes to Statement of Work

Reporting Requirements

Travel and Site Visits

Sellalot Corporation Responsibilities

Contractor Experience Requirements

Confidentiality and Nondisclosure

Chapter 17 Intelligence Gathering for Decision Making: Performance Measurement Systems

Key Points

Measures, Intelligence and Decisions

Performance Measurements Systems

Issues in Performance Measurement Systems

Conclusion

Reflection Questions

Chapter 18 The Future of Evaluation in Performance Improvement

Key Points

Introduction

Evaluation and Measurement in Performance Improvement Today

What Does the Future Hold?

Conclusion

References and Related Readings