1. Decisions, Decisions: What Policymakers
Need to Know About Cost-Benefit Analysis
October 25, 2012
Craig Prins, Executive Director, Oregon Criminal Justice Commission
Sarah Galgano, Policy Analyst, Vera Institute of Justice
Slide 1

2. Decisions, Decisions: What Policymakers
Need to Know About Cost-Benefit Analysis
Craig Prins Sarah Galgano
Executive Director Policy Analyst
Oregon Criminal Justice Vera Institute of Justice
Commission
Slide 2

3. The Cost-Benefit Knowledge Bank for Criminal Justice (CBKB)
is a project of the Vera Institute of Justice funded by the
U.S. Department of Justice’s Bureau of Justice Assistance.
• Website (cbkb.org)
• Cost-benefit analysis toolkit
• Snapshots of CBA literature
• Podcasts, videocasts, and webinars
• Roundtable discussions
• Community of practice
• Technical assistance
Slide 3

4. Agenda
Introductions & housekeeping 5 minutes
Cost-benefit analysis (CBA) basics 5 minutes
Case study: The use of CBA in Oregon 10 minutes
CBA and decision making 15 minutes
How CBA can answer questions
about policy options 10 minutes
Q&A 15 minutes
Slide 4

5. Housekeeping items
Questions:
Use the Chat feature to send us
questions during the webinar.
Slide 5

6. Housekeeping items (continued)
• If you need webinar support or for troubleshooting:
 Type questions into the chat box.
 Call 800-843-9166.
 Send e-mail to help@readytalk.com.
• This webinar is being recorded.
• The recording and PowerPoint slides will be posted
on cbkb.org.
Slide 6

7. Preview
This webinar will discuss:
 The role of CBA in decision making,
even when data is imperfect
 Questions frequently asked about
CBA results
 The ways a CBA can address questions
relevant to decision making
Slide 7

8. CBA Basics
Slide 8

9. What is cost-benefit analysis?
A type of economic analysis that
compares the costs and benefits of
policies and programs, and that:
• includes the perspectives of
multiple stakeholders;
• presents a long-term picture;
• uses dollars as a common
measurement; and
• allows you to compare
programs and policies that
have different outcomes.
Slide 9

10. The CBA process
1. Determine the impact of the initiative.
2. Determine which perspectives to include.
3. Measure costs and benefits (in dollars).
4. Compare costs and benefits.
5. Assess the reliability of the results.
For a glossary of CBA terms, refer to
cbkb.org/basics/glossary
Slide 10

11. Case Study:
The Use of CBA in Oregon
Slide 11

12. Justice policy is multifaceted
• Justice policy often has these goals:
 Deter new criminal activity.
 Reduce recidivism.
 Improve public safety cost-effectively.
• A new policy has resource costs, but can also save
government resources and benefit society.
• CBA monetizes savings, benefits, and resource costs.
Slide 12

13. Building a capacity for CBA in Oregon
• In 2006, Oregon Criminal Justice Commission began
the development of statewide cost-benefit model for
the criminal justice system.
• Oregon leveraged the cost-benefit work of the
Washington State Institute for Public Policy (WSIPP).
• The Oregon model was designed to provide
information to policymakers and the public about the
relative costs and effectiveness of justice investments.
Oregon’s cost-benefit methodology is available at
http://www.oregon.gov/cjc/docs/cost_benefit_methodology_090106.pdf
Slide 13

14. Oregon invests in reentry programs
• Oregon made an investment decision based
on credible CBA results from another state.
• WSIPP had estimates of reentry program success
in Washington.
• Would the results translate to Oregon?
 The state has 36 diverse counties.
 WSIPP cost estimates may not apply to
rural areas.
 The program’s effect in Oregon may differ from
the WSIPP estimate.
Slide 14

15. Example of WSIPP estimates
Slide 15

16. Oregon’s reentry program
• Continues prison treatment services for people
once they are in the community
• Coordinates ancillary, community-based services for
recently released individuals:
 Mental health services
 Employment counseling
 Educational attainment programs
• Increases public safety and reduces recidivism
Slide 16

17. CBA of Oregon’s reentry program
• Researchers at the Oregon Criminal Justice
Commission (CJC) conducted an outcome evaluation
and a cost-benefit analysis.
• Researchers used local data to:
 Determine program-specific costs for the
reentry center.
 Track individuals who use services:
Did they reoffend?
 Measure the benefits specific to Oregon.
Slide 17

18. CBA results: Oregon’s reentry program
The statewide reentry program:
 Reduces recidivism by 27%
 Results in an estimated 3.5 fewer felony convictions
for every 10 participants over a 10-year period
 Costs about $3,400 per participant
 Generates benefits of $8,600 to taxpayers
and $14,000 in avoided victimizations per participant
 Results in benefit-cost ratio of $6.73 to every $1 spent
Program evaluation and cost-benefit analysis available at:
http://www.oregon.gov/CJC/docs/reentry_eval_final.pdf
Slide 18

19. Lessons learned from Oregon CBA
• Local data adds value when using a CBA to inform
policy decisions. Statewide averages can be
misleading.
• Success is not a sure thing; program fidelity is key.
• CBA adds important information to the policy dialogue.
Slide 19

20. Questions?
Please use the Chat feature to send us your questions.
Craig Prins Sarah Galgano
craig.prins@state.or.us sgalgano@vera.org
Slide 20

21. CBA and Decision Making
Slide 21

22. CBA is a decision-making tool
• CBA is a way to consider the pros and cons
systematically.
• Justice policy is a long-term investment decision.
• Beyond good intentions: Is a given policy the most
effective way to improve public safety?
• CBA is a business-like approach to increasing public
safety and reducing cost as our “nonprofit’s” goal.
Slide 22

23. Frequently asked questions about CBA
Decision makers (e.g., legislators, legislative staff,
budget planners) often have questions about CBA:
1. Can I believe the results?
2. Are the results guaranteed?
3. Who gets the benefits?
4. What is the alternative?
Slide 23

24. Can I believe the results?
• Confidence in the results is related to the quality of the
CBA’s inputs.
 Are the predicted policy effects based on a
rigorous analysis?
 Are the program’s costs measured accurately?
• Sensitivity analysis should be used to “kick the tires”
on the results.
• Transparency with the details will increase readers’
confidence in the study.
Slide 24

25. Are the results guaranteed?
• No. No one has a crystal ball.
• “Past performance does not guarantee future results.”
• The policy effect can vary from the CBA results for
these reasons:
 Measurement of the predicted policy effect
 Fidelity of the program to the policy that was
evaluated
• The costs and benefits of the policy effect may be
hard to monetize.
Slide 25

26. Who gets the benefits?
• The return on investment is both to the government
budget (taxpayer benefits) and to society (such as
victim benefits).
• These benefits can be presented separately to provide
clarity as to how they accrued.
• Some benefits may accrue in the long term.
• Budget savings are realized when there is less crime
and therefore less spending on courts, policing, and
corrections.
Slide 26

27. Who gets the benefits? (continued)
• CJC estimate: A 1% drop in recidivism results in
$4.3 million avoided in annual victim and taxpayer
costs due to crime.
• Who gets these savings?
 Savings may be reallocated to other sectors of
corrections/state budgets.
 Taxpayer resources may be used more efficiently.
Slide 27

28. What is the alternative?
• CBA is based on programmatic/initiative investments.
• A single cost-benefit analysis reports the net benefit
versus “business as usual.”
• Several CBAs can provide a menu of policy choices
(similar to the ratings in Consumer Reports).
Slide 28

29. Questions?
Please use the Chat feature to send us your questions.
Craig Prins Sarah Galgano
craig.prins@state.or.us sgalgano@vera.org
Slide 29

30. How CBA Can Answer
Questions About Policy
Options
Slide 30

31. Frequently asked questions about CBA
1. Can I believe the results?
2. Are the results guaranteed?
3. Who gets the benefits?
4. What is the alternative?
Slide 31

32. How CBA can answer these questions
Question: How CBA can answer:
1. Can I believe the results? Base-case scenario
Partial sensitivity analysis
2. Are the results guaranteed? Break-even analysis
Best- and worse-case scenarios analysis
Monte Carlo analysis
3. Who gets the benefits? Analysis by perspective
4. What is the alternative? Menu of CBA results
Slide 32

33. Hypothetical prison education program
• This example will be used to illustrate how CBA can
address questions 1-3.
• Hypothetical program:
 Aims to reduce number of people who commit new
crimes after their release
 Uses estimates based on evaluations of similar
programs because information on program impact is
not available
 Has reduced recidivism by 15 percent in other states
 Costs an estimated $10,000
Slide 33

34. 1. Can I believe the results?
A CBA can earn the trust of readers by:
 Documenting why “base-case” scenario
assumptions were chosen
 Using partial sensitivity analysis to illustrate how
the results vary if the assumptions are inaccurate
Slide 34

35. Base-case scenario
Table 1: Hypothetical Prison Education Program: Base Case
Total Cost Total Benefit Net Benefit
Prison education $10,000 $17,000 $7,000
Note: The program is assumed to reduce recidivism by 15 percent
over three years, based on studies of other similar programs.
Slide 35

36. Base-case scenario
• Provides information on the best estimate of a policy’s
net benefits
• Is based on direct evaluation of the initiative or
evaluations of similar programs
• Explains what assumptions were used
Slide 36

37. Partial sensitivity analysis
Table 2: Hypothetical Prison Education Program:
Partial Sensitivity Analysis
Change in recidivism +5% 0% -5% -10% -15% -20%
Program costs $10,000 $10,000 $10,000 $10,000 $10,000 $10,000
Program benefits -$2,000 $5,000 $8,000 $10,000 $17,000 $22,000
Net benefits
-$12,000 -$5,000 -$2,000 $0 $7,000 $12,000
(Benefit minus cost)
Slide 37

38. Partial sensitivity analysis
• Provides information on how variation from the
base-case scenario will affect the CBA results
• Determines how sensitive the results are
to changes in a single variable
• Involves selecting a single variable in the analysis
and changing its value (holding other variables
constant)
Slide 38

39. 2. Are the results guaranteed?
• A CBA cannot provide guaranteed results,
but can address uncertainty by:
 Reporting the policy effect necessary for the costs
and benefits to break even
 Reporting the best-case and worse-case
scenarios
 Modeling the probability of potential net benefit
using Monte Carlo analysis
Slide 39

40. Break-even analysis
Table 3: Hypothetical Prison Education Program:
Break-Even Analysis
Change in recidivism +5% 0% -5% -10% -15% -20%
Program costs $10,000 $10,000 $10,000 $10,000 $10,000 $10,000
Program benefits -$2,000 $5,000 $8,000 $10,000 $17,000 $22,000
Net benefits
-$12,000 -$5,000 -$2,000 $0 $7,000 $12,000
(Benefit minus cost)
Break-even
point
Slide 40

41. Break-even analysis
• Provides information on the policy effect necessary
for the program benefits to equal the program costs
• Useful when information about program impact
is unavailable
Slide 41

42. Best-case/worse-case scenario analysis
Table 4: Hypothetical Prison Education Program:
Best-case and Worse-case Scenarios
Change in recidivism +5% 0% -5% -10% -15% -20%
Program costs $10,000 $10,000 $10,000 $10,000 $10,000 $10,000
Program benefits -$2,000 $5,000 $8,000 $10,000 $17,000 $22,000
Net benefits
-$12,000 -$5,000 -$2,000 $0 $7,000 $12,000
(Benefit minus cost)
Worst Best
case case
Slide 42

43. Best-case/worse-case scenario analysis
• Provides policymakers with information on how a
broad range of a policy’s possible outcomes affect the
bottom line.
• Establish upper and lower boundaries of a cost-benefit
study’s results.
• Worst-case scenario analysis is based on using all the
least favorable assumptions.
• Best-case scenario analysis is based on all the most
favorable assumptions.
Slide 43

44. Example of Monte Carlo results
Slide 44

45. Monte Carlo analysis
• Provides information about the probability that
outcomes will occur
• Examines multiple variables simultaneously and
simulates thousands of scenarios
• Produces a range of possible results and their
associated probabilities
For more on Monte Carlo analysis, watch the recording of the CBKB
webinar “Sensitivity Analysis for Cost-Benefit Studies of Justice
Policies” at cbkb.org.
Slide 45

46. 3. Who gets the benefits?
Table 5: Hypothetical Prison Education Program,
by Perspective
Taxpayer Victim Participant Total
Costs $10,000 $0 $0 $10,000
Benefits $12,000 $2,000 $3,000 $17,000
Net benefit $2,000 $2,000 $3,000 $7,000
Slide 46

47. Reporting costs and benefits
by perspective
• CBA can answer this question by reporting the costs
and benefits for each perspective (or stakeholder) in
the analysis.
• Outcomes related to perspectives that were not
measured quantitatively should be discussed
qualitatively.
For more on perspectives, refer to the CBKB Perspectives tool at
cbkb.org/toolkit/perspectives.
Slide 47

48. 4. What is the alternative?
CBA can be performed on more than one policy option
to provide a menu of options.
Slide 48

49. Policy options in Washington State
Slide 49

50. Wrap-up
Question: How CBA can answer:
1. Can I believe the results? Base-case scenario
Partial sensitivity analysis
2. Are the results guaranteed? Break-even analysis
Best- and worse-case scenarios analysis
Monte Carlo analysis
3. Who gets the benefits? Analysis by perspective
4. What is the alternative? Menu of CBA results
Slide 50

51. Questions and Answers
Slide 51

52. Questions?
Please use the Chat feature to send us your questions.
Craig Prins Sarah Galgano
craig.prins@state.or.us sgalgano@vera.org
Slide 52

53. Wrap-up
Slide 53

54. Review
• We covered:
• How CBA adds to policy dialogue
• What questions arise about CBA results
• How CBA can address frequently asked questions
Slide 54

55. Takeaways
• CBA is a useful decision-making tool.
• CBAs is not just about the results. The analysis
adds important information to the policy dialogue.
Slide 55

56. Follow-up
Before you log out of the webinar, please complete
the evaluation form.
To receive information and notifications about upcoming
webinars and other events:
• Visit the Cost-Benefit Knowledge Bank for Criminal Justice
at cbkb.org.
• Subscribe to receive updates from CBKB.
• Follow us on Twitter at twitter.com/CBKBank.
Slide 56

57. Contact information
Craig Prins Sarah Galgano
Executive Director Policy Analyst
Oregon Criminal Justice Vera Institute of Justice
Commission
craig.prins@state.or.us sgalgano@vera.org
Slide 57

58. The Cost-Benefit Knowledge Bank for Criminal Justice (CBKB)
is a project of the Vera Institute of Justice funded by the
U.S. Department of Justice’s Bureau of Justice Assistance.
• Website (cbkb.org)
• Cost-benefit analysis toolkit
• Snapshots of CBA literature
• Podcasts, videocasts, and webinars
• Roundtable discussions
• Community of practice
• Technical assistance
Slide 58

59. Thank you!
Slide 59

60. This project is supported by Grant No. 2009-MU-BX K029
awarded by the Bureau of Justice Assistance. The Bureau of
Justice Assistance is a component of the Office of Justice
Programs, which also includes the Bureau of Justice Statistics,
the National Institute of Justice, the Office of Juvenile Justice
and Delinquency Prevention, and the Office of Sex Offender
Sentencing, Monitoring, Apprehending, Registering, and
Tracking. Points of view or opinions in this document are those
of the author and do not represent the official position or
policies of the United States Department of Justice.
Slide 60