This seminar will focus on how countries can establish a policy framework to enable effective local action using an evidence-based approach, choosing between different measures when resources are scarce. The latest evidence from OECD countries on ‘what works’ and ‘what doesn’t’ will be considered, with participants sharing their own experiences from their perspectives as policy makers, researchers, practitioners and social entrepreneurs.

1. OECD LEED Seminar: What works, what
doesn’t? Evidence-based approaches
OECD, 15th October 2014
The long-term impact of
employment services
David Grubb
Employment Analysis and Policy Division

2. Q. Do intensive case-management interviews with
jobseekers push them into short-term, unstable jobs?
A. Rigorous random-assignment evaluations have
identified large and relatively long-term impacts
2
Background – employment services /
work-first

3. • Target group: unemployment benefit recipients in 1989
• Restart process “combines counseling and encouragement with
tighter enforcement of conditions necessary qualify for
unemployment benefits”
• The control group was interviewed after 6 months of
unemployment (the norm at the time); the treatment group after
12 months
• By month 12, the treatment group’s unemployment rate was
about 10 percentage points higher
• There was little evidence of an longer-term impact for women.
But for men, the treatment group’s unemployment rate in years
3, 4 and 5 stayed about 6 points higher (c.40% instead of 34%).
• Gender difference: in response to interviews, women more
often left the labour force while men more often started work
3
UK Restart interviews

4. • Target group: AFDC recipients (lone parents)
• Meta-analysis of 24 evaluation studies covering 64 programs
that operated from 1983 to 1996
• The interventions studies were classified as “work first” (referral
to jobs, work experience) vs. “human capital” (training)
• For “work first” the average impact on total quarterly earnings
was immediately positive. For “human capital”, the impact was
at first zero. In both cases, the impact increased for some time
but started to decline after 3 years, and fell to zero by year 6.
(Based on other studies, the authors suggest that the impact of
voluntary, high-cost, training programs may “linger longer”.)
• Over 24 quarters (6 years), the impact on earnings of “work first”
($4000) was much greater than that of “human capital” ($1500)
4
US Welfare to Work experiments

5. Canada’s Self-Sufficiency Project Plus
• SSP (at random assignment in 1993/94) offered parents on welfare for a
year a large cash supplement (up to 3 years) if they worked 30 h/week
• SSP Plus offered, in addition, an employment plan, resumé service, job
clubs, workshops, coaching, and job leads (find/retain/regain a job)
• One year into the program, SSP participants were twice as likely to be
working 30 h/week as controls. In total they earned $3400 more (+20%).
But when the cash supplement stopped, the impact stopped.
• In years 1-2, SSP Plus had little impact over regular SSP. But by years 4
to 5Q2, SSP Plus participants were earning about $100/month more.
• In total SSP Plus participants earned about $3200 more up to year 5Q2,
and probably kept earning $100/month more – so that the employment
services had more total impact than 3 years of cash supplements did
• Analysis suggests SSP Plus helped those who secured employment to
stay in work for a longer time. SSP Plus is likely to have cost little more
than regular SSP (the main impact on the services received was on:
“Took part in job-search program such as job club or job-search”). 5

6. • New UI recipients from July to December 2009 (in deep
recession) were referred to a benefit eligibility interview which
also offered an “array” of services designed to enhance the
quality of job search
• For the treatment group compared with control:
– the % exhausting regular UI benefits (i.e. unemployed for six
months) fell by 10 points (-16%)
– total earnings over 18 months increased by $2600 (+18%)
– employment was 25% in Q2 after random assignment, and still
14% higher in Q6
• “the largest portion of the impacts is attributed to the
effectiveness of the services..” “conditional on employment,
treatment group recipients had higher earnings”.
6
Re-employment services in Nevada

7. Conclusions about the impact
• Controlled experiments have often found a large multi-year
positive impact from employment services – though:
– There are also findings of no/negative impact: women, in the
UK experiment; National Employment Action Plan in Ireland
– Declining returns may apply: plausibly impact is greatest
when the alternative is no or only limited services
– Quality matters – the UI program in Nevada had more impact
than UI programs in several other States.
– Counselling is not pure “work first” - it delivers assistance and
can act as a gateway to more-expensive services
• Despite the caveats: an ongoing/repeated service offer to
out-of-work clients – which is delivered by a well-managed
PES – plausibly has a cost-effective lifetime impact on
unemployment, employment and earnings 7

8. • Non-experimental evaluations may not be reliable
• Outcomes need to be tracked for all individuals who were
assigned to treatment vs. control groups – this requires
significant sample sizes
• Short-term impacts may or may not fade to zero in the
following few years – or impacts may start small, but then
become significant after two years. To limit the cost of
tracking long-term outcomes, administrative data are needed.
• Observed impacts are influenced by the broader policy
context, target group, jobseeker expectations, etc. The
interpretation of findings in one specific context and their
application to a different context requires some expertise and
judgement.
8
Accurate evaluation is difficult

9. Slide 3: Dolton, P. and D. O’Niell (2002), “The Long-Run Effects of
Unemployment Monitoring and Work-Search Programs: Experimental Evidence
from the United Kingdom”, Journal of Labour Economics, Vol. 20, No. 2, Pt. 1.
Slide 4: Greenberg, D., K. Ashworth, A. Cebulla and R. Walker (2004), “Do
Welfare-to-Work Programmes Work for Long?”, Fiscal Studies, vol. 25, no. 1.
Slide 5: Michalopoulos, C., D. Tattrie, C. Miller, P. Robins, P. Morris, D.
Gyarmati, C. Redcross, K. Foley and R. Ford (2002), Making Work Pay: Final
Report on the Self-Sufficiency Project for Long-Term Welfare Recipients, Social
Research and Demonstration Corporation.
Slide 6: Michaelides, M. (2013), Are Reemployment Services Effective in
Periods of High Unemployment? Experimental Evidence from the UI System,
University of Cyprus, mimeo., December; and Michaelides, M. E. Poe-
Yamagata, J. Benus, and D. Tirumalasetti (2012), Impact of the Reemployment
Eligibility Initiative In Nevada, Impaq International.
Slide 7: McGuinness, S., P. O’Connell, E. Kelly and J. Walsh (2011), Activation
in Ireland: An Evaluation of the National Employment Action Plan, ESRI
Research Series 20.
9
References