Dr. Robert Berenson's slides from the Center for Health Journalism webinar "Does ‘Pay for Performance’ Work?" 6.28.16 http://www.centerforhealthjournalism.org/content/does-pay-performance-work

1. Pay-for-Performance: An Idea
Whose Time Has Passed
Robert A. Berenson, MD
Institute Fellow, Urban Institute
rberenson@urban.org
Webinar on P4P
Center for Health Journalism
28 June 2016
1

2. “If you can’t measure it, you can’t
manage it”
• And its cousin, “If something… cannot be
measured, it cannot be improved.”
– Has been called a truism – something so obvious it
doesn’t even need to be said
• The quote is commonly attributed to W. Edwards
Deming, a revered management expert who helped
turn around the Japanese economy after WWII and
is considered a father of “total quality management”
in U.S. industry
2
URBAN INSTITUTE

3. What Deming Actually Wrote
• “It is wrong to suppose that if you can’t measure it, you can’t
manage it – a costly myth.”
– The New Economics, 1994, page 35.
– So not just taken out of context, but an overt misquote
• Other consistent Deming quotes (of many available):
– “The most important figures one needs for management are unknown
or unknowable, but successful management must nevertheless take
account of them.” Out of the Crisis, 1982, p 121
– “Management by numerical goal is an attempt to manage without
knowledge of what to do, and in fact is usually management by fear.”
Out of the Crisis, p. 76
3
URBAN INSTITUTE

4. Dueling aphorisms
• “If you can’t measure it, you can’t manage it”
– Commonly attributed to Deming (sometimes, to
Peter Drucker, another management scholar,
who also did not believe it because he thought
the most important thing to manage is
relationships, which are hard to measure)
• “Not everything that can be counted counts,
and not everything that counts can be
counted.”
– Not Albert Einstein, as broadly believed, but
rather a sociologist, William Bruce Cameron
URBAN INSTITUTE
4

5. No wonder we don’t do evidence-
based policy making
We can’t even get quotes right
5
URBAN INSTITUTE

6. Of Course, Measurement Often Helps,
and Sometimes Is Indispensable
• Like for controlling hypertension – you have to
measure the blood pressure to manage it
• Yet, even here, we still don’t have uniform, high
standards for measuring blood pressure, either
clinically or for purposes of public reporting and
P4P
6
URBAN INSTITUTE

7. Accurate Measurement is Difficult
• Even relying on a seemingly straight-forward metric
such as a hospital readmission rate can be
misleading
– Like when a health care system successfully reduces
both readmissions and admissions – the change in the
ratio may not reflect its success
– Like when hospitals serving very different patient
populations in geographic areas with different resources
are compared to each other
– Like when there may be an opportunity for hospitals to
engage in “regulatory evasion” by calling an admission an
“observation” stay (this one is in dispute in fact but not in
theory)
7
URBAN INSTITUTE

8. Need to distinguish use of measures for
public reporting (and P4P) and internal QI
• Of course, for internal management, having data to
assess intervention impact is desirable – my
concern is about public reporting and P4P
• For internal QI, one can be less rigorously accurate
– For example, false positive measures can be readily
accepted in internal QI – want to cast a wide net and then
home in, at which time the validity and implications of the
screening measure can be determined
8
URBAN INSTITUTE

9. Let’s Stipulate That:
• The “status quo ante” – no accountability –
produced mediocre quality and too much spending
• Public reporting (as opposed to P4P) has some
value and in fact does involve financial incentives,
i.e., reputation can move market share
• At this point, we can’t put the genie of performance
measurement back in the bottle
• In certain circumstances, some P4P can be useful
as part of a comprehensive payment approach
9
URBAN INSTITUTE

10. Some Conceptual Concerns
About P4P
• From behavioral economists – “crowding out”
intrinsic motivation that health professionals already
have
• “Testing to the test”
• Together, these two could result in overall
performance decline even if incentivized
performance improves -- we measure much less
than what we care about
10
URBAN INSTITUTE

11. Operational Challenges with P4P
• Major gaps in available measures, which rely largely on
claims data, with no other reliable and cost-reasonable
sources on the horizon (except perhaps for patient surveys)
• Small numbers, making statistically valid inferences of
individual clinician performance problematic
• High administrative costs – a recent Health Affairs paper
estimated $15.4 billion/year just for physician practices
• Provider “gaming” behavior in response to P4P, to the
detriment of patient care
– Avoidance of more difficult patients
– Increase in 31-day mortality rates
11
URBAN INSTITUTE

12. And the Current Policy Infatuation with
Measurement, Reporting, and P4P
Leads to This Perverse Policy Result:
What we measure is considered important
and demands attention
What we can’t or don’t measure is
marginalized or ignored altogether -- such
as diagnosis errors, a major quality
problem in U.S. health care
URBAN INSTITUTE
12

13. How Policy Should Improve the Use of
Measures (and perhaps P4P)
• Use measures strategically as part of major quality
improvement initiatives;
• Measure at the level of the organization, not the clinician;
• Try to move from process to outcome measurement;
• Place greater emphasis on patient experience and patient-
reported outcome measures;
• Invest more in the “basic science” of measurement
development, tasking a single entity with defining standards
for measuring and reporting performance:
– to improve the validity and comparability of publicly-reported quality
data and
– to anticipate and prevent unintended adverse consequences
13
URBAN INSTITUTE

14. Alternatives to P4P Would
Emphasize:
• Reducing incentives in basic payment that produce
too much care -- hastily and carelessly provided --
instead of trying to counter these dominant
incentives with relatively small, P4P dollars
• “The most powerful methods for reducing medical
harm are: feedback, learning from the best, and
working in collaboration”
– Lucian Leape, M.D. commenting on the success of the
Michigan Keystone Project at eliminating central line-
associated blood stream infections in MI hospitals.
14
URBAN INSTITUTE

15. Thanks
15
URBAN INSTITUTE