The Relationship Between Quality of Care and Choice of Clinical Computing System: GP Performance Under the Quality and Outcomes Framework

1. Background
Methods
Findings
Summary
The Relationship Between Quality of Care and
Choice of Clinical Computing System
GP Performance Under the Quality and Outcomes Framework
Evan Kontopantelis12 Iain Buchan1 David Reeves12
Kath Checkland12 Tim Doran3
1Institute of Population Health, University of Manchester
2NIHR School for Primary Care Research
3Department of Health Sciences, University of York
CHI meeting, 10 July 2013
Kontopantelis Clinical Computing Systems and QOF

2. Background
Methods
Findings
Summary
Outline
1 Background
2 Methods
3 Findings
Variation
Model results
4 Summary
Kontopantelis Clinical Computing Systems and QOF

3. Background
Methods
Findings
Summary
Clinical Computing Systems
Promoted as a means to improve the quality of health care,
with advantages over paper based systems:
improved data recording
integration and accessibility
feedback and alerts
Drive improvements in efficiency, process performance,
clinical decision making and medication safety?
In practice the results of implementing information
technology systems have been mixed:
variable quality of software systems by multiple providers
clinicians adapting at different rates to systems that may not
address their specific requirements or challenge their
approach to existing practice
Kontopantelis Clinical Computing Systems and QOF

4. Background
Methods
Findings
Summary
Clinical Computing Systems
in the UK Primary Care
Developed from multiple systems in the 1980s into a
handful using Read codes by 2000
From 1998 practices partially subsidised for the costs of
installing clinical computing systems
Full subsidies provided from 2003 in preparation for the
implementation of a national pay-for-performance scheme,
the Quality and Outcomes Framework, in April 2004
Kontopantelis Clinical Computing Systems and QOF

5. Background
Methods
Findings
Summary
The Quality and Outcome Framework (QOF)
Provides large financial bonuses to practices based on
achievement on over 100 quality of care indicators
Practices awarded points for each quality indicator based
on the proportion of patients for whom targets are met
Each point worth £126, adjusted for the relative prevalence
of the disease and the size of the practice population
Practices can exclude (’exception report’) inappropriate
patients from achievement calculations for various
reasons:
logistical (e.g. registration close to end of financial year)
clinical (e.g. a contraindication to treatment)
patient informed dissent
Kontopantelis Clinical Computing Systems and QOF

6. Background
Methods
Findings
Summary
The Quality and Outcome Framework (QOF)
continued
Performance data drawn from clinical computing systems
and collated on a national database (the Quality
Management and Analysis System - QMAS)
systems must conform to standard interoperability
requirements
QMAS scores on each indicator are used for feedback and
payment calculations
The QOF is in its 9th year and has cost on average ≈£1bn
per annum
Kontopantelis Clinical Computing Systems and QOF

7. Background
Methods
Findings
Summary
QOF impact
Substantial impact on use of clinical computing systems
Practices are required to keep disease registers and
because bonus payments increase with disease
prevalence there is an incentive to case-find
QOF business rules specify criteria and permissible Read
codes for identifying patients with particular conditions:
greater uniformity of code usage
even changing diagnostic behaviour
Software providers have adapted their systems to facilitate
better practice performance on the QOF:
incorporating pop-up alerts and management tools
QOF-oriented training programmes for practice staff
Kontopantelis Clinical Computing Systems and QOF

8. Background
Methods
Findings
Summary
The question
There is no independent evidence to date on whether
practice performance on the QOF and recorded quality of
care is associated with the practice’s choice of clinical
computing system
We used a unique dataset to assess these relationships in
English family practices operating under the national
pay-for-performance scheme.
Retrospective study of QOF performance by English family
practices from 2007-8 to 2010-11, identifying practice
predictors, including choice of clinical computing system
Kontopantelis Clinical Computing Systems and QOF

9. Background
Methods
Findings
Summary
Data
Quality Management and Analysis System (QMAS), which
holds data for almost all English practices (over 8,200)
Clinical computing systems info not publicly reported;
obtained relevant dataset from the HSCIC
Practice characteristics and the populations they serve
obtained from the General Medical Services (GMS)
Statistics database, also provided by the HSCIC
Area deprivation, as measured by the IMD, from the
Communities and Local Government website
Urban/non-urban classification, from the ONS website
Data were complete for all practices
Kontopantelis Clinical Computing Systems and QOF

10. Background
Methods
Findings
Summary
Outcomes
Practice performance on the 62 QOF clinical quality
indicators that were continually incentivised over the study
period
Three performance measures used:
reported achievement (RA), % of eligible patients for whom
targets achieved (not including exception reported patients)
population achievement (PA), % of eligible patients for
whom targets actually achieved (including exception
reported patients)
% of QOF points scored (PQ), the metric on which
remuneration is based
Kontopantelis Clinical Computing Systems and QOF

11. Background
Methods
Findings
Summary
Indicators
some of the 62
Table A-6: Quality and Outcomes Framework indicators included in the main analysis
Indicator
Min
Threshold
Max
Threshold
Points Description
Type of
indicator
AF 4 40 90 10 Diagnosis confirmed by ECG or specialist Measurement
AF 3 40 90 15 Treated with anti-coagulation/anti-platelet Treatment
ASTHMA 8 40 80 15 Variability/reversibility measured Measurement
ASTHMA 3 40 80 6
Smoking status recorded
(ages 14-19)
Measurement
ASTHMA 6 40 70 20 Care reviewed Measurement
BP 4 40 90 20 Blood pressure measured Measurement
BP 5 40 70 57 Last blood pressure ≤ 150/90 mmHg Outcome
CANCER 3 40 90 6 Care reviewed Measurement
CHD 2 40 90 7 Referred for exercise testing/specialist assessment Measurement
CHD 5 40 90 7 Blood pressure measured Measurement
CHD 6 40 70 19 Blood pressure ≤ 150/90 mmHg Outcome
CHD 7 40 90 7 Total cholesterol measured Measurement
CHD 8 40 70 17 Total cholesterol ≤5 mmol/l Outcome
CHD 9 40 90 7 Treated with aspirin/anti-platelet/anti-coagulant Treatment
CHD 10 40 60 7 Treated with beta blocker Treatment
CHD 11 40 80 7
Treated with ACE-inhibitor
(history of myocardial infarction)
Treatment
CHD 12 40 90 7 Influenza vaccination Treatment
HF 2 40 90 6 Diagnosis confirmed by echocardiogram/specialist Measurement
HF 3 40 80 10 Treated with ACE inhibitor/angiotensin receptor blocker Treatment
Kontopantelis Clinical Computing Systems and QOF

12. Background
Methods
Findings
Summary
Outcomes
continued
Across all three outcome measures (RA, PA and PQ)
practice composite scores were calculated
Overall, across all 62 clinical indicators
By three categories of activity:
measurement activities, 35 indicators (e.g. measurement of
blood pressure for hypertension patients)
treatment activities, 11 indicators (e.g. influenza
immunisation to CHD patients)
intermediate outcomes, 16 indicators (e.g. HbA1C level
control for diabetes patients)
Kontopantelis Clinical Computing Systems and QOF

13. Background
Methods
Findings
Summary
Statistical Modelling
Multilevel mixed effects multiple linear regression models
to identify population, practice, and clinical computing
system predictors of RA PA and PQ
Model 1: overall summary measure of 62 ind as outcome
estimate effect of each clinical system on outcome and
assess whether there were significant differences across
clinical systems
Model 2: indicator group (measurement, treatment,
outcome) summary measure as outcome
included interaction terms (clinical system by indicator
group) to estimate the system effect in each indicator group
and assess differences across clinical systems, in each of
the indicator groups
Kontopantelis Clinical Computing Systems and QOF

14. Background
Methods
Findings
Summary
Statistical Modelling
Analyses controlled for year, practice list size, local area
deprivation, rurality, type of GP contract, % female
patients, % patients aged 65+, mean GP age, % female
GPs, % UK qualified GPs, % GP providers (partners,
single-handed, or shareholders)
Linear predictions, and their 95% CIs, were calculated for
each clinical system from the regression models
These can be described as mean predicted outcome levels
for each factor (clinical system and indicator group by
clinical system) that are controlled for all model covariates
and set to their mean values in the models, thus allowing us
a fairer comparison of the systems
Kontopantelis Clinical Computing Systems and QOF

15. Background
Methods
Findings
Summary
Variation
Model results
System usage
Fifteen clinical computing systems from eight providers
Seven systems accounted for ≈ 99% of the market
Provider Product
Year
2007-8 2008-9 2009-10 2010-11
EMIS LV
3811
(46.0%)
3661
(44.5%)
3498
(42.2%)
3284
(39.9%)
In Practice Systems Vision 3
1572
(19.0%)
1599
(19.5%)
1564
(18.9%)
1492
(18.1%)
TPP ProdSysOneX
697
(8.4%)
851
(10.4%)
1164
(14.0%)
1466
(17.8%)
EMIS PCS
1103
(13.3%)
1160
(14.1%)
1259
(15.2%)
1216
(14.8%)
iSoft Synergy
541
(6.5%)
487
(5.9%)
444
(5.4%)
401
(4.9%)
Microtest Practice Manager
166
(2.0%)
164
(2.0%)
156
(1.9%)
156
(1.9%)
iSoft Premiere
248
(3.0%)
212
(2.6%)
169
(2.0%)
132
(1.6%)
Other*
145
(1.8%)
86
(1.0%)
31
(0.4%)
93
(1.1%)
Total 8283 8220 8285 8240
* Excluded from analyses: EMISWeb, GV (EMIS); HealthyV5, Crosscare (Healthy Software), Seetec GP
Enterprise (Seetec), Ganymede, System 6000 (iSoft), Exeter GP System (Protechnic Exeter Ltd)
Kontopantelis Clinical Computing Systems and QOF

16. Background
Methods
Findings
Summary
Variation
Model results
System and supplier distribution
North East
North West
London
West Midlands
Yorkshire & the Humber
South West
East Midlands
East of England
South Central
South East Coast
(85.4,85.7]
(85.1,85.4]
(84.8,85.1]
(84.5,84.8]
(84.2,84.5]
[83.9,84.2]
LV
Vision 3
ProdSysOneX
PCS
Synergy
Practice Manager
Premiere
NOTE: Chart size proportional to number of practices in area
Average practice scores by Strategic Health Authority, 2010−11
Overall population achievement (62 indicators)
and GP systems products
North East
North West
London
West Midlands
Yorkshire & the Humber
South West
East Midlands
East of England
South Central
South East Coast
(90.2,90.5]
(89.9,90.2]
(89.6,89.9]
(89.3,89.6]
(89,89.3]
(88.7,89]
[88.4,88.7]
EMIS
In Practice Systems
TPP
Microtest
ISoft
NOTE: Chart size proportional to number of practices in area
Average practice scores by Strategic Health Authority, 2010−11
Overall reported achievement (62 indicators)
and GP systems suppliers
Kontopantelis Clinical Computing Systems and QOF

17. Background
Methods
Findings
Summary
Variation
Model results
Performance and points achieved
by Primary Care Trust, 2010-11
(87,88]
(86,87]
(85,86]
(84,85]
(83,84]
(82,83]
(81,82]
(80,81]
[79,80]
Average practice scores by Primary Care Trust, 2010−11
Overall population achievement (62 indicators)
(91,92]
(90,91]
(89,90]
(88,89]
(87,88]
(86,87]
[85,86]
Average practice scores by Primary Care Trust, 2010−11
Overall reported achievement (62 indicators)
(99,100]
(98,99]
(97,98]
(96,97]
(95,96]
(94,95]
(93,94]
(92,93]
[91,92]
Average practice scores by Primary Care Trust, 2010−11
Overall % of points achieved (62 indicators)
Kontopantelis Clinical Computing Systems and QOF

18. Background
Methods
Findings
Summary
Variation
Model results
Reported achievement
by Primary Care Trust, 2010-11
(95,96]
(94,95]
(93,94]
(92,93]
(91,92]
(90,91]
[89,90]
Average practice scores by Primary Care Trust, 2010−11
Overall reported achievement (35 measurement ind)
(92,93]
(91,92]
(90,91]
(89,90]
(88,89]
(87,88]
[86,87]
Average practice scores by Primary Care Trust, 2010−11
Overall reported achievement (16 treatment ind)
(83,85]
(81,83]
(79,81]
(77,79]
(75,77]
[73,75]
Average practice scores by Primary Care Trust, 2010−11
Overall reported achievement (11 outcome ind)
Kontopantelis Clinical Computing Systems and QOF

19. Background
Methods
Findings
Summary
Variation
Model results
Population achievement
by Primary Care Trust, 2010-11
(92,93]
(91,92]
(90,91]
(89,90]
(88,89]
(87,88]
(86,87]
(85,86]
[84,85]
Average practice scores by Primary Care Trust, 2010−11
Overall population achievement (35 measurement ind)
(83,85]
(81,83]
(79,81]
(77,79]
[75,77]
Average practice scores by Primary Care Trust, 2010−11
Overall population achievement (16 treatment ind)
(78,80]
(76,78]
(74,76]
(72,74]
(70,72]
(68,70]
[66,68]
Average practice scores by Primary Care Trust, 2010−11
Overall population achievement (11 outcome ind)
Kontopantelis Clinical Computing Systems and QOF

20. Background
Methods
Findings
Summary
Variation
Model results
Percentage of points achieved
by Primary Care Trust, 2010-11
(98,100]
(96,98]
(94,96]
(92,94]
[90,92]
Average practice scores by Primary Care Trust, 2010−11
Overall % of points achieved (35 measurement ind)
(99,100]
(98,99]
(97,98]
(96,97]
(95,96]
(94,95]
(93,94]
(92,93]
[91,92]
Average practice scores by Primary Care Trust, 2010−11
Overall % of points achieved (16 treatment ind)
(99,100]
(98,99]
(97,98]
(96,97]
(95,96]
(94,95]
[93,94]
Average practice scores by Primary Care Trust, 2010−11
Overall % of points achieved (11 outcome ind)
Kontopantelis Clinical Computing Systems and QOF

21. Background
Methods
Findings
Summary
Variation
Model results
More practice variability
by Primary Care Trust, 2010-11
(50,60]
(40,50]
(30,40]
(20,30]
(10,20]
[0,10]
By Primary Care Trust, 2010−11
Average practice location deprivation
(9000,10000]
(8000,9000]
(7000,8000]
(6000,7000]
(5000,6000]
[4000,5000]
By Primary Care Trust, 2010−11
Average practice list size
Kontopantelis Clinical Computing Systems and QOF

22. Background
Methods
Findings
Summary
Variation
Model results
Practice characteristics by system
2010-11
LV Vision 3
ProdSys-
OneX PCS Synergy
Practice
Manager Premiere
List size
6903
(4174)
6681
(4286)
6587
(4333)
5715
(3922)
8053
(4287)
6654
(3791)
7104
(3703)
IMD 2010
25.2
(17.0)
25.8
(16.6)
28.3
(18.0)
31.1
(18.4)
23.5
(16.7)
25.1
(13.7)
21.1
(15.0)
Percentage of female patients
49.7
(2.9)
49.8
(2.6)
49.6
(3.1)
49.4
(3.5)
50.0
(2.2)
50.3
(1.7)
50.0
(1.7)
Percentage of patients aged 65 or over
15.4
(6.2)
15.0
(5.8)
15.4
(5.9)
14.3
(6.9)
16.8
(5.5)
19.6
(5.3)
16.3
(4.1)
General Practitioner (GP) age
47.6
(7.5)
48.3
(7.9)
47.3
(7.8)
48.2
(8.5)
46.3
(6.4)
47.4
(6.2)
47.9
(7.3)
Percentage of female GPs
43.5
(26.3)
39.5
(27.3)
38.6
(26.5)
38.9
(28.8)
45.9
(22.8)
37.3
(24.4)
42.1
(27.5)
Percentage of UK qualified GPs
70.6
(35.3)
62.8
(37.2)
63.0
(37.7)
60.3
(39.4)
77.7
(29.9)
84.9
(27.4)
73.5
(34.3)
Percentage of GP providers‖
74.0
(25.1)
73.4
(27.6)
73.0
(30.7)
72.2
(31.1)
72.0
(23.0)
78.1
(22.4)
79.1
(21.7)
Percentage of Urban practices 82.90% 88.80% 83.90% 89.60% 85.30% 66.70% 84.80%
Percentage of GMS practices 59.00% 59.00% 40.90% 54.00% 46.60% 65.40% 64.40%
* Reporting means (sd) or percentages
Kontopantelis Clinical Computing Systems and QOF

23. Background
Methods
Findings
Summary
Variation
Model results
Predictors of practice performance
2007-8 to 2010-11
Little change over time
Most practice and patient characteristics had significant
but small effects
for every additional 1000 patients on the practice list: RA
decreased by 0.11%, PA by 0.14%, PQ increased by 0.08%
practices in more deprived areas performed worse across
all three outcomes: -0.01% for RA, -0.02% for PA and
-0.006% for PQ, per 1 point increase on the IMD scale
(equivalent to 0.8%, 1.6% and 0.48% in the most affluent
compared with the most deprived areas)
rural practices scored lower on RA and PQ but not on PA
practices with a higher % of female patients performed
better on all three outcomes
Kontopantelis Clinical Computing Systems and QOF

24. Background
Methods
Findings
Summary
Variation
Model results
Predictors of practice performance
2007-8 to 2010-11
Overall performance differed significantly by clinical system
used, for all three outcomes
Best: Vision 3 (RA), Synergy (PA & PQ)
Worst: PCS, across all three measures
Synergy practices on average gain £602 more than PCS
practices each year
Performance by type of activity varied significantly across
clinical systems, for all three outcomes
Similar system rankings
Biggest differences across outcome indicators
Kontopantelis Clinical Computing Systems and QOF

25. Background
Methods
Findings
Summary
Variation
Model results
Predictions of practice performance, by system
2007-8 to 2010-11
Reported
Achievement
Population
Achievement % of points scored
Reported
Achievement
Population
Achievement % of points scored
Predictions
(95% CI) rank
Predictions
(95% CI) rank
Predictions
(95% CI) rank
Predictions
(95% CI) rank
Predictions
(95% CI) rank
Predictions
(95% CI) rank
Overall (aggregate over all 62 indicators), from regression model 1 Outcome (aggregate over 11 indicators), from regression model 2
Vision 3
90.1
(90.0,90.2) 1
85.4
(85.3,85.5) 2
97.8
(97.7,97.9) 3
81.0
(80.9,81.1) 2
75.3
(75.2,75.4) 2
98.3
(98.1,98.6) 1
Practice
Manager
89.8
(89.5,90.1) 2
84.7
(84.4,85.0) 5
97.4
(97.0,97.8) 6
81.4
(81.0,81.7) 1
75.0
(74.6,75.4) 3
98.2
(97.8,98.7) 3
Synergy
89.8
(89.7,89.9) 3
85.6
(85.4,85.7) 1
98.1
(97.9,98.2) 1
80.4
(80.2,80.6) 4
75.4
(75.2,75.6) 1
98.0
(97.7,98.3) 4
Premiere
89.8
(89.5,90.0) 4
85.1
(84.8,85.3) 3
98.1
(97.8,98.4) 2
80.7
(80.4,81.1) 3
74.9
(74.6,75.3) 4
98.3
(97.9,98.7) 2
ProdSysOneX
89.6
(89.4,89.7) 5
84.5
(84.4,84.6) 6
97.6
(97.5,97.7) 5
80.3
(80.1,80.5) 5
74.1
(73.9,74.2) 6
97.8
(97.5,98.1) 5
LV
89.4
(89.3,89.5) 6
85.0
(85.0,85.1) 4
97.7
(97.6,97.8) 4
79.5
(79.5,79.6) 6
74.3
(74.2,74.4) 5
97.8
(97.5,90.0) 6
PCS
88.7
(88.6,88.8) 7
84.3
(84.2,84.4) 7
97.3
(97.2,97.4) 7
78.7
(78.5,78.8) 7
73.3
(73.2,73.5) 7
97.4
(97.1,97.7) 7
Kontopantelis Clinical Computing Systems and QOF

26. Background
Methods
Findings
Summary
Variation
Model results
Relative monetary gains, compared to PCS
2007-8 to 2010-11
£602
£563
£379
£308
£219
£79
-£200
-£100
£0
£100
£200
£300
£400
£500
£600
£700
Overall
Measurement
Treatment
Outcome
Kontopantelis Clinical Computing Systems and QOF

27. Background
Methods
Findings
Summary
Findings
Performance levels differed across systems, even after
controlling for practice and patient characteristics
Differences were small in absolute terms, but:
the association between system and performance was
stronger than for any other patient or practice characteristic
substantial at the population level e.g. 1% difference in
achievement of BP control targets for HT patients ≈ 9
patients in average practice; over 71,000 patients nationally
Overall, best average performance across all outcomes
measures was by Vision 3, Synergy or Premiere systems
Synergy & Premiere in a diminishing minority of practices
and are now likely to be withdrawn from the market
Remuneration differences were practically negligible
Kontopantelis Clinical Computing Systems and QOF

28. Background
Methods
Findings
Summary
Conclusions
QOF performance is partly dependent on the clinical
computing system used by practices
Raises the question of whether particular characteristics of
computing systems facilitate higher quality of care, better
data recording, or both
Inconsistency across clinical computer systems which
needs to be understood and addressed
Researchers who use primary care databases, which
collect data from a single clinical system, need to be
cautious when generalising their findings
Messages relevant to international health care systems
Kontopantelis Clinical Computing Systems and QOF

29. Appendix Thank you!
Kontopantelis E, Buchan I, Reeves D, Checkland C and Doran T. The
Relationship Between Quality of Care and Choice of Clinical Computing
System: Retrospective Analysis of Family Practice Performance Under
the UK’s Quality and Outcomes Framework. BMJ Open. In print.
This project is supported by the School for Primary Care Research
which is funded by the National Institute for Health Research (NIHR).
The views expressed are those of the author(s) and not necessarily
those of the NHS, the NIHR or the Department of Health.
Comments, suggestions: e.kontopantelis@manchester.ac.uk
Kontopantelis Clinical Computing Systems and QOF