SlideShare a Scribd company logo
1 of 78
Download to read offline
Benefits Realization
Study
Medical Records in
Ontario
Final
eHealth Ontario
January 2013
Benefits Realization
Study for Electronic
Medical Records in
Ontario
Final Report to
eHealth Ontario
Benefits Realization
for Electronic
Medical Records in
Benefits Realization Study for EMRs in Ontario
PwC
Acknowledgements
This study would not have been possible without the support of Steering Committee members and several clinical
leaders who participated in this study and facilitated access to their clinics and colleagues. We thank them and their
teams for their commitment of time, knowledge and expertise. Their insights and first-hand experience in
pioneering and advancing the use of electronic medical records in primary care settings in Ontario provided a rich
study environment to better understand the current and potential benefits that can be realized through the use of
electronic medical records in Ontario.
Clinical Leaders Steering Committee Members
Dr. David Barber
Queen’s Family Health Team
Emmanuel Casalino
Senior Director, Physician eHealth Program
eHealth Ontario
Dr. Sonny Cejic
Commissioners West Family Health Organization
Dr. Anne Duvall
Peer Leader OntarioMD
Barrie & Community Family Health Team
Dr. Anne Duvall
Barrie & Community Family Health Team
Dennis Ferenc
Director, Funding, Reporting and Change Management
OntarioMD
Dr. Sanjeev Goel
Wise Elephant Family Health Team
Simon Hagens
Director, Benefits Realization & Quality Improvement
Canada Health Infoway
Dr. David Kaplan
North York Family Health Team
Dr. David Kaplan
Primary Care Physician Lead Central LHIN
North York Family Health Team
Dr. Stephen McLaren
Markham Family Health Team
Dr. Wei Qiu
Director, EMR Adoption and Benefit Realization
eHealth Ontario
Christine Sham
Manager, eHealth Liaison Branch
Ministry of Health and Long-Term Care
Patricia Sullivan-Taylor
Manager, Primary Health Care Information
Canadian Institute for Health Information
Benefits Realization Study for EMRs in Ontario
PwC
Table of Contents
Acknowledgements 2
1. Executive Summary 1
2. Introduction 10
3. Methodology 12
3.1 Phase 1: Validation and Evolution of EMR Benefits Realization Framework 12
3.2 Phase 2: Analysis of Benefits Realization 14
3.3 Study Limitations 19
4. Case Study Results 21
4.1 Provider Survey Results 21
4.2 Indicator and Interview Results 24
5. Modeling and Forecasting 33
5.1 Diabetes Management 33
5.2 Increased Influenza Immunization Rates 35
5.3 Increased Colon Cancer Screening Rates 37
5.4 Staff Time Reduction Spent on Administrative Tasks 39
5.4 Summary Findings 41
6. Discussion 43
6.1 Laboratory Management 43
6.2 Communication and Coordination of Care 44
6.3 Chronic Disease Management 45
6.4 Health Promotion, Screening and Prevention 47
6.5 Efficiency 48
6.6 Medication Management 49
7. Recommendations 51
8. Concluding Remarks 54
9. References 55
Benefits Realization Study for EMRs in Ontario
PwC
Appendix A: Interviewee List
Appendix B: Consultation Guide
Appendix C: Revised EMR BR Framework
Appendix D: EMR BR Framework- Case Study Indicator Subset
Appendix E: EMR Maturity Model Criteria Description
Appendix F: Organizational Survey
Appendix G: Provider Survey
Appendix H: Site Visit Interview Guide
Benefits Realization Study for EMRs in Ontario
PwC 1
1. Executive Summary
Electronic medical records (EMRs) are a key enabler of health system transformation, with the potential to achieve
widespread benefits, including: delivering improvements in patient care processes; enabling positive health
outcomes at individual and population levels; creating efficiencies; and reducing costs.
Ontario’s Action Plan for Health Care (Ministry of Health and Long-Term Care, 2012) defines “faster access to
stronger family health care” as a key imperative, and also focuses on improving the overall health of the population
while ensuring that the right care is delivered at the right time, in the right place. In addition, the government’s
transformative Excellent Care for All Act (S.O. 2010, c14) will ensure that Ontarians receive health care of the
highest possible quality and value, and puts the needs of patients first, placing greater accountability on providers
to ensure that the best available evidence is used to make decisions about patient care. Most recently in December
2012, the government announced two very impactful changes to the health system, including the signing of the
Physician Services Agreement with the Ontario Medical Association the creation of Health Links, placing primary
care providers at the centre of the health system and ensuring that patients receive faster care, spend less time
waiting for services and are supported by a team of health care providers at all levels of the health care system
(Ministry of Health and Long Term Care, 2012).
With these and other initiatives underway, it is clear that Ontario’s health system will be undergoing significant
transformation, and changes in the delivery of primary care figure prominently in the government’s strategic
initiatives. With 80% of health care encounters occurring in primary care settings (Canadian Medical Association,
2011), the vast majority of patient data is collected and managed at the primary care level, and the transformative
changes to be undertaken will be reliant on information management programs and tools. The broad and mature
adoption of electronic medical records (EMRs) by primary care providers and their staff will support the required
transformation and the realization of benefits such as improvements in patient care, positive health outcomes at
individual and population levels, efficiencies for providers and system-wide cost reductions. The strategic and
ongoing focus on advancing the availability and use of electronic medical records (EMRs) across the province is
essential.
A key lever in this strategic transformation is the Physician eHealth Program (PeHP), which is now in its fourth
year investing in community-based physician offices to support the adoption and use of EMRs. The PeHP has
achieved several outcomes in the past four years, including the enrolment of over 9,000 physicians in the EMR
Adoption Program. Having achieved a critical mass of EMR adoption across the province, the PeHP has further
developed its focus on benefits realization in 2012 and beyond, opting to undertake a study to identify the nature of
the benefits realized through program investments to date, and those that can potentially be realized with
continued investment and broader adoption of EMRs. To further understand and assess the potential benefits to
be realized, PeHP engaged PwC to conduct a benefits realization study with a focus on:
 The benefits that a select number of advanced users of EMRs in primary care practice settings sites have
realized, directly and indirectly through EMR adoption and usage; and
 The potential benefits that may be realized through widespread, mature adoption of EMR use across
Ontario in the future.
Methodology
The methodology for the study was designed by PwC in consultation with the PeHP team to articulate current and
potential benefits from the use of EMRs and recommendations for widespread EMR benefits realization across
Ontario. Two phases of activities from August to December 2012 were undertaken to complete this study:
Benefits Realization Study for EMRs in Ontario
PwC 2
 Phase 1 entailed the validation and evolution of a comprehensive framework for assessing the benefits
realized in Ontario as a result of EMR use in primary care settings; and
 Phase 2 focused on the analysis of current and potential benefits realized across Ontario through increased
and sustained use of EMRs.
The work completed during Phase 1 set the stage for successful evaluation while simultaneously building the
necessary support among stakeholders through validation of the existing benefits realization (BR) framework. The
approach emphasized the purpose of the framework and the effective positioning of potential benefits that can be
realized by patients, providers and the health system.
Of the original indicators, 21 were identified as priority indicators, laying the foundation for a longer term BR
Framework and measurements that may be used for future analysis. The priority indicators were identified,
specifically for assessment in Phase 2 of the Benefits Realization study based on known available EMR
functionality, maturity of use and feasibility of measurement at case study sites. Based on the current availability of
information and the maturity of EMR users, a set of indicators was selected to assess benefits through a case study
approach at six primary care practices across Ontario. Following the case studies, the set of indicators was further
distilled to reflect data availability and relevance, and a final selection of 11 indicators in five categories was used to
evaluate benefits:
 Laboratory Management,
 Communication and Coordination of Care,
 Chronic Disease Management,
 Health Promotion, Screening and Prevention, and
 Efficiency.
Phase 2 focused on assessing both current and potential benefits realized through EMR use. Case studies were
designed to understand the benefits realized through the use of EMRs in the six selected clinics across Ontario that
were identified as high performing users of EMRs. While the emphasis of the study was to understand both
quantitative and qualitative processes and outcomes as per the indicators defined in the BR framework, due
consideration was given to identifying relevant insights, experiences and lessons learned that supported benefits
realization and demonstrate the role of EMRs (both direct and indirect) in realizing these benefits. Emerging from
the cases studies was a sixth category of study (Medication Management) and for which qualitative benefits have
been discussed.
Evidence for the case studies was collected from surveys, interviews, direct observation and data extraction from
EMRs. Following the collection of data from the Case Study sites, the PwC team undertook a modeling exercise to
forecast the potential benefits of EMR use in Ontario in five to ten years. Benefits were extrapolated from selected
indicator values collected during the site visits and further validated and substantiated with peer reviewed
literature. The modeling involved three steps:
1. A comparison of current values of indicators for case study sites considered high performing, or advanced
users of EMRs and those for other practices in Ontario.
2. An assessment of the relative benefit realized through advanced EMR use by the case study sites compared
to other practices in Ontario.
3. Extrapolation of relative potential benefits to the entire province, providing benefit estimates (such as
potential avoided costs or quality of care outcomes), if all providers in Ontario adopted an EMR and
achieved similar results to the high performing users of EMRs examined in the case studies practices.
Benefits Realization Study for EMRs in Ontario
PwC 3
Case Study Results
The surveys of providers at case study sites indicated very high support for benefit statements attributed to EMRs.
Key results of the provider surveys include the following:
 Quality of Care: 93.0% of survey respondents strongly or moderately agreed with eight statements
presented for benefits of EMR use related to quality of care;
 Communication and Coordination of Care: 98.0% of survey respondents strongly or moderately agreed
with three statements presented for benefits of EMR use related to communication and coordination of
care;
 Efficiency of Practice: 81.5% of survey respondents strongly or moderately agreed with nine statements
presented for benefits of EMR use related to efficiency of practice; and
 Patient Experience: 94.4% of survey respondents strongly or moderately agreed with two statements
presented for benefits of EMR use related to overall patient experience.
Indicator and Interview Results
As described above, the priority indicators that were identified in Phase 1 were assessed for each of the six case
study sites. Findings from indicators and interviews were reviewed together with a sixth category, Medication
Management, which emerged in discussion with several providers during the interviews.
Key results that were identified through the case studies are as follows:
Turnaround time for lab result availability to receipt in EMRs has declined from as much as 5 days to virtually
instaneously. For most study sites, a minimum of 50% decrease in turn around time was reported with results
returned into their EMR. This timely access to test results affords the opportunity to expedite referrals and improve
access to care, and to make timely decisions related to treatments. In addition, physicians perceived the EMR to
improve ordering efficiencies.
The time to receive discharge summaries after patient discharge has declined from as much as 14 days to
virtually immediately where EMRs are integrated with tools such as the Hospital Report Manager or through
direct connectivity with hospitals. These reductions in time spent waiting for discharge summaries and referrals
expedite and facilitate the coordination of care such that patient needs can be addressed in a timely fashion.
The time from referral decision to when the referral is sent to specialist has declined from as much as 7 days to
less than 1 day. Some physicians were able to complete the referral and send it to the specialist with the patient in
the room. Overall, it was observed that the EMR improves physicians’ ability to make timely referrals, expediting
the care process and facilitating inter-office communications.
100% of the care team members have remote and local access to EMRs. This access to information at any time
and in any place was noted by all as a tremendous asset to providers and their patients, improving the ability to
communicate within clinics and often with providers outside of clinics, with overall results including improvements
in the efficiency of patient care.
Up to 70% of diabetic patients, 18 years and over, have an HbA1c level of 7% or less. With the ability to identify a
target diabetic population, physicians and care teams are better able to develop care plans tailored to populations.
Overall, provider survey respondents agreed that EMRs improve the management of chronic diseases. 92.5% of
survey respondents also reported that the EMR system supports patient education. For example, the EMR provides
access to handouts, references or tools to trend patters for BMI, blood sugar, labs. Collectively, the educational
materials and tools help patients better understand and manage their chronic conditions.
Benefits Realization Study for EMRs in Ontario
PwC 4
Up to 80% of practice populations received an influenza immunization, up to 56% of practice populations had a
screening test ordered for colon cancer, and up to 82% of practice populations have had their blood pressure
measured by their primary care provider within the last 15 months. Provider survey results emphasize the role of
the EMR system in promotion, prevention and screening activities. Provider survey results report that 92.6% of
survey respondents agree that EMRs improve patient safety and the proactive monitoring of overdue tests/ exams.
It should be noted that the provincial target for colorectal cancer screening is 40% (Health Quality Ontario, 2012)
and that the values reported by all the case study sites exceeded the provincial average.
Up to 50% less time was required by high performing practices to complete clinical/administrative
documentation by using pre-populated templates, forms and stamps. Overall, 77.7% of survey respondents agreed
with the statement that EMRs “improve the efficiency (reduction in effort) of my practice” and of those 44.4%
strongly agreed with the statement while 33.3% moderately agreed. These survey responses mirror the comments
collected during the interviews on overall practice efficiencies.
Up to 89% of survey respondents agree that EMRs support patient safety through the ability to identify patients
on prescriptions that have drug recalls. Physicians and care team members reported an important patient safety
benefit of the EMR is the ability to quickly extract patient lists for specific medications that have been recalled.
Additional patient safety benefits were identified in relation to improved legibility of prescriptions.
Modeling and Forecasting
A model was developed to forecast potential benefits and “the art of the possible” in Ontario if all providers were
adopting and using EMRs at the same level as the advanced EMR users in the case study sites. Benefits were
extrapolated from indicators collected through the case studies, and supported with peer reviewed literature.
Four indicators were selected from the BR framework for forecasting based on the availability of data for each
indicator at the various sites as well as the availability of supporting evidence to extrapolate these indicators to
benefits for Ontario. These include:
 Chronic Disease Management - Improved diabetes management
 Health Promotion, Screening and Prevention – Increased influenza immunization rates
 Health Promotion, Screening and Prevention – Increased colon cancer screening rates
 Efficiency - Staff time reduction spent on administrative tasks
Each indicator along with its associated benefit and benefit estimate is provided in the table that follows. Although
the selected benefits forecast only a portion of benefits expected from EMRs and that a variety of contributing
factors influence the values of these indicators (e.g. financial incentives to increase screening rates), they indicate
substantial potential benefits to be realized if all providers in the province were to become high performing users of
EMRs in 5 – 10 years.
Benefits Realization Study for EMRs in Ontario
PwC 5
Summary of Forecasted Benefits
Indicator Qualitative Annual Benefits Potential Annual Financial
Impact by 2017
Percentage of patients, 18 years and
over, with diabetes mellitus in whom
the last HbA1c was 7.0% or less (or
equivalent test/reference range
depending on local laboratory) in the
last 15 months
Patient and health system benefits
from management of complications
and co-morbid conditions arising
from diabetes, including:
- 566 fewer foot amputations
- 341 fewer cases of ESRD
- 17,400 fewer MI cases
- 3,100 fewer stroke cases
$125 million
($17 M from reduced foot
amputations, $26 M from
reduced kidney disease, $44M
from fewer MI, $38M from
fewer strokes)
Percentage of patient population, age
65 and older, who received an
influenza immunization
Patient and health system benefits
related to illness prevention,
avoidance of clinic and/or hospital
visits
$40.6 M in reduced health care
system costs
Percentage of practice population,
age 50 to 74, who had a screening
test ordered for colon cancer
Identification and diagnosis of
cancers and malignancies at early
stages for improved prognosis and
quality of life for patients
- 220 fewer Ontarians diagnosed
with stage 4 cancer;
- 217 fewer Ontarians diagnosed
with stage 3 cancer;
- 267 fewer Ontarians diagnosed
with stage 2 cancer;
- 703 additional Ontarians with no
cancer or stage 0/stage 1 cancers
$38.0 M in reduced colon cancer
treatment costs
Percentage reduction in time spent
on administrative tasks
Increased staff capacity through a
reduction of 1.4 million hours that
can be allocated to other tasks,
including the provision of patient
care
$40.0 M in increased staff
capacity
Discussion
Through an approach that has focused on case studies of advanced EMR users in Ontario and forecasting of further
potential benefits, this study has demonstrated that Ontario’s investments in EMR to date have yielded noteworthy,
tangible benefits. These benefits are diverse and have accrued to patients, providers and the broader health system.
Case study participants provided a unique vantage point to convey valuable insights into benefits from their first-
hand experience in pioneering and optimizing the use of a variety of EMR tools and functionalities across their
patient care teams.
While the earliest benefits have been shown to be realized by providers and their patients, modeling and forecasting
has demonstrated that continued investment in EMRs across the province can heighten the impact of EMR use on
the broader health system, with the potential for EMRs to have very significant direct and indirect impacts.
Benefits Realization Study for EMRs in Ontario
PwC 6
While it is acknowledged that the potential benefits of EMRs are broad, the current study focused on six categories
with key benefits discussed below.
1. Laboratory Management
EMRs provide clinicians with more timely access to laboratory information, aiding care decisions
and enhancing the patient experience. The ability to receive lab results through direct transmission to EMRs
has reduced the time to receive those results by 50% on average (compared to a paper-based environment).
Consequently, clinicians are now well enabled to respond to results quickly and effectively. This timely access to
test results, while either in their clinics or from remote locations, affords the opportunity to make timely decisions,
provide prompt and appropriate care, expedite referrals, and improve patients’ access to care. The value of having
comprehensive lab information in a timely manner through all potential sources of lab information provides a
comprehensive profile for patients to form a basis for clinical decision-making. In addition, the increasing
prevalence of patient portals linked to EMRs, patients and/or proxies can have online access to results, allowing
them to review lab results when and where it is convenient for them. These initiatives can improve patients’
experiences with their care, and provide them with some comfort, understanding and ownership in care processes.
2. Communication and Coordination of Care
EMRs facilitate improved scheduling and coordination of patient visits, improving access to care.
EMRs provide physicians and care team members with the improved ability to schedule patient visits, improving
patients’ access to care and efficiency of the care team. Physicians and team members reported that improvements
in scheduling and organization facilitate their ability to hold same day appointments open, improving access to
care. Ontario’s Health Action Plan (Ministry of Health and Long-Term Care, 2012) identifies improved availability
of same-day appointments as a key focus in improving access to primary care.
EMRs improve the availability and sharing of information among interdisciplinary team members
and enhances quality and efficiency of care. The ability to access patient information at any time and in any
place was noted by all as a tremendous asset to providers and their patients. EMRs also improve the ability to
communicate within clinics and often with providers outside of clinics, with overall results improving the efficiency
of patient care. Quality of care, and the patient experience overall, is improved for patients by ensuring all
providers have access to the same patient information.
EMRs facilitate the sharing of information with specialists, thereby improving the continuity and
efficiency of care. EMRs support the ability to make much more informed and efficient referrals to specialists.
With the ease and improved efficiency of making referrals (an approximate reduction in time of 85% to make a
referral), the EMR allows primary care clinicians to quickly provide the specialist with key pieces of information
needed to understand the patient’s condition.
Expedited delivery of hospital reports to EMRs facilitates timely and appropriate care. Case study
sites reported a reduction of 85% in time spent waiting for discharge summaries as a result of the transfer of this
information to EMRs. The timely access to this information can further reduce patients’ wait times for required
post-discharge care, and allow providers to put appropriate follow-up care into place in the out-patient setting.
3. Chronic Disease Management
EMRs are a necessary and effective tool to manage the health of defined patient populations. EMRs
are being used increasingly by clinicians to manage the health of patient populations, such as patients with
diabetes, chronic obstructive pulmonary disease (COPD), and others. Physicians and their care teams are
increasingly relying on their EMRs to effectively manage the care of patients with chronic conditions that are costly
Benefits Realization Study for EMRs in Ontario
PwC 7
to the system overall and have many related co-morbidities constraining health system capacity. It was evident that
case study sites were using their EMRs for these purposes as well; for example, clinicians reported the ability to
identify patients with specific conditions and plan and monitor their care, and identified this as a very significant
benefit of EMRs. Clinicians reported a high degree of willingness and enthusiasm to better manage the health of
defined patient populations in their practice through the support of their EMRs, and acknowledged that in the
absence of EMRs this undertaking would be quite costly and time consuming.
EMRs provide valuable tools to help both care providers and patients with care management and
education. Engaging patients through education and care planning activities ensures that they are active
participants in their health. Patients that are better informed on what their target values for key indicators (e.g.,
HbA1c) should be are better able to keep their conditions under control. As data capture and reporting capabilities
improve, EMRs will further allow practices to identify a baseline and trend information to ensure their diabetic
cohort of patients are monitored for those clinical needs which require careful management and are prone to
downstream co-morbid complications.
The broad and mature use of EMRs can reduce the costs and burden of illness associated with
caring for Ontario’s growing diabetic population. Through the modelling and forecasting exercise,
potential savings and a reduction in complications associated with diabetes were identified. These complications
and illnesses include foot amputations, diabetic kidney disease, stroke and myocardial infarction, all of which
typically require acute care hospitalization. While yet to be realized, EMRs are a contributing factor to potential
savings related to diabetic-related illnesses that are in the range of $125 million annually by 2017. Diabetes must
be actively managed in the community and primary care settings and with active patient participation, in order to
complications such as those forecasted above. Without EMRs and related enablers, it would be very challenging to
do so, given the support that EMRs provide to actively identify diabetic patients, keep their conditions under
control, and communicate on an ongoing basis.
4. Health Promotion, Screening and Prevention
EMRs allow clinicians to survey patients and to proactively arrange screening and prevention
activities, while concurrently improving the efficiency of preventative care. While several reporting
capabilities related to screening and prevention are in their early stages of development, advanced users of EMRs
indicated that they were able to generate information to identify and communicate with patients for preventative
purposes. Without the EMR, this type of prevention activity is much more complicated, requiring manual and
time-consuming chart reviews. Now, patients are contacted easily and in a timely fashion for preventative care.
The potential benefits of increased screening and prevention (including vaccination) activities over time can have a
tremendous impact on the health of the population as a whole, and on the sustainability of the health system.
The widespread use of EMRs can increase the rates of influenza vaccination and yield potential
related health system savings. The forecasted reductions in illness and costs are highly dependent on the
mature, proactive use of EMRs on a province-wide basis to identify those at risk, to facilitate communication to
encourage those patients to receive the flu shot, and to track compliance. The management of influenza must be
actively managed in the community and primary care settings and with active patient participation and willingness
to receive vaccination. EMRs enable this challenge, with many interviewees discussing their ability to vaccinate a
broader group of patients by leveraging information available in their EMRs.
Use of EMRs can support the prevention of colon and other cancers through improved screening
rates and other preventative care. Case studies revealed that EMRs can greatly facilitate the ease with which
clinicians manage and deliver preventative care for their patients. Although preventive care is a relatively new (and
in some cases, advanced) area of EMR use for many clinicians, the ability to identify patients requiring screening
Benefits Realization Study for EMRs in Ontario
PwC 8
and in turn receive and act upon results is possible with EMR use today, and many provincial organizations (e.g.,
Cancer Care Ontario) recognize the power and potential of EMRs to support preventive care. Modeling and
forecasting activities suggested that the potential costs that can be realized through the prevention of four stages of
colon cancer through advanced EMR use is in the range of $38 million annually by 2017. By ensuring that
screening protocols and alerts are incorporated in all EMR specifications, there will be an increased ability to
identify those patients requiring screening on a widespread basis.
5. Efficiency
EMRs facilitate clinical transformation in the primary care setting, improving the effectiveness and
efficiency of clinical and administrative activities. The introduction of EMRs into the primary care setting
has the potential to be transformative in nature. EMRs enable changes in workflow and clinical decision-making,
and can greatly improve the way that clinicians interact with their patients on a regular basis. Benefits associated
with the effectiveness and efficiency of patient encounters and the general flow of patient activity allow greater
throughput and access to care, including same day visits. These benefits have tremendous impact on patients,
providers and the system as a whole, by reducing wait times for primary care, allowing providers to see more
patients daily, and reducing the number of unnecessary visits to hospitals.
EMRs improve the productivity of administrative staff. It is evident that there is some productivity lost in
the early stages of adoption of EMRs with many clinicians and administrators, with many reporting that there is a
period of approximately one year while all grow accustomed to working with electronic records. However,
interviewees and the forecasting model both suggest that there are significant opportunities to improve the overall
capacity among clinic staff, particularly among administrative staff and/or nurses who were previously spending
time doing administrative tasks. The greatest value in this increased capacity lies in the opportunities and potential
benefits associated with redirecting time from non-value added tasks, to those that improve efficiency, allow for
direct patient interaction, and improve the overall patient experience. The reductions in the time required for
administrative tasks through the use of EMRs, estimated at 50%, was used to model the potential benefits if EMRs
were broadly and maturely used across Ontario. The exercise estimated that approximately 1.8 million hours or a
possible $40 million could be saved annually by 2017, providing increased capacity for clinic staff.
6. Medication Management
EMRs support the ability to rapidly identify impacted patients when drug warnings are issued,
improving patient safety. A tremendous benefit of EMR use identified by many providers through the course of
the study was the ability to identify large numbers of patients to whom certain drugs have been prescribed. These
patients can be very quickly identified, and alternate means of treatment can quickly be administered, preventing
any downstream implications. As such, EMRs can have a tremendous impact on quality of care, and most
importantly, patient safety. The ability to quickly target these patients and act is almost impossible in a paper-
based clinic environment.
Access to complete medication profiles in the EMR increases efficiency and improves the accuracy
of medication management. With accurate and complete medication profiles in EMRs, clinicians are able to
quickly and accurately manage care. Prescriptions are easily monitored, and patient concerns can be addressed.
There are also potential costs avoided by the health system due to unnecessary hospitalization from adverse
medication effects.
Electronic prescribing and renewals via EMRs have improved medication management efficiencies
and patient safety. The sharing of medication information across the care team has resulted in greater
efficiencies in the patient care process and improved patient safety. Perhaps the most significant change noted by
physicians has been the generation of a printed prescription, eliminating error-prone handwritten prescriptions
Benefits Realization Study for EMRs in Ontario
PwC 9
and reducing the need for delays arising from “call backs” from pharmacists to physicians seeking clarification on
prescriptions. With EMRs, this risk to patient safety and disruption in workflow is avoided.
Recommendations
Findings from this study provide compelling evidence to continue to advance EMR adoption and maturity across
Ontario. The benefits that have been demonstrated by EMR use in the selected case study settings and the
accompanying forecasting of province-wide benefits demonstrate the “art of the possible” for Ontario. With the
implementation of a number of focused recommendations, the potential for wide ranging and transformative
benefits of EMR use can be further realized by providers, patients and the health care system as a whole.
The following five recommendations are presented to policy-makers, funders, implementers and adopters of EMRs
in Ontario in support of continued benefits realization.
1. As an essential enabler and one of many important health information technology tools to
improve care delivery and related patient outcomes in primary care settings and beyond,
continued investments in EMRs should be made to ensure broad adoption and realization of
benefits across Ontario. Through the course of the study it was widely acknowledged that without the use
of EMRs, the ability to realize the identified benefits is compromised. Indirectly and directly, EMRs are critical
enablers of enhanced patient care. Continued investment in EMRs and increased physician participation in the
EMR Adoption Program are essential.
2. Continue to support increased maturity of use among current and future adopters of EMRs.
The effective realization of benefits is highly supported by EMR maturity (defined as the level of adoption and
use of the EMR in the practice setting). A continued focus in advancing EMR maturity among users will
contribute to a greater diversity of benefits with system-wide impact.
3. Continue to invest in effective change management strategies and user support that extends
beyond the initial period of EMR implementation. Mature use of the EMR requires access to training
when and where it is needed by all types of users, and should be available well beyond the initial
implementation phase as users transition through the “adoption curve”. For example, following an initial
period of use that allows clinicians to master essential EMR functions, training could be further made available
to address and support more sophisticated needs associated with reporting and analytics for population-based
planning and care.
4. Improve the management of information within and across patient care settings through
focused efforts related to interoperability of systems, improved quality of data, and the flow of
data across care settings. The ability to achieve advanced use and benefits of EMRs will be supported by
increased systems integration, improved quality of data, and improved sharing of data across care providers.
Improvements in information management through initiatives such as OntarioMD’s Hospital Report Manager
and CIHI’s Voluntary Reporting System are showing promising benefits, and similar initiatives should be
encouraged.
5. Continue to invest in focused benefits realization studies. Focused studies will afford the opportunity
to measure more of the indicators that were defined in this study as part of the Benefits Realization Framework
and have a greater understanding of the full scope of current and potential benefits realized by EMRs in
Ontario.
Benefits Realization Study for EMRs in Ontario
PwC 10
2. Introduction
Health care systems across Canada and around the world are undergoing transformations to improve quality of
care, access to care, value for money, and the patient experience (Commonwealth Fund, 2011). In Ontario, there is
an increasing emphasis on ensuring that these attributes are all central to the delivery of primary care, providing
the foundation of a strong health care system.
Ontario’s Action Plan for Health Care (Ministry of Health and Long-Term Care, 2012) defines “faster access to
stronger family health care” as a key imperative, and also focuses on improving the overall health of the population
while ensuring that the right care is delivered at the right time, in the right place. In addition, the government’s
transformative Excellent Care for All Act (S.O. 2010, c14) will ensure that Ontarians receive health care of the
highest possible quality and value, and puts the needs of patients first, placing greater accountability on providers
to ensure that the best available evidence is used to make decisions about patient care. Most recently in December
2012, the government announced two very impactful changes to the health system, including:
 the Physician Services Agreement reached between the Ontario Medical Association and the government in
December 2012 (Ministry of Health and Long-Term Care, 2012), supporting the realization of significant cost
savings and efficiencies across the health system and promoting the use of electronic communications and
consultations to increase access to care; and
 the creation of Health Links (Ministry of Health and Long-Term Care, 2012), placing primary care providers at
the centre of the health system and ensuring that patients receive faster care, spend less time waiting for
services and are supported by a team of health care providers at all levels of the health care system.
With these and other initiatives underway, it is clear that Ontario’s health system will be undergoing significant
change, and changes in the delivery of primary care figure prominently in the government’s strategic initiatives.
With 80% of health care encounters occurring in primary care settings (Canadian Medical Association, 2011), the
vast majority of patient data is collected and managed at the primary care level, and the transformative changes to
be undertaken will be reliant on information management programs and tools. The broad and mature adoption of
electronic medical records (EMRs) by primary care providers and their staff will support the required
transformation and the realization of benefits such as improvements in patient care, positive health outcomes at
individual and population levels, efficiencies for providers and system-wide cost reductions. The strategic and
ongoing focus on advancing the availability and use of electronic medical records (EMRs) across the province is
essential.
In supporting, promoting and accelerating the adoption of EMRs across the province, eHealth Ontario’s Physician
eHealth Program (PeHP) is a key initiative in advancing the province’s transformative agenda and Ontario’s
eHealth strategy and primary goal of establishing and maintaining electronic health records (EHRs) for all of
Ontario’s 13 million residents (eHealth Ontario, 2013). EMRs are a partial and necessary component of EHRs,
which hold all relevant health information about a person over his/her lifetime (Hodge, 2011). Since 2009, the
PeHP has invested in community-based physician offices to support the adoption and use of EMRs. Together with
its delivery agent OntarioMD, the PeHP has achieved several outcomes, including the enrolment of over 9,000
physicians in the EMR Adoption Program. The program is currently focused on:
 Further equipping and enabling community-based physicians with tools to enhance the use of EMRs;
 Maximizing clinical and business value of EMRs;
 Providing change management support and promoting the adoption of best practices; and
 Fostering the evolution and sustainment of EMRs and related benefits.
Benefits Realization Study for EMRs in Ontario
PwC 11
Having achieved a critical mass of EMR adoption across the province, the PeHP has further developed its focus on
benefits realization in 2012, opting to undertake a study to identify the nature of the benefits realized through
program investments to date, and those that can potentially be realized with continued investment and broader
adoption of EMRs. Specifically, the current and potential impacts of the use of EMRs on patients, providers and
the health system are of interest.
To further understand and assess the current and potential benefits to be realized, PeHP engaged PwC to conduct a
benefits realization study with a focus on:
 The benefits that a select number of advanced users of EMRs in primary care practice settings sites have
realized, directly and indirectly through EMR adoption; and
 The potential benefits that may be realized through widespread, mature adoption of EMR use across
Ontario in the future.
In collaboration with the PeHP team, PwC conducted this study through a two-phased approach from August to
December 2012. This report presents the study in its entirety as follows:
 Section 3: Methodology, outlining the methodology applied to Phases 1 and 2
 Section 4: Results, presenting the findings arising from the studies of six primary care clinics considered
high performing adopters of EMRs
 Section 5: Modeling of Benefits, identifying potential system-wide benefits from EMR use
 Section 6: Discussion, reporting on the themes and insights emerging from the results
 Section 7: Recommendations, outlining key actions to be taken in order to realize potential benefits across
Ontario
 Section 8: Concluding Remarks
Benefits Realization Study for EMRs in Ontario
PwC 12
3. Methodology
The methodology for the study was designed by PwC in consultation with the PeHP team to articulate current and
potential benefits, and to develop recommendations for widespread EMR benefits realization across Ontario. Two
phases of activities from August to December 2012 were undertaken to complete this study:
 Phase 1 entailed the validation and evolution of a comprehensive framework for assessing the benefits
realized in Ontario as a result of EMR use in primary care settings; and
 Phase 2 focused on the analysis of current and potential benefits realized across Ontario through increased
and sustained use of EMRs.
3.1 Phase 1: Validation and Evolution of EMR Benefits Realization
Framework
The work completed during Phase 1 set the stage for successful evaluation while simultaneously building the
necessary support among stakeholders through validation of the existing benefits realization (BR) framework. The
approach emphasized the purpose of the framework and the effective positioning of potential benefits that can be
realized by patients, providers and the health system.
3.1.1 External Validation of the Framework
The PeHP program had developed an initial draft of a BR framework, inclusive of several categories of
measurement, hypotheses and sixty indicators. This framework was used as the basis for consultations to obtain
feedback and was the subject of further refinement and validation.
A total of 21 consultations were conducted with a variety of representatives including physicians, family health
teams, the Ministry of Health and Long Term Care, government agencies, Local Health Integration Networks
(LHINs), and other jurisdictional EMR programs (Alberta and British Columbia), (see Appendix A for a complete
list of interviewees). Interviewees were provided with a copy of the interview guide and the draft EMR BR
Framework for review prior to the interview (see Appendix B, Consultation Guide).
Stakeholders were asked to select indicators that could describe whether or not EMRs have had an overall impact
on patients, providers and the health system. A formal system for scoring indicators was not used, however
stakeholders reviewed the indicator list and based on their experience and perspective, were able to identify those
indicators they felt would be meaningful and useful to measure.
3.1.2 Development of BR Framework and Indicators
A workshop was held in September 2012 to review all feedback and define the preferred BR framework and
indicators for the case studies and any future benefits realization studies. Workshop discussions were informed by
insights from consultations, a preliminary BR framework assessment, benefits literature and PwC’s experience with
related benefits realization studies.
At the workshop, project team members from the PeHP, Ontario MD and PwC assessed potential indicators based
on the following criteria:
Benefits Realization Study for EMRs in Ontario
PwC 13
 Relevance: Assessment of the importance of the indicator for the objective/hypothesis question e.g. Does
the indicator support an understanding of the impact of the EMR on quality of care from the patient’s
perspective?
 Feasibility: Assessment of the ease and cost of measurement e.g. is the data currently available?
 Ease of interpretation: Assessment of the ease of interpreting the indicator, e.g. Is the indicator well
understood by multiple stakeholders, without requiring extensive explanation?
 Traceability to the EMR: Assessment of whether the indicator can be directly attributed to EMR use e.g.
is there a clear and demonstrable correlation between the use of EMRs and the indicator?
Of the original indicators, 21 were identified as priority indicators, laying the foundation for a longer term BR
Framework and measurements that may be used for future analysis. The indicators were also classified into nine
categories of benefits (see Appendix C, Revised EMR Benefits Realization Framework). Priority indicators were
identified, specifically for assessment in Phase 2 of the Benefits Realization study based on known available EMR
functionality, maturity of use and feasibility of measurement at case study sites. A total of 14 indicators were
initially selected for the EMR BR Framework - Case Study Indicator Subset (see Appendix D), with study questions
in related categories including the following:
Table 1 : Case Study Questions
Category Study Questions
Laboratory
Management
Does EMR use reduce lab result turnaround time?
Communication and
Coordination of Care
Does EMR use improve access to information between settings?
Does EMR use facilitate referral to specialists?
Does EMR use facilitate interdisciplinary/team care?
Do physicians access the EMR remotely to provide patient care?
Chronic Disease
Management
Does EMR use improve chronic disease management?
Health Promotion,
Screening and
Prevention
Does EMR use improve preventative services provided?
Efficiency Does EMR use improve efficiency of care?
Benefits Realization Study for EMRs in Ontario
PwC 14
3.2 Phase 2: Analysis of Benefits Realization
Phase 2 focused on assessing both current and potential benefits realized through EMR use. In order to undertake
this assessment, a series of case studies were undertaken within six primary care settings. The information
collected within these sites together with peer reviewed studies served as inputs to a modelling exercise that defined
potential benefits that could be realized with advanced EMR use and adoption across Ontario.
3.2.1 Case Studies
At the core of the second phase were the case studies, designed to understand the benefits realized and key lessons
learned through the use of EMRs in selected clinics across Ontario that were identified as high performing users of
EMRs. While the emphasis of the study was to understand both quantitative and qualitative processes and
outcomes as per the indicators defined in the BR framework, due consideration was given to identifying relevant
insights, experiences and lessons learned that supported benefits realization and demonstrate the role of EMRs
(direct and indirect) in realizing these benefits.
Case Study Site Selection Process
In order to select case study sites, a preliminary list of ten primary care clinics was compiled by the PeHP and PwC
with input from stakeholders, including OntarioMD. Each potential site was assessed based on the following
preferred criteria.
 EMR Maturity/Experience
 Location / Geography / Site Type
 Vendor System and Integration
 Available and Accessible Documentation
 Innovation
 Willingness to Participate
1. EMR Maturity/Experience: The case study site will ideally have an EMR in use by clinicians for at least
three years and demonstrate mature use of the EMR. The OntarioMD EMR Maturity Model (EMM) illustrated
in Figure 1 was used as a guide to provide information on EMR effectiveness and to identify sites that were
considered “high performing”, with the ability to measure the impacts on the various indicators. See Appendix
E for a more comprehensive description of the EMM.
Benefits Realization Study for EMRs in Ontario
PwC 15
Figure 1: OntarioMD EMR Maturity Model
2. Location / Geography / Site Type: To ensure representation from across Ontario and different clinic types,
geographic factors were considered e.g. urban and rural, in addition to different primary care clinic types, e.g.
small practice groups, large groups.
3. Vendor System and Integration: Representation from clinics using a variety of Ontario-certified vendor
systems was considered. Another site consideration was to select those with various degrees of connectivity
and interoperability (connectivity with OLIS, and access to electronic receipt of hospital reports).
4. Available and Accessible Documentation: The ability of the sites to provide access to documentation that
would facilitate measurement of the indicators in the framework was an important criterion.
5. Innovation: Practices that were seen as being innovative and leading edge in the way they use their EMR
were rated more highly.
6. Willingness to Participate: The willingness of sites to participate in all aspects of the study was an
important criterion.
Candidate sites were contacted and asked to participate. Six practices agreed and were the subjects of the case
studies.
Case Study Tools and Assessment
Case studies were conducted at the selected six practices. The case studies focused on addressing the following high
level study questions as a means of gathering information related to the subset of indicators, advanced application
of EMRs within the clinical setting and lessons learned:
 How are EMRs used in high performing clinical practices?
 What are the best practices and lessons learned with regards to adoption and use of EMRs?
Benefits Realization Study for EMRs in Ontario
PwC 16
Evidence for the case studies was collected from the following four sources. Descriptions of the tools and methods
are described below:
 Surveys (Organizational and Provider)
 Interviews
 Direct observation
 Data extraction
Surveys
Prior to the case study site visits, two surveys were administered by PwC:
1. Organizational Survey: This survey focussed on gathering general background information related to
context, organizational resources and EMR users. It was completed by the Clinical Lead or designate at the
site.
2. Provider Survey: Physicians, physician assistants and nurse practitioners were invited to complete an
online provider survey prior to the site visit. This survey focussed on gathering information on EMR
adoption and current use. All providers were given the option of completing the tool; however, those who
had been selected for an interview were required to complete it.
a. Provider Survey Part 1: EMR Benefits- Part 1 of the survey focused on providers’
perception of benefits realized from EMR use.
b. Provider Survey Part 2: OntarioMD Progress Survey- Part 2 of the survey focused
on the measurement of maturity across key functional areas, and was representative of
OntarioMD’s EMR progress survey.
The survey responses provided valuable information prior to the site visits about each practice, as well as: EMR use
and adoption; EMR maturity; and, general perceptions around EMR use. The information obtained was used to
tailor the interview guide for each individual on-site visit. The Organizational and Provider Surveys are presented
in Appendices F and G respectively.
Interviews
On-site interviews were conducted with clinicians and team members (including administrative staff) whose
responsibilities involved interaction with some aspect of the EMR. For the physicians, the Clinical Lead along with
other physicians (a representative number relative to the size of the practice) were interviewed. A total of thirty-
nine physicians and clinical staff were interviewed across the six sites.
The interviews contained open-ended and closed questions to expand the data gathering and to increase the
number of sources of information. A structured interview guide was developed and emailed to each interviewee
prior to the interview (see Appendix H). As well, the case study subset of EMR indicators was also provided to the
site prior to the site visit (Appendix D). Qualitative data from the interviews were used as a proxy for indicator
measurement where EMR date extraction was not feasible.
Direct Observation
Where possible, the study team observed clinician and staff use of the EMR while on-site. This supported the
information gathering related to workflow and specific use of EMRs in the practice setting. In addition, it served to
validate the information obtained through the surveys and interviews. Because of Personal Health Information
restrictions, the study team did not partake in any activities related to direct patient care or where unique patient
identifiers were visible to the study team.
Benefits Realization Study for EMRs in Ontario
PwC 17
Data Extraction
With assistance from Clinical Leads, IT leads, or data analysts, EMR reports were extracted from the EMR systems
in each site, where possible, in order to measure benefits related to specific indicators.
3.2.2 Final Revisions to Benefits Realization Framework
Following the completion of all case study site visits, the framework and indicators were reviewed and discussed
with members of the Steering Committee. Based on data obtained and feedback received, the framework was
revised for analysis and reporting purposes. The Steering Committee agreed to focus case study analysis on
indicators that would be representative of the BR framework and were most readily accessed and quantified and/or
qualified. The revised BR framework with five categories and eleven indicators is presented in Table 2:
Table 2: Final BR Framework for Case Study Reporting Purposes
Category Indicator
Reference
Number
Indicator
Laboratory Management LM2 Average time between laboratory time of service and test results
available in EMR
Communication and
Coordination of Care
CC1 Average time to receive discharge summary following inpatient
discharge
CC3 Average time from referral decision to when the referral is sent
CC4 % of practices where the care team has access to and uses the
EMR system
CC5 % of physicians who have remote access to EMR and use it for
patient care
Chronic Disease
Management
CDM1 % of PHC clients/patients, 18 years and over, with diabetes
mellitus in whom the last HbA1c was 7.0% or less in the last 15
months
CDM2 % of patient population, age 18 and older, with diabetes mellitus
who received testing for diabetic complications
Health Promotion,
Screening and
Prevention
HPSP1 % of practice population, age 65 and older, who received an
influenza immunization
HPSP2 % of practice population, age 50 to 74, who had a screening test
ordered for colon cancer
HPSP3 % of practice population, age 18 and older, who have had their
blood pressure measured by their primary health care provider
within last 15 months
Efficiency E1 % change in time to complete clinical/admin documentation
Benefits Realization Study for EMRs in Ontario
PwC 18
The values collected from case study sites for the indicators in this framework are referenced and discussed in
Section 4, Case Study Findings.
3.2.3 Modeling and Forecasting of Benefits
An important activity undertaken in Phase 2 was the modeling and forecasting of the potential benefits associated
with broad, mature use of EMRs in Ontario in five to ten years. A fundamental assumption is that this timeframe
will allow widespread and more mature use of EMRs, similar to the advanced users of EMRs today, as represented
by the case study sites. Benefits were extrapolated from indicators collected through the EMRs during the site
visits and further validated with peer reviewed literature. It is to be noted that two studies (Hillestad et al, 2005
and Manitoba Health, 2012) have been referenced on multiple occasions for forecasting purposes, as the scope of
these studies are in alignment with this benefits realization study.
As outlined in Figure 2, development of each of the benefit estimates was an iterative approach. The EMR BR
Framework was used to guide the development of potential EMR benefits. Indicators from the framework were
selected for forecasting based on the availability of data for each indicator at the various sites as well as the
availability of supporting evidence for the extrapolation of these indicators to province-wide benefits.
Figure 2: Approach to Developing Benefit Estimates
Comparative Indicators
The first step consisted of comparing current values of indicators for leading practices and those for other practices
in Ontario. For each indicator selected, minimum, maximum and average values were calculated based on data
collected on site. In order to compare this with the current standard in Ontario, a search of the literature and data
collection agencies (e.g. Statistics Canada and Canadian Institute for Health Information) was conducted. Where
possible, Ontario estimates were used, although in certain instances Canadian or other provincial estimates had to
be used as a proxy. This comparative estimate is an average value of the indicator for the entire province.
The maximum value for each indicator collected on site was used as the projected indicator for Ontario practices in
five and ten years. This was done in order to assess the full potential if all Ontario practices realized benefits similar
to the high performing EMR-enabled practices, as represented by the case study sites.
Associated Relative Benefit
The second step involved assessing the relative benefit for the high-performing practices compared to other
practices in Ontario. Although indicators provide some sense of the expected benefit, they are not realized benefits
per se. For example, the share of the population that is immunized suggests improved health outcomes and avoided
health care costs, however the immunization rate must be extrapolated in order to estimate these actual benefits. A
search of the literature was conducted to attribute a benefit to each indicator. The search prioritized impacts on
resource use or costs.
Indicator estimate
from site visits
Comparative indicator for
average Ontario practice
Associated Relative
Benefit
OntarioExtrapolation OntarioBenefit Estimate
Benefit Indicator
Benefits Realization Study for EMRs in Ontario
PwC
Extrapolation to Ontario Population
In the third step, relative benefits were extrapolated to the entire p
collected for each benefit such as the number of benefit recipients in the province and the costs of various
resources. In order to estimate the benefit
projected to 2017 and 2022. The Ontario data was estimated using a number of sources including Ministry of
Finance population projections and Cancer Care Ontario
estimated using various techniques to extrapolate to provincial levels.
Ontario Benefit Estimate
The obtained benefit estimates provide an indication of the potential avoided costs
providers in Ontario adopted an EMR and
are calculated using identified formulae and financial impacts are presented in 2012 dollars
Selected Indicators
The figure below outlines the indicators selected for modelling by benefit category (Effic
Screening & Prevention and Chronic Disease Management). As mentioned above, other indica
feasibly modeled due to the lack of data extracted
interviews conducted.
Due to study limitations and the inability to model and forecast benefit values for a
that the findings represent a subset of potential province
EMR use. In addition, the above indicators themselves
populations. For instance, benefits associated with influenza immunization were estimated in the over 65 age group
only based on available evidence.
3.3 Study Limitations
Limitations of the study approach include the following:
 Generalization of Findings: The
the representativeness of the findings
considered as directional and a foundation for
through broad and mature use of
• Improved diabetes management
Chronic Disease Management
• Increased influenza immunization rates
• Increased colon cancer screening rates
Health Promotion, Screening & Prevention
• Staff reduction in time spent on administrative tasks
Efficiency
Extrapolation to Ontario Population
third step, relative benefits were extrapolated to the entire province. In order to do this, additional data was
collected for each benefit such as the number of benefit recipients in the province and the costs of various
resources. In order to estimate the benefit five and ten years from now, the number of benefit re
projected to 2017 and 2022. The Ontario data was estimated using a number of sources including Ministry of
Finance population projections and Cancer Care Ontario data. Where local data were not available, proxies were
echniques to extrapolate to provincial levels.
The obtained benefit estimates provide an indication of the potential avoided costs or quality of care outcomes
providers in Ontario adopted an EMR and achieved similar results to the high performing case study sites
and financial impacts are presented in 2012 dollars.
The figure below outlines the indicators selected for modelling by benefit category (Efficiency,
Prevention and Chronic Disease Management). As mentioned above, other indica
led due to the lack of data extracted from EMRs or the inability to derive estimates from the
Due to study limitations and the inability to model and forecast benefit values for all indicators, it is acknowledg
that the findings represent a subset of potential province-wide benefits that can be derived from broad and mature
ition, the above indicators themselves capture a subset of benefits and often for specific patient
. For instance, benefits associated with influenza immunization were estimated in the over 65 age group
imitations of the study approach include the following:
: The intentional focus on six practices with advanced use of
the representativeness of the findings for the province as a whole. Accordingly, the findings should be
a foundation for “the art of the possible” for benefits that can be realized
broad and mature use of EMRs in primary care settings across Ontario.
Improved diabetes management
Chronic Disease Management
Increased influenza immunization rates
Increased colon cancer screening rates
Health Promotion, Screening & Prevention
Staff reduction in time spent on administrative tasks
19
rovince. In order to do this, additional data was
collected for each benefit such as the number of benefit recipients in the province and the costs of various
years from now, the number of benefit recipients was
projected to 2017 and 2022. The Ontario data was estimated using a number of sources including Ministry of
. Where local data were not available, proxies were
or quality of care outcomes, if all
case study sites. Values
iency, Health Promotion,
Prevention and Chronic Disease Management). As mentioned above, other indicators could not be
s or the inability to derive estimates from the
ll indicators, it is acknowledged
wide benefits that can be derived from broad and mature
and often for specific patient
. For instance, benefits associated with influenza immunization were estimated in the over 65 age group
practices with advanced use of EMRs limits
, the findings should be
benefits that can be realized
Benefits Realization Study for EMRs in Ontario
PwC 20
 Sampling: Case study sites were selected based on identified criteria and in consultation with OntarioMD
and the PeHP at eHealth Ontario. Accordingly, the results reflect this sample bias. In addition, survey
participants were identified by the clinical leader rather than being randomly selected, which helped ensure
that interviewees were advanced EMR users but also introduced a sample bias.
 Attribution of Benefits to EMRs: It is difficult to establish a strong correlation between EMR use and
measures among the different practice environments because there are many other variables that
contribute to outcomes measured by the indicators (e.g., interdisciplinary care, well established workflows,
payment models, etc.). Anecdotal evidence was acquired to support findings, and define the nature of the
attribution of EMR use to benefits realized. In many instances, the benefits are indirectly attributable to
EMR use, but it was widely acknowledged that several benefits cannot be realized as quickly nor as
effectively in the absence of EMR use, i.e. in a paper-based clinical setting.
 EMR Maturity: While maturity of EMR use is improving, the use of EMRs is still evolving across
Ontario. Although the case study sites were identified as advanced users of EMR, there was variability in
EMR maturity and connectivity to other systems. Accordingly, interview data was not always comparable
across sites as interviewees were not able to provide a consistent perspective on EMR capability and, as a
result, provided varying levels of detail regarding EMR use and impact.
 Lack of Data Quality, Standardization, and Extraction/Reporting Capabilities: To varying
degrees, each site had difficulty extracting high quality, standardized data from its EMR. Accordingly, it
was difficult for the sites to report EMR impact on specific indicators.
 Lack of Baseline Measures: All case study sites estimated pre-implementation indicator values in the
absence of measured baseline information for the selected indicators. Indicator values prior to EMR
implementation were typically estimated from respondents’ memories and as a result, may not be accurate.
 Ability to Forecast EMR Adoption and Use: The modeling approach assumes that, by 2017, all
community-based practices in Ontario will have adopted EMRs and will benefit from EMRs to the same
degree as the case study sites. The pace of EMR adoption in Ontario over the next five years is uncertain
and will likely not be on a “straight line” basis, i.e., rates of adoption and benefits realization will likely vary
on a year-to-year basis. Although there was a large increase in EMR adoption in Canada from 2009 to 2012
with the Commonwealth Fund Survey reporting an increase from 37% to 56%, a linear increase to 2017 and
in turn 2022 cannot be assumed.
 Limited Evidence for Modeling: The model addresses a limited number of benefits based on data
collected by EMR practices to date and available evidence in the literature. For some indicators, practices
were not able to extract the data from their EMR. For other indicators, although values could be extracted,
there was insufficient evidence regarding the impact of the indicator on cost or quality of care outcomes
(e.g., average time to receive charts after emergency department). In addition, some benefits of EMRs are
not yet reflected by indicators so the benefits cannot be estimated (e.g., immunization in young children).
Benefits Realization Study for EMRs in Ontario
PwC 21
4. Case Study Results
Findings obtained from the six primary care case study sites are reported in the section below. The findings reflect
the quantitative and qualitative benefits that have been measured and realized by the six sites, with data compiled
from provider surveys, site visit data collection as per the selected indicators and in-person interviews. This section
presents results of the provider surveys, followed by indicator and interview results in alignment with indicator
categories.
Following the presentation of findings and modelling and forecasting results in Section 5 that follows, a discussion
is presented in Section 6 with a focus on the broader potential benefits that can be realized through mature EMR
adoption and use across Ontario in primary care settings.
4.1 Provider Survey Results
The survey of 28 physicians, physician assistants and nurse practitioners from the six case study sites focused on
providers’ assessments of the impact of EMRs on quality, communication and coordination of care, efficiency, and
the patient experience in the primary care setting. Results, demonstrating levels of agreement with specific impact
statements, were very positive and are presented in the four bar chart figures that follow.
Figure 3: EMR Impact on Quality
*Statements have been truncated (See Appendix G for complete survey questions).
70.4 70.4
55.6
40.7 48.1
85.2
33.3
55.6
29.6 25.9
37
48.1 37
14.8
51.9
40.7
3.7 7.4 7.4 14.8 7.4
3.73.7 7.4
Enhancesthequalityofcare
deliveredtopatients*
Improvesthemanagementof
chronicdiseases
Improvespatientsafetyand
proactivemonitoring*
Enablesidenitificationof
patientsforchangesin
management*
Improvesthedecision-
makinge.g.decisionsupport
tools*
Supportsremoteaccessand
useforpatientcare
Enablesthepracticetodo
needsplanning*
Enablesthepracticetoaudit
andimprovethepractice
PercentAgreementorDisagreement
Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
Benefits Realization Study for EMRs in Ontario
PwC 22
The eight survey questions posed reflect a very broad definition of “quality”, addressing aspects that include:
 Quality of care delivered
 Chronic disease management
 Patient safety and proactive monitoring of conditions
 Clinical decision-making
 Remote access
 Needs-based planning
 Practice improvement
Very strong levels of agreement were received in response to all of the questions posed, reflecting the significant
range of impact and benefits that providers believe may be directly or indirectly attributed to the use of EMRs. On
average, 93.0% of respondents either strongly agreed or agreed with statements presented for EMR benefits related
to quality of care.
Figure 4: EMR Impact on Communication and Coordination of Care
Questions around the impact of EMRs on communication and coordination of care were more narrowly focused on
internal sharing of information, supporting interdisciplinary care coordination and overall practice’s ability to
coordinate patient care. 100% of survey respondents agreed or strongly agreed that the use of EMRs made a
positive impact on those activities related to communication and coordination of care, indicating the tremendous
impact that EMRs can have in the clinical practice setting.
92.6 92.6
74.1
7.4 7.4
25.9
Improves the sharing of patient
information with providers internal
to our practice
Supports interdisciplinary care
coordination in our practice
Enhances our practice’s ability to
coordinate patient care
PercentAgreementorDisagreement
Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
Benefits Realization Study for EMRs in Ontario
PwC 23
Figure 5: EMR Impact on Efficiency
*Statements have been truncated (See Appendix G for complete survey questions).
Nine survey questions were posed around efficiency, again reflecting a broad definition of “efficiency”, addressing
aspects that include:
 Efficiency of ordering lab tests and prescriptions
 Practice productivity (increased output and reduction of effort)
 Time spent responding to call backs or other pharmacist requests
 Availability of test results (turnaround time)
 Administrative efficiencies
 Preventative care incentives
 Claims management processes
 Management of overhead costs
Overall, 81.5% of survey respondents strongly agreed or agreed with statements presented for EMR benefits related
to efficiency of practice. Findings suggest that while the majority of respondents perceive their EMR system to
positively impact the efficiency of their practice, there is some level of disagreement around the efficiency of certain
aspects of EMR use (vs. a paper-based environment). This is most pronounced around the ability of EMR use to
positively impact claims management processes and the management of overhead costs.
63 59.3
44.4
33.3
77.8 70.4
44.4 44.4
14.8
29.6 33.3
33.3
44.4
18.5
18.5
44.4
29.6
29.6
7.4 7.4
18.5 18.5
7.4 3.7
7.4
18.5
3.7 3.7 3.7
7.4
3.7 7.4
18.5
29.6
Enhancestheefficiencyofordering
labtests,prescriptions,etc.
Improvestheproductivity(output)of
mypractice
Improvestheefficiency(reductionin
effort)ofmypractice
Reducesthenumberofcallbacks
and/ortimespentrespondingto
pharmacistrequests*
Reducesthetimefromwhenalab
testresultisavailabletowhenthe
resultisreceivedbytheEMR
Improvesadministrativeefficiencies
e.g.useofpre-populatedtemplates
Facilitatespreventativecare
incentivese.g.throughcohort
management
Improvestheclaimssubmission
processe.g.decreaseinbillingerrors
Hasfacilitatedareductioninnet
overheadcostse.g.space,office
suppliesetc.
PercentAgreementorDisagreement
Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
Benefits Realization Study for EMRs in Ontario
PwC 24
Figure 6: EMR Impact on the Patient Experience
The two questions posed around patient satisfaction and patient education indicated strong levels of agreement,
with 94.4% indicating that EMRs have a positive impact on both. However, the majority of respondents simply
agreed (rather than strongly agreed) with these statements. This suggests that the large majority of respondents
perceive their EMR improves the overall patient experience, but that there could be other, and perhaps stronger,
contributing factors to patient experience.
Survey results are further referenced in support of the findings presented on the indicators and interviews below.
4.2 Indicator and Interview Results
As described in Section 3, above, the priority indicators that were identified in Phase 1 were assessed for each of the
six case study sites. Data to support the priority indicators were obtained from interviews, provider surveys and
EMR data extractions. Benefits are assessed and presented as per the five categories and indicators identified in the
EMR BR Framework Case Study Subset and in Table 2. In addition, findings and anecdotes from interviews related
to each of the five categories are presented, with selected supporting quotes from providers. A sixth category,
Medication Management, emerged in discussion with several providers during the interviews. Although there were
no indicators studied for Medication Management, the interview findings are presented.
1. Laboratory Management
2. Communication and Coordination of Care
3. Chronic Disease Management
4. Health Promotion, Screening and Prevention
5. Efficiency
6. Medication Management
37 44.4
59.3 48.1
3.7 7.4
Improves patient satisfaction/experience with the
care they receive
Supports patient education (e.g. trending patterns
for BMI, blood sugar levels etc.)
PercentAgreementorDisagreement
Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
Benefits Realization Study for EMRs in Ontario
PwC 25
4.2.1 Laboratory Management
The management of laboratory information was determined to be a key area of interest and a feasible area to assess
in the current EMR environment. Primary care settings are connected (to an extent) to community (private)
laboratories through a direct feed enabling them to receive laboratory results electronically into their EMRs. This
simulates the benefits that will be obtained as the Ontario Laboratories Information System (OLIS) is rolled out in
primary care settings. Some physicians indicated very recent connection to the OLIS but results and/or trends were
not available at this time.
The impact of EMR use on Laboratory Management was assessed through a combination of provider survey results
and interviews for one case study indicator. Results are presented in Table 3 below.
Table 3: Laboratory Management Indicator Results
ID # Indicator Definition Source Indicator Values
LM2
Average time from laboratory time of
service and test results available in
EMR
Interview Pre EMR: Total turnaround ≈12 hours
to 5 days
Post EMR: Real time – 24 hours
Responses from the provider survey indicated that 100% of participants felt that the EMR reduces the time from
when a laboratory test result is available to when the result is received by the EMR. However, indicator values
varied among case study sites. Some case study sites reported having laboratories physically on site. Their pre-EMR
laboratory turn around time averaged 24 hours for routine bloodwork and was generally within 12-24hours with an
EMR. As expected, the time for transmission of results was not a significant factor prior to EMR implementation as
results were commonly “walked over”. However, sites that previously relied on paper delivery of results from
community laboratories and which now receive electronic reports directly into their EMR reported a noticable
difference in the turnaround time post-EMR implementation. Paper-based results were reported to take
approximately 3-5 days, but those sites now report receiving information in “real time” when the result is ready for
distribution. For most study sites, a minimum of 50% decrease in lab result turn around time was reported with
results received electronically in their EMR.
Physicians also perceived EMRs to improve the efficiency of ordering lab tests. Findings from the provider survey
indicated that 63% of respondents strongly agreed that the EMR enhances the efficiency of ordering lab tests,
prescriptions etc. contributing to the overall management of laboratory information.
Additional Interview Findings
Some physicians reported accessing their EMRs remotely in the morning before their clinic day or on weekends to
prepare for the day or week. Consequently, test results requiring immediate follow-up were able to be addressed
sooner by scheduling the patient for a follow-up visit, making a referral, or sending a prescription to the pharmacy
(depending on EMR capability). It was also reported that some physicians are able to access lab reports on their
smart phones, making remote access even easier.
Physicians are able to access lab results in “real time” throughout the day in
many locations. Some physicians indicated that they enjoyed the ability to,
throughout the day, go through their “inbox” (the results also were in the
patient chart) and sign off results as they come in, demonstrating how some
physicians have adopted a change in workflow to manage laboratory
information. Previously, results may have been signed off in batches at the
end of the day; now, physicians can access their EMR in the clinic room, in
their offices or in hallways, enabling them to review and sign laboratory
“I access labs and charts
from many locations
throughout the day. As a
result I can make faster
decisions related to next
steps in patient care”.
Benefits Realization Study for EMRs in Ontario
PwC 26
results from many locations throughout the day.
A few case study sites reported the use of a patient portal in conjunction with their EMRs. Some sites reported that
their patients’ experiences were improved through the sharing of laboratory results via a secure patient portal.
Physicians reported a reduction in patients’ anxiety by enabling them to check their laboratory results themselves
from home, and others indicated that their patient portal allowed them to post comments along with the results
they selected for sharing with patients. This helped patients better understand the context of the result.
4.2.2 Communication and Coordination of Care
Communication among health care providers and coordination of care were both identified as key areas of interest
for the case studies. In particular, the need was identified to better understand how the use of EMRs has impacted
both intra-office and inter-office communications. Study sites were more responsive to discussing and identifying
benefits related to intra-office communication and coordination, as the use of EMRs in supporting inter-office
communication is perceived to be in its very early stages.
Case study findings related to five defined indicators regarding communication and coordination of care are
presented in Table 4.
Table 4: Communication and Coordination of Care Indicator Results
ID # Indicator Definition Source Indicator Values
CC1
Average time to receive discharge
summary following inpatient discharge Interview
Pre EMR: 4 to 14 days
Post EMR: Real time to 48 hours
CC3
Average time from referral decision to
when the referral is sent Interview
Pre EMR: 1 to 7 days
Post EMR: Real time to 1 day
CC4
% of practices where the care team has
access to and uses the EMR system
Provider Survey
Interview
100%
CC5
% of physicians who have remote access
to EMR and use it for patient care
Provider Survey
Interview
100%
For indicators CC1 and CC3, interviewees were able to estimate a pre-implementation value depending on their
personal experiences. While those estimates of pre-implementation values ranged significantly among providers,
the reduction in times quoted were, on average, significantly lower demonstrating reductions in time spent waiting
for discharge information and/or referrals in the range of 85% or more. These reductions in time spent waiting
for discharge summaries and referrals expedite and facilitate the coordination of care such that patient needs can
be addressed in a timely fashion. For indicator CC1, physicians reported that, prior to EMR implementation,
hospital discharge summaries were available within 1-2 weeks of discharge and are now available with tools such as
the Hospital Report Manager or direct connectivity, either in real-time or within two days.
With respect to referrals and indicator CC3 specifically, physicians were able to estimate the average time to
complete a referral letter and send it to the specialist. Physicians widely reported that the referral process1 is much
more efficient with EMRs because of the pre-populated data in the referral forms. Some physicians also reported
using macros or templates to further support the process. This finding was also supported in the provider survey
1 Herein refers to as “from referral decision being made to the reference request letter completed/faxed.”
Benefits Realization Study for EMRs in Ontario
PwC 27
with 70.4% of respondents reporting that EMRs improve administrative efficiencies through the use of features
such as pre-populated templates, forms and stamps. Following the development of the referral letter, there was a
mix of workflow processes reported. Some physicians reported they sent out the referral letter themselves through
the EMR with attachments while other physicians reported consolidating the letter and supporting documentation
to send to a referral clerk/ nurse for submission and tracking. Overall, it was observed that the EMR improves
physicians’ ability to make timely referrals, expediting the care process and facilitating inter-office
communications.
For indicator CC4, 100% of practices reported that their entire care team has access to the EMR. Survey responses
further emphasized the access and importance of EMRs in providing interdisciplinary care and intra-office
communication; 92.6% of survey respondents strongly agreed that the EMR system supports interdisciplinary care
coordination in their practice. In addition, 92% of respondents strongly agreed that the EMR system improves the
sharing of patient information among providers internal to the practice.
For indicator CC5, it was evident at all sites that providers are accessing the EMR and using it remotely for patient
care purposes. 85.2% of survey respondents strongly agreed that EMRs support remote access and use of
information by providers for patient care. The ability to access patient information at any time and in any place was
noted by all as a tremendous asset to providers and their patients. EMRs also improve the ability to communicate
within clinics and often with providers outside of clinics, with overall results improving the efficiency of patient
care.
Additional Interview Findings
Interviews with clinic staff revealed many additional benefits of care coordination
from EMRs. For example, EMRs facilitate the ability of physicians to set aside
dedicated time for same/next day appointments for patients with acute needs.
Physicians and team members reported that improvements in scheduling and
practice organization facilitate their ability to hold same/next day appointments
open. In addition, EMRs provide physicians and care team members with the ability
to view everyone’s schedule to better coordinate care across team members.
With respect to communication across the team, it
was overwhelmingly reported that the illegibility of
hand-written clinical notes was formerly a challenge
when sharing information among team members. Interviewees reported that the
sharing of legible information in the EMR has improved the quality of care delivered.
Team members are now able to communicate with one another in real time, ask
questions and assign tasks among the team. The patient chart is now accessible to
many multiple care team members at the same time, a feature that was not possible
in paper.
Through the use of EMRs, most physicians reported that the information they
provided in referral letters to specialists was more comprehensive, especially for complex cases. It was also
reported that specialists appreciate the extensive clinical information provided as part of the referral and facilitated
by EMRs. Some physicians reported that they were able to quickly select attachments e.g. diagnostic test results, to
send along with the referral letter.
“EMRs improve
scheduling,
resulting in
increased capacity
and organization
of our practice, all
of which results in
increased access
for patients”.
“EMRs improve
transfer of
information to
specialists - it is
easier to provide
consultations with
lots of relevant
information”.
Benefits Realization Study for EMRs in Ontario
PwC 28
Lastly, there was consensus among users of the Hospital Report
Manager (HRM) (whereby dictated and transcribed hospital reports are
transformed into a standards-based data schema that can be directly
updated into the patient’s chart in an electronic format) that it has
decreased the average time to receive hospital reports. The
implementation of HRM has reduced transcription time and
significantly decreased hospital report turnaround time. Some practices
reported receiving hospital reports directly into their EMRs within
thirty minutes of transcription. As a result, proper clinical decisions can
be made more effectively to improve the quality of care for patients.
4.2.3 Chronic Disease Management
Chronic disease management is an emerging area of interest related to EMR use. Case study sites reported that
EMRs facilitate significant and improved changes to the way chronic conditions are managed. Both quantitative
and qualitative findings were significant, with physicians expressing a desire to expand their use of EMRs for this
purpose.
The impact of EMRs on the management of patients with chronic conditions was assessed through a combination
of provider survey results, interviews and EMR data extraction. Findings related to the three defined indicators are
presented in Table 5.
Table 5: Summary Chronic Disease Management Indicator Results
ID # Indicator Definition Source Indicator Values
CDM1
% of PHC clients/patients, 18 years and
over, with diabetes mellitus in whom
the last HbA1c was 7.0% or less in the
last 15 months
EMR Data Extraction
Range: 41 - 70%
CDM2
% of patient population, age 18 and
older, with diabetes mellitus who
received testing for all of the following:
 Hemoglobin A1c
 Full fasting lipid profile screening
 Nephropathy screening
 Foot examination
 Blood pressure measurement
 Obesity/overweight screening
EMR Data Extraction 34%
Overall, provider survey respondents agreed that EMRs improve the management of chronic diseases. Specifically,
70.4% strongly agreed with the statement and 25.9% moderately agreed. 92.5% of survey respondents also reported
that the EMR system supports patient education. For example, the EMR provides access to handouts, references or
tools to trend patters for BMI, blood sugar, and other labs. Collectively, the educational materials and tools help
patients better understand and manage their chronic conditions.
CDM1 and CDM2 were measured across all sites, with values ranging from 41-70% and 34% respectively. These
results demonstrate the ability to identify a target diabetic population and deliver appropriate, evidence-based care.
Physicians and care teams are better able to monitor specific clinical indicators such as HbA1c, and develop care
plans tailored to populations. 85.2% of provider survey respondents agreed that EMRs enable practices to perform
“Through HRM and my EMR, I
receive reports from the local
hospital as soon as they are
prepared. No more calling the
hospital for reports. I have the
information immediately
available to review”.
Benefits Realization Study for EMRs in Ontario
PwC 29
needs-based planning e.g. plan for a specific patient population/community. Without EMRs, this is very time
consuming, costly and a nearly impossible task.
Additional Interview Findings
It was evident in discussions with physicians and care team members that they were all motivated to better manage
the health of various patient populations. There appears to be a movement away from “chair to chair care” with an
increasing emphasis on patient populations. Sites reported that because they are easily able to identify patient
populations, they are able to proactively follow up with patients, schedule recall appointments or send them for
further testing as needed.
Nurse practitioners reported that their practices had many patients with chronic diseases, and they found that the
use of EMR tools, e.g. diabetes template/ flow sheet, was very useful when seeing diabetic patients. Relevant
information was quickly accessible and consolidated, and the flow sheet ensures that best practice guidelines are
incorporated into care delivery. Care team members are aware of what needs to be attended to during the visit,
improving efficiency and what they should be planning for in the coming months in order to manage the condition.
Most physicians and care team members reported using trending or
graphing tools as patient education materials e.g. blood pressure,
weight, HbA1C were commonly trended. It was reported that the ability
to review consult notes, labs etc with patients builds confidence of care
and enhances communication”. In addition, some physicians noted that
they were beginning to communicate with patients (especially those
with chronic conditions) via secure email.
4.2.4 Health Promotion, Screening and Prevention
Keeping Ontarians healthy through health promotion, prevention and screening initiatives is an area of benefits to
be explored through the use of EMRs. This aspect of the study is complementary to that of chronic disease
management, with a focus on targeting at risk populations to improve quality of care.
The impact of EMRs on health promotion, prevention and screening was assessed through a combination of
provider survey results, interviews and EMR data extraction. Findings related to the three defined indicators are
presented in Table 6.
Table 6: Health Promotion, Screening and Prevention Indicator Results
ID # Indicator Definition Source Indicator Values
HPSP1
% practice population, age 65 and
older, who received an influenza
immunization*
EMR Data Extract Range: 52 - 80%
HPSP2
% of practice population, age 50 to 74,
who had a screening test ordered for
colon cancer
EMR Data Extract Range: 51 - 60%
HPSP3
% of practice population, age 18 and
older, who have had their blood
pressure measured by their primary
health care provider within last 15
months
EMR Data Extract Range: 61 - 82%
“For diabetes care I can trend
A1C for several years – you
can discuss and show the
patient how they are doing”.
Benefits Realization Study for EMRs in Ontario
PwC 30
Overall, EMR data extraction was quite feasible for the above selection of indicators, suggesting that case study
sites have been working towards standardizing their data input in order to use the data in a meaningful way to
maintain a healthy population. For each of the three indicators, identified patients who had received appropriate
preventative treatments (as per the indicators) were greater than 50% and as high as 82%. The role of the EMR in
facilitating the identification of patients, communication with those patients, and ordering and/or documentation
of the required testing is very important. Providers indicated that these tasks are greatly enabled by EMRs, and
much more difficult to accomplish in a paper-based environment.
Provider survey results further emphasize the role of the EMR system in promotion, prevention and screening
activities, with 92.6% of survey respondents agreeing that EMRs improve patient safety and the proactive
monitoring of overdue tests/ exams. Enabling case study sites to track their patient populations through the EMR
and compare their indicator values to provincial benchmarks gives them an opportunity to identify areas for
improvement in health promotion, screening and prevention activities.
Sites are in the early stages of generating reports to proactively screen their patient populations, and priority areas
of focus can be linked to financial incentives for preventative care. It should be noted that the provincial target for
colorectal cancer screening is 40% (Health Quality Ontario, 2012) and that the values reported by all the case study
sites exceeded the provincial average. In addition, 44.4% of provider survey respondents strongly agree, while
44.4% moderately agree that EMRs facilitate preventative care incentives e.g. through cohort management for
influenza immunizations.
Additional Interview Findings
All of the case study sites reported that they were motivated to use
EMRs for population health purposes. Physicians and care teams
performed simple searches with minimal parameters to identify target
populations for disease prevention. For example, as part of its family
health team quality improvement plan, one site reported pulling lists of
eligible/ not eligible patients for activities such as cervical cancer
screening, breast cancer screening and colon cancer screening, and
sharing the list among physicians to coordinate screening tests.
Another common example discussed with interviewees was the EMR’s
ability to support efforts to increase influenza immunization rates. Not only have practices adopted processes for
targeting populations and sending reminder letters (or emails where available) but they have also developed and
tested new ways of measuring their flu vaccination rates, including a dashboard that presents vaccination rates
among patients of physician peers.
Some physicians and care team members reported that they were better able to track preventative health tests and
recalls, with the EMR generating reminders for the proactive scheduling of wellness visits. Sites also reported that
they engage patients in community wellness talks through identification of those in target populations, and speak to
them about prevention and promotion topics relevant to their cohort. EMRs afford providers the ability to be very
proactive and interactive with patients in managing their own health.
“EMRs provide the ability to
survey your practice to find
people who are due for
preventative procedures or
care, including finding all
the people who have not had
a pneumovax.”
Benefits Realization Study for EMRs in Ontario
PwC 31
4.2.5 Efficiency
Obtaining a better understanding of the overall practice efficiencies realized through the use of EMRs was a focus
for the case studies. As discussions evolved, it became evident that practice and provider efficiency was a recurring
theme that was applicable to several other discussion categories as well. Consequently, results are presented for
one indicator only in this section, but findings about efficiencies created are also referenced elsewhere in this
report.
Table 7: Summary Efficiency Indicator Findings
ID # Indicator Definition Source Indicator Values
E1
% change in time to complete
clinical/admin documentation by using
pre-populated templates, forms and
stamps
Interview ~ 50% decrease
Overall, 77.7% of survey respondents agreed with the statement that EMRs “improve the efficiency (reduction in
effort) of my practice” and of those 44.4% strongly agreed with the statement while 33.3% moderately agreed.
These survey responses mirror the comments collected during the interviews on overall practice efficiencies.
With respect to clinical and administrative documentation (indicator E1), physicians were able to report overall
that there have been efficiency changes related to certain activities. For example, 92.6% of survey respondents
agreed with the statement that EMRs “enhance the efficiency of ordering lab tests, prescriptions, etc” (63.0%
strongly agree and 29.6% moderately agree). During interviews, physicians reported that the auto-population of
templates (where appropriate, e.g. lab requisition form, s.o.a.p. format encounter notes and others) was an added
benefit and saved time for themselves and their administrative staff in completing fields such as demographics.
For indicator E1, forms (including templates with the ability to auto-populate existing EMR data) and stamps (e.g.
s.o.a.p note or diabetic assessment template used to capture information) were acknowledged as a benefit to clinical
and administrative documentation. However, physicians and care team members found it difficult to quantitatively
assess the impact, with most estimating the time savings at 50% when compared to a paper-based environment.
Survey results support the benefit of EMRs in administrative efficiencies with 70.4% of providers perceiving EMRs
to improve administrative efficiencies such as through the use of pre-populated templates, forms and stamps.
Additional Interview Findings
In addition to the discussions around the above indicator, there was significant
discussion on the efficiencies gained (or transferred to other activities) related
to EMR use. There was a general consensus from interviewees that EMR
features such as reminders allow for more efficient clinical encounters (e.g.
patient physical exams), and not surprisingly, physicians and care team
members reported that their time is not spent “flipping through papers” or
searching for misfiled results. Instead, clinicians reported being able to address
their list of planned activities such as reminding patients they are due for certain tests, procedures, medication
refills, etc.
“I have more efficient
clinical encounters.
The time is spent on
more important things
rather than writing
out prescription
renewals etc.”
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013
Final eHO EMR Benefits Report Jan2013

More Related Content

What's hot

Emerging Technologies for Patient Engagement and Mobile Health
Emerging Technologies for Patient Engagement and Mobile HealthEmerging Technologies for Patient Engagement and Mobile Health
Emerging Technologies for Patient Engagement and Mobile HealthP. Kenyon Crowley
 
Lannes - Improving health worker performance The patient-perspective
Lannes - Improving health worker performance The patient-perspectiveLannes - Improving health worker performance The patient-perspective
Lannes - Improving health worker performance The patient-perspectivelaurencelannes
 
Meaningful Use and the Path to Population Health and Quality in a Transformin...
Meaningful Use and the Path to Population Health and Quality in a Transformin...Meaningful Use and the Path to Population Health and Quality in a Transformin...
Meaningful Use and the Path to Population Health and Quality in a Transformin...Phytel
 
A review of the total knee replacement pathway: Integrated care is quality care
A review of the total knee replacement pathway: Integrated care is quality careA review of the total knee replacement pathway: Integrated care is quality care
A review of the total knee replacement pathway: Integrated care is quality careApollo Hospitals
 
Health IT Summit Houston 2014 - Case Study "EHR Optimization for Organization...
Health IT Summit Houston 2014 - Case Study "EHR Optimization for Organization...Health IT Summit Houston 2014 - Case Study "EHR Optimization for Organization...
Health IT Summit Houston 2014 - Case Study "EHR Optimization for Organization...Health IT Conference – iHT2
 
Michigan Hospital Association Governance meeting
Michigan Hospital Association Governance meetingMichigan Hospital Association Governance meeting
Michigan Hospital Association Governance meetingMary Beth Bolton
 
Leonard Kingwara publication
Leonard Kingwara publicationLeonard Kingwara publication
Leonard Kingwara publicationLeonard Kingwara
 
Digital health: Ontario Hospitals
Digital health: Ontario HospitalsDigital health: Ontario Hospitals
Digital health: Ontario HospitalsTrustRobin
 
NHS Atlas of Variation in Healthcare for People with Liver Disease
NHS Atlas of Variation in Healthcare for People with Liver DiseaseNHS Atlas of Variation in Healthcare for People with Liver Disease
NHS Atlas of Variation in Healthcare for People with Liver Diseaserightcare
 
RGC 2015 Telehealth Report
RGC 2015 Telehealth ReportRGC 2015 Telehealth Report
RGC 2015 Telehealth ReportKarl McKinnie
 
Connected Care: Heightened Imperatives
Connected Care: Heightened ImperativesConnected Care: Heightened Imperatives
Connected Care: Heightened ImperativesCognizant
 
EHR Meaningful Use for Medical Practices
EHR Meaningful Use for Medical PracticesEHR Meaningful Use for Medical Practices
EHR Meaningful Use for Medical PracticesMichael Patrick
 
Meaningful Use Workgroup Stage 3 Recommendations
Meaningful Use Workgroup Stage 3 Recommendations Meaningful Use Workgroup Stage 3 Recommendations
Meaningful Use Workgroup Stage 3 Recommendations Brian Ahier
 
I reland feb 2014
I reland feb  2014 I reland feb  2014
I reland feb 2014 Paul Grundy
 
Cadth symposium 2015 d3 pro presentation apr 2015 - for deb
Cadth symposium 2015 d3 pro presentation   apr 2015 - for debCadth symposium 2015 d3 pro presentation   apr 2015 - for deb
Cadth symposium 2015 d3 pro presentation apr 2015 - for debCADTH Symposium
 
EHR Meaningful Use ONC Policy Commitee June 16 2009
EHR Meaningful Use ONC Policy Commitee June 16 2009EHR Meaningful Use ONC Policy Commitee June 16 2009
EHR Meaningful Use ONC Policy Commitee June 16 2009Michael Duffy
 

What's hot (20)

Emerging Technologies for Patient Engagement and Mobile Health
Emerging Technologies for Patient Engagement and Mobile HealthEmerging Technologies for Patient Engagement and Mobile Health
Emerging Technologies for Patient Engagement and Mobile Health
 
Lannes - Improving health worker performance The patient-perspective
Lannes - Improving health worker performance The patient-perspectiveLannes - Improving health worker performance The patient-perspective
Lannes - Improving health worker performance The patient-perspective
 
Meaningful Use and the Path to Population Health and Quality in a Transformin...
Meaningful Use and the Path to Population Health and Quality in a Transformin...Meaningful Use and the Path to Population Health and Quality in a Transformin...
Meaningful Use and the Path to Population Health and Quality in a Transformin...
 
A review of the total knee replacement pathway: Integrated care is quality care
A review of the total knee replacement pathway: Integrated care is quality careA review of the total knee replacement pathway: Integrated care is quality care
A review of the total knee replacement pathway: Integrated care is quality care
 
Health IT Summit Houston 2014 - Case Study "EHR Optimization for Organization...
Health IT Summit Houston 2014 - Case Study "EHR Optimization for Organization...Health IT Summit Houston 2014 - Case Study "EHR Optimization for Organization...
Health IT Summit Houston 2014 - Case Study "EHR Optimization for Organization...
 
Michigan Hospital Association Governance meeting
Michigan Hospital Association Governance meetingMichigan Hospital Association Governance meeting
Michigan Hospital Association Governance meeting
 
Leonard Kingwara publication
Leonard Kingwara publicationLeonard Kingwara publication
Leonard Kingwara publication
 
Mbareng601week12
Mbareng601week12Mbareng601week12
Mbareng601week12
 
Digital health: Ontario Hospitals
Digital health: Ontario HospitalsDigital health: Ontario Hospitals
Digital health: Ontario Hospitals
 
NHS Atlas of Variation in Healthcare for People with Liver Disease
NHS Atlas of Variation in Healthcare for People with Liver DiseaseNHS Atlas of Variation in Healthcare for People with Liver Disease
NHS Atlas of Variation in Healthcare for People with Liver Disease
 
RGC 2015 Telehealth Report
RGC 2015 Telehealth ReportRGC 2015 Telehealth Report
RGC 2015 Telehealth Report
 
Connected Care: Heightened Imperatives
Connected Care: Heightened ImperativesConnected Care: Heightened Imperatives
Connected Care: Heightened Imperatives
 
HLST4250Paper (1)
HLST4250Paper (1)HLST4250Paper (1)
HLST4250Paper (1)
 
10th Annual Utah's Health Services Research Conference - Clinical and Economi...
10th Annual Utah's Health Services Research Conference - Clinical and Economi...10th Annual Utah's Health Services Research Conference - Clinical and Economi...
10th Annual Utah's Health Services Research Conference - Clinical and Economi...
 
EHR Meaningful Use for Medical Practices
EHR Meaningful Use for Medical PracticesEHR Meaningful Use for Medical Practices
EHR Meaningful Use for Medical Practices
 
Meaningful Use Workgroup Stage 3 Recommendations
Meaningful Use Workgroup Stage 3 Recommendations Meaningful Use Workgroup Stage 3 Recommendations
Meaningful Use Workgroup Stage 3 Recommendations
 
I reland feb 2014
I reland feb  2014 I reland feb  2014
I reland feb 2014
 
Cadth symposium 2015 d3 pro presentation apr 2015 - for deb
Cadth symposium 2015 d3 pro presentation   apr 2015 - for debCadth symposium 2015 d3 pro presentation   apr 2015 - for deb
Cadth symposium 2015 d3 pro presentation apr 2015 - for deb
 
Impact diabetes
Impact diabetesImpact diabetes
Impact diabetes
 
EHR Meaningful Use ONC Policy Commitee June 16 2009
EHR Meaningful Use ONC Policy Commitee June 16 2009EHR Meaningful Use ONC Policy Commitee June 16 2009
EHR Meaningful Use ONC Policy Commitee June 16 2009
 

Similar to Final eHO EMR Benefits Report Jan2013

150408 Posters for Evidence Live
150408 Posters for Evidence Live150408 Posters for Evidence Live
150408 Posters for Evidence LiveAlison Turner
 
Journal review of EHR use and benefits
Journal review of EHR use and benefitsJournal review of EHR use and benefits
Journal review of EHR use and benefitsJ. Don Soriano
 
Small County Strategic Planning Presentation v1.2
Small County Strategic Planning Presentation  v1.2Small County Strategic Planning Presentation  v1.2
Small County Strategic Planning Presentation v1.2Saumitra SenGupta
 
Implementing and Evaluating the Hospital Guide to Reducing Medicaid Readmissions
Implementing and Evaluating the Hospital Guide to Reducing Medicaid ReadmissionsImplementing and Evaluating the Hospital Guide to Reducing Medicaid Readmissions
Implementing and Evaluating the Hospital Guide to Reducing Medicaid ReadmissionsJSI
 
Clinical Assignment Quality Improvement Final Project Goal
Clinical Assignment Quality Improvement Final Project GoalClinical Assignment Quality Improvement Final Project Goal
Clinical Assignment Quality Improvement Final Project GoalWilheminaRossi174
 
Assessing the Utility of Consumer Surveysfor Improving the Q.docx
Assessing the Utility of Consumer Surveysfor Improving the Q.docxAssessing the Utility of Consumer Surveysfor Improving the Q.docx
Assessing the Utility of Consumer Surveysfor Improving the Q.docxfredharris32
 
Alcohol and drugs
Alcohol and drugsAlcohol and drugs
Alcohol and drugsJulie Pal
 
Population Health Management & Meaningful Use
Population Health Management & Meaningful UsePopulation Health Management & Meaningful Use
Population Health Management & Meaningful UsePhytel
 
A Review on Clinical Decision Support System
A Review on Clinical Decision Support SystemA Review on Clinical Decision Support System
A Review on Clinical Decision Support SystemIRJET Journal
 
Running head INITIAL PLAN DEVELOPMENT1INITIAL PLAN DEVELOPM.docx
Running head INITIAL PLAN DEVELOPMENT1INITIAL PLAN DEVELOPM.docxRunning head INITIAL PLAN DEVELOPMENT1INITIAL PLAN DEVELOPM.docx
Running head INITIAL PLAN DEVELOPMENT1INITIAL PLAN DEVELOPM.docxjeanettehully
 
The National Academies Health and Medicine DivisionAbout U.docx
The National Academies Health and Medicine DivisionAbout U.docxThe National Academies Health and Medicine DivisionAbout U.docx
The National Academies Health and Medicine DivisionAbout U.docxdennisa15
 
Myanmar Strategic Purchasing 5: Continuous Learning and Problem Solving
Myanmar Strategic Purchasing 5: Continuous Learning and Problem SolvingMyanmar Strategic Purchasing 5: Continuous Learning and Problem Solving
Myanmar Strategic Purchasing 5: Continuous Learning and Problem SolvingHFG Project
 
Encouraging best practice and improve the rational use of medicines
Encouraging best practice and improve the rational  use of medicinesEncouraging best practice and improve the rational  use of medicines
Encouraging best practice and improve the rational use of medicinesMeTApresents
 
Benefits of implementing_the_primary_care_pcmh
Benefits of implementing_the_primary_care_pcmhBenefits of implementing_the_primary_care_pcmh
Benefits of implementing_the_primary_care_pcmhVicki Harter
 
How to Improve Healthcare Reporting Management System.pptx
How to Improve Healthcare Reporting Management System.pptxHow to Improve Healthcare Reporting Management System.pptx
How to Improve Healthcare Reporting Management System.pptxFlutter Agency
 
Optimising the Model of Care for Patient Management at The Tweed Cancer Care ...
Optimising the Model of Care for Patient Management at The Tweed Cancer Care ...Optimising the Model of Care for Patient Management at The Tweed Cancer Care ...
Optimising the Model of Care for Patient Management at The Tweed Cancer Care ...Cancer Institute NSW
 
TransforMED_Pediatric_Care_Coordination_Case_Study
TransforMED_Pediatric_Care_Coordination_Case_StudyTransforMED_Pediatric_Care_Coordination_Case_Study
TransforMED_Pediatric_Care_Coordination_Case_StudyCecilia Saffold
 

Similar to Final eHO EMR Benefits Report Jan2013 (20)

150408 Posters for Evidence Live
150408 Posters for Evidence Live150408 Posters for Evidence Live
150408 Posters for Evidence Live
 
T0 numtq0odu=
T0 numtq0odu=T0 numtq0odu=
T0 numtq0odu=
 
Journal review of EHR use and benefits
Journal review of EHR use and benefitsJournal review of EHR use and benefits
Journal review of EHR use and benefits
 
Small County Strategic Planning Presentation v1.2
Small County Strategic Planning Presentation  v1.2Small County Strategic Planning Presentation  v1.2
Small County Strategic Planning Presentation v1.2
 
Implementing and Evaluating the Hospital Guide to Reducing Medicaid Readmissions
Implementing and Evaluating the Hospital Guide to Reducing Medicaid ReadmissionsImplementing and Evaluating the Hospital Guide to Reducing Medicaid Readmissions
Implementing and Evaluating the Hospital Guide to Reducing Medicaid Readmissions
 
Clinical Assignment Quality Improvement Final Project Goal
Clinical Assignment Quality Improvement Final Project GoalClinical Assignment Quality Improvement Final Project Goal
Clinical Assignment Quality Improvement Final Project Goal
 
Assessing the Utility of Consumer Surveysfor Improving the Q.docx
Assessing the Utility of Consumer Surveysfor Improving the Q.docxAssessing the Utility of Consumer Surveysfor Improving the Q.docx
Assessing the Utility of Consumer Surveysfor Improving the Q.docx
 
Alcohol and drugs
Alcohol and drugsAlcohol and drugs
Alcohol and drugs
 
Population Health Management & Meaningful Use
Population Health Management & Meaningful UsePopulation Health Management & Meaningful Use
Population Health Management & Meaningful Use
 
Blank clinical audit report template
Blank clinical audit report templateBlank clinical audit report template
Blank clinical audit report template
 
A Review on Clinical Decision Support System
A Review on Clinical Decision Support SystemA Review on Clinical Decision Support System
A Review on Clinical Decision Support System
 
Running head INITIAL PLAN DEVELOPMENT1INITIAL PLAN DEVELOPM.docx
Running head INITIAL PLAN DEVELOPMENT1INITIAL PLAN DEVELOPM.docxRunning head INITIAL PLAN DEVELOPMENT1INITIAL PLAN DEVELOPM.docx
Running head INITIAL PLAN DEVELOPMENT1INITIAL PLAN DEVELOPM.docx
 
The National Academies Health and Medicine DivisionAbout U.docx
The National Academies Health and Medicine DivisionAbout U.docxThe National Academies Health and Medicine DivisionAbout U.docx
The National Academies Health and Medicine DivisionAbout U.docx
 
Myanmar Strategic Purchasing 5: Continuous Learning and Problem Solving
Myanmar Strategic Purchasing 5: Continuous Learning and Problem SolvingMyanmar Strategic Purchasing 5: Continuous Learning and Problem Solving
Myanmar Strategic Purchasing 5: Continuous Learning and Problem Solving
 
Encouraging best practice and improve the rational use of medicines
Encouraging best practice and improve the rational  use of medicinesEncouraging best practice and improve the rational  use of medicines
Encouraging best practice and improve the rational use of medicines
 
Meaningful Use When 5 19 10
Meaningful Use When 5 19 10Meaningful Use When 5 19 10
Meaningful Use When 5 19 10
 
Benefits of implementing_the_primary_care_pcmh
Benefits of implementing_the_primary_care_pcmhBenefits of implementing_the_primary_care_pcmh
Benefits of implementing_the_primary_care_pcmh
 
How to Improve Healthcare Reporting Management System.pptx
How to Improve Healthcare Reporting Management System.pptxHow to Improve Healthcare Reporting Management System.pptx
How to Improve Healthcare Reporting Management System.pptx
 
Optimising the Model of Care for Patient Management at The Tweed Cancer Care ...
Optimising the Model of Care for Patient Management at The Tweed Cancer Care ...Optimising the Model of Care for Patient Management at The Tweed Cancer Care ...
Optimising the Model of Care for Patient Management at The Tweed Cancer Care ...
 
TransforMED_Pediatric_Care_Coordination_Case_Study
TransforMED_Pediatric_Care_Coordination_Case_StudyTransforMED_Pediatric_Care_Coordination_Case_Study
TransforMED_Pediatric_Care_Coordination_Case_Study
 

Final eHO EMR Benefits Report Jan2013

  • 1. Benefits Realization Study Medical Records in Ontario Final eHealth Ontario January 2013 Benefits Realization Study for Electronic Medical Records in Ontario Final Report to eHealth Ontario Benefits Realization for Electronic Medical Records in
  • 2. Benefits Realization Study for EMRs in Ontario PwC Acknowledgements This study would not have been possible without the support of Steering Committee members and several clinical leaders who participated in this study and facilitated access to their clinics and colleagues. We thank them and their teams for their commitment of time, knowledge and expertise. Their insights and first-hand experience in pioneering and advancing the use of electronic medical records in primary care settings in Ontario provided a rich study environment to better understand the current and potential benefits that can be realized through the use of electronic medical records in Ontario. Clinical Leaders Steering Committee Members Dr. David Barber Queen’s Family Health Team Emmanuel Casalino Senior Director, Physician eHealth Program eHealth Ontario Dr. Sonny Cejic Commissioners West Family Health Organization Dr. Anne Duvall Peer Leader OntarioMD Barrie & Community Family Health Team Dr. Anne Duvall Barrie & Community Family Health Team Dennis Ferenc Director, Funding, Reporting and Change Management OntarioMD Dr. Sanjeev Goel Wise Elephant Family Health Team Simon Hagens Director, Benefits Realization & Quality Improvement Canada Health Infoway Dr. David Kaplan North York Family Health Team Dr. David Kaplan Primary Care Physician Lead Central LHIN North York Family Health Team Dr. Stephen McLaren Markham Family Health Team Dr. Wei Qiu Director, EMR Adoption and Benefit Realization eHealth Ontario Christine Sham Manager, eHealth Liaison Branch Ministry of Health and Long-Term Care Patricia Sullivan-Taylor Manager, Primary Health Care Information Canadian Institute for Health Information
  • 3. Benefits Realization Study for EMRs in Ontario PwC Table of Contents Acknowledgements 2 1. Executive Summary 1 2. Introduction 10 3. Methodology 12 3.1 Phase 1: Validation and Evolution of EMR Benefits Realization Framework 12 3.2 Phase 2: Analysis of Benefits Realization 14 3.3 Study Limitations 19 4. Case Study Results 21 4.1 Provider Survey Results 21 4.2 Indicator and Interview Results 24 5. Modeling and Forecasting 33 5.1 Diabetes Management 33 5.2 Increased Influenza Immunization Rates 35 5.3 Increased Colon Cancer Screening Rates 37 5.4 Staff Time Reduction Spent on Administrative Tasks 39 5.4 Summary Findings 41 6. Discussion 43 6.1 Laboratory Management 43 6.2 Communication and Coordination of Care 44 6.3 Chronic Disease Management 45 6.4 Health Promotion, Screening and Prevention 47 6.5 Efficiency 48 6.6 Medication Management 49 7. Recommendations 51 8. Concluding Remarks 54 9. References 55
  • 4. Benefits Realization Study for EMRs in Ontario PwC Appendix A: Interviewee List Appendix B: Consultation Guide Appendix C: Revised EMR BR Framework Appendix D: EMR BR Framework- Case Study Indicator Subset Appendix E: EMR Maturity Model Criteria Description Appendix F: Organizational Survey Appendix G: Provider Survey Appendix H: Site Visit Interview Guide
  • 5. Benefits Realization Study for EMRs in Ontario PwC 1 1. Executive Summary Electronic medical records (EMRs) are a key enabler of health system transformation, with the potential to achieve widespread benefits, including: delivering improvements in patient care processes; enabling positive health outcomes at individual and population levels; creating efficiencies; and reducing costs. Ontario’s Action Plan for Health Care (Ministry of Health and Long-Term Care, 2012) defines “faster access to stronger family health care” as a key imperative, and also focuses on improving the overall health of the population while ensuring that the right care is delivered at the right time, in the right place. In addition, the government’s transformative Excellent Care for All Act (S.O. 2010, c14) will ensure that Ontarians receive health care of the highest possible quality and value, and puts the needs of patients first, placing greater accountability on providers to ensure that the best available evidence is used to make decisions about patient care. Most recently in December 2012, the government announced two very impactful changes to the health system, including the signing of the Physician Services Agreement with the Ontario Medical Association the creation of Health Links, placing primary care providers at the centre of the health system and ensuring that patients receive faster care, spend less time waiting for services and are supported by a team of health care providers at all levels of the health care system (Ministry of Health and Long Term Care, 2012). With these and other initiatives underway, it is clear that Ontario’s health system will be undergoing significant transformation, and changes in the delivery of primary care figure prominently in the government’s strategic initiatives. With 80% of health care encounters occurring in primary care settings (Canadian Medical Association, 2011), the vast majority of patient data is collected and managed at the primary care level, and the transformative changes to be undertaken will be reliant on information management programs and tools. The broad and mature adoption of electronic medical records (EMRs) by primary care providers and their staff will support the required transformation and the realization of benefits such as improvements in patient care, positive health outcomes at individual and population levels, efficiencies for providers and system-wide cost reductions. The strategic and ongoing focus on advancing the availability and use of electronic medical records (EMRs) across the province is essential. A key lever in this strategic transformation is the Physician eHealth Program (PeHP), which is now in its fourth year investing in community-based physician offices to support the adoption and use of EMRs. The PeHP has achieved several outcomes in the past four years, including the enrolment of over 9,000 physicians in the EMR Adoption Program. Having achieved a critical mass of EMR adoption across the province, the PeHP has further developed its focus on benefits realization in 2012 and beyond, opting to undertake a study to identify the nature of the benefits realized through program investments to date, and those that can potentially be realized with continued investment and broader adoption of EMRs. To further understand and assess the potential benefits to be realized, PeHP engaged PwC to conduct a benefits realization study with a focus on:  The benefits that a select number of advanced users of EMRs in primary care practice settings sites have realized, directly and indirectly through EMR adoption and usage; and  The potential benefits that may be realized through widespread, mature adoption of EMR use across Ontario in the future. Methodology The methodology for the study was designed by PwC in consultation with the PeHP team to articulate current and potential benefits from the use of EMRs and recommendations for widespread EMR benefits realization across Ontario. Two phases of activities from August to December 2012 were undertaken to complete this study:
  • 6. Benefits Realization Study for EMRs in Ontario PwC 2  Phase 1 entailed the validation and evolution of a comprehensive framework for assessing the benefits realized in Ontario as a result of EMR use in primary care settings; and  Phase 2 focused on the analysis of current and potential benefits realized across Ontario through increased and sustained use of EMRs. The work completed during Phase 1 set the stage for successful evaluation while simultaneously building the necessary support among stakeholders through validation of the existing benefits realization (BR) framework. The approach emphasized the purpose of the framework and the effective positioning of potential benefits that can be realized by patients, providers and the health system. Of the original indicators, 21 were identified as priority indicators, laying the foundation for a longer term BR Framework and measurements that may be used for future analysis. The priority indicators were identified, specifically for assessment in Phase 2 of the Benefits Realization study based on known available EMR functionality, maturity of use and feasibility of measurement at case study sites. Based on the current availability of information and the maturity of EMR users, a set of indicators was selected to assess benefits through a case study approach at six primary care practices across Ontario. Following the case studies, the set of indicators was further distilled to reflect data availability and relevance, and a final selection of 11 indicators in five categories was used to evaluate benefits:  Laboratory Management,  Communication and Coordination of Care,  Chronic Disease Management,  Health Promotion, Screening and Prevention, and  Efficiency. Phase 2 focused on assessing both current and potential benefits realized through EMR use. Case studies were designed to understand the benefits realized through the use of EMRs in the six selected clinics across Ontario that were identified as high performing users of EMRs. While the emphasis of the study was to understand both quantitative and qualitative processes and outcomes as per the indicators defined in the BR framework, due consideration was given to identifying relevant insights, experiences and lessons learned that supported benefits realization and demonstrate the role of EMRs (both direct and indirect) in realizing these benefits. Emerging from the cases studies was a sixth category of study (Medication Management) and for which qualitative benefits have been discussed. Evidence for the case studies was collected from surveys, interviews, direct observation and data extraction from EMRs. Following the collection of data from the Case Study sites, the PwC team undertook a modeling exercise to forecast the potential benefits of EMR use in Ontario in five to ten years. Benefits were extrapolated from selected indicator values collected during the site visits and further validated and substantiated with peer reviewed literature. The modeling involved three steps: 1. A comparison of current values of indicators for case study sites considered high performing, or advanced users of EMRs and those for other practices in Ontario. 2. An assessment of the relative benefit realized through advanced EMR use by the case study sites compared to other practices in Ontario. 3. Extrapolation of relative potential benefits to the entire province, providing benefit estimates (such as potential avoided costs or quality of care outcomes), if all providers in Ontario adopted an EMR and achieved similar results to the high performing users of EMRs examined in the case studies practices.
  • 7. Benefits Realization Study for EMRs in Ontario PwC 3 Case Study Results The surveys of providers at case study sites indicated very high support for benefit statements attributed to EMRs. Key results of the provider surveys include the following:  Quality of Care: 93.0% of survey respondents strongly or moderately agreed with eight statements presented for benefits of EMR use related to quality of care;  Communication and Coordination of Care: 98.0% of survey respondents strongly or moderately agreed with three statements presented for benefits of EMR use related to communication and coordination of care;  Efficiency of Practice: 81.5% of survey respondents strongly or moderately agreed with nine statements presented for benefits of EMR use related to efficiency of practice; and  Patient Experience: 94.4% of survey respondents strongly or moderately agreed with two statements presented for benefits of EMR use related to overall patient experience. Indicator and Interview Results As described above, the priority indicators that were identified in Phase 1 were assessed for each of the six case study sites. Findings from indicators and interviews were reviewed together with a sixth category, Medication Management, which emerged in discussion with several providers during the interviews. Key results that were identified through the case studies are as follows: Turnaround time for lab result availability to receipt in EMRs has declined from as much as 5 days to virtually instaneously. For most study sites, a minimum of 50% decrease in turn around time was reported with results returned into their EMR. This timely access to test results affords the opportunity to expedite referrals and improve access to care, and to make timely decisions related to treatments. In addition, physicians perceived the EMR to improve ordering efficiencies. The time to receive discharge summaries after patient discharge has declined from as much as 14 days to virtually immediately where EMRs are integrated with tools such as the Hospital Report Manager or through direct connectivity with hospitals. These reductions in time spent waiting for discharge summaries and referrals expedite and facilitate the coordination of care such that patient needs can be addressed in a timely fashion. The time from referral decision to when the referral is sent to specialist has declined from as much as 7 days to less than 1 day. Some physicians were able to complete the referral and send it to the specialist with the patient in the room. Overall, it was observed that the EMR improves physicians’ ability to make timely referrals, expediting the care process and facilitating inter-office communications. 100% of the care team members have remote and local access to EMRs. This access to information at any time and in any place was noted by all as a tremendous asset to providers and their patients, improving the ability to communicate within clinics and often with providers outside of clinics, with overall results including improvements in the efficiency of patient care. Up to 70% of diabetic patients, 18 years and over, have an HbA1c level of 7% or less. With the ability to identify a target diabetic population, physicians and care teams are better able to develop care plans tailored to populations. Overall, provider survey respondents agreed that EMRs improve the management of chronic diseases. 92.5% of survey respondents also reported that the EMR system supports patient education. For example, the EMR provides access to handouts, references or tools to trend patters for BMI, blood sugar, labs. Collectively, the educational materials and tools help patients better understand and manage their chronic conditions.
  • 8. Benefits Realization Study for EMRs in Ontario PwC 4 Up to 80% of practice populations received an influenza immunization, up to 56% of practice populations had a screening test ordered for colon cancer, and up to 82% of practice populations have had their blood pressure measured by their primary care provider within the last 15 months. Provider survey results emphasize the role of the EMR system in promotion, prevention and screening activities. Provider survey results report that 92.6% of survey respondents agree that EMRs improve patient safety and the proactive monitoring of overdue tests/ exams. It should be noted that the provincial target for colorectal cancer screening is 40% (Health Quality Ontario, 2012) and that the values reported by all the case study sites exceeded the provincial average. Up to 50% less time was required by high performing practices to complete clinical/administrative documentation by using pre-populated templates, forms and stamps. Overall, 77.7% of survey respondents agreed with the statement that EMRs “improve the efficiency (reduction in effort) of my practice” and of those 44.4% strongly agreed with the statement while 33.3% moderately agreed. These survey responses mirror the comments collected during the interviews on overall practice efficiencies. Up to 89% of survey respondents agree that EMRs support patient safety through the ability to identify patients on prescriptions that have drug recalls. Physicians and care team members reported an important patient safety benefit of the EMR is the ability to quickly extract patient lists for specific medications that have been recalled. Additional patient safety benefits were identified in relation to improved legibility of prescriptions. Modeling and Forecasting A model was developed to forecast potential benefits and “the art of the possible” in Ontario if all providers were adopting and using EMRs at the same level as the advanced EMR users in the case study sites. Benefits were extrapolated from indicators collected through the case studies, and supported with peer reviewed literature. Four indicators were selected from the BR framework for forecasting based on the availability of data for each indicator at the various sites as well as the availability of supporting evidence to extrapolate these indicators to benefits for Ontario. These include:  Chronic Disease Management - Improved diabetes management  Health Promotion, Screening and Prevention – Increased influenza immunization rates  Health Promotion, Screening and Prevention – Increased colon cancer screening rates  Efficiency - Staff time reduction spent on administrative tasks Each indicator along with its associated benefit and benefit estimate is provided in the table that follows. Although the selected benefits forecast only a portion of benefits expected from EMRs and that a variety of contributing factors influence the values of these indicators (e.g. financial incentives to increase screening rates), they indicate substantial potential benefits to be realized if all providers in the province were to become high performing users of EMRs in 5 – 10 years.
  • 9. Benefits Realization Study for EMRs in Ontario PwC 5 Summary of Forecasted Benefits Indicator Qualitative Annual Benefits Potential Annual Financial Impact by 2017 Percentage of patients, 18 years and over, with diabetes mellitus in whom the last HbA1c was 7.0% or less (or equivalent test/reference range depending on local laboratory) in the last 15 months Patient and health system benefits from management of complications and co-morbid conditions arising from diabetes, including: - 566 fewer foot amputations - 341 fewer cases of ESRD - 17,400 fewer MI cases - 3,100 fewer stroke cases $125 million ($17 M from reduced foot amputations, $26 M from reduced kidney disease, $44M from fewer MI, $38M from fewer strokes) Percentage of patient population, age 65 and older, who received an influenza immunization Patient and health system benefits related to illness prevention, avoidance of clinic and/or hospital visits $40.6 M in reduced health care system costs Percentage of practice population, age 50 to 74, who had a screening test ordered for colon cancer Identification and diagnosis of cancers and malignancies at early stages for improved prognosis and quality of life for patients - 220 fewer Ontarians diagnosed with stage 4 cancer; - 217 fewer Ontarians diagnosed with stage 3 cancer; - 267 fewer Ontarians diagnosed with stage 2 cancer; - 703 additional Ontarians with no cancer or stage 0/stage 1 cancers $38.0 M in reduced colon cancer treatment costs Percentage reduction in time spent on administrative tasks Increased staff capacity through a reduction of 1.4 million hours that can be allocated to other tasks, including the provision of patient care $40.0 M in increased staff capacity Discussion Through an approach that has focused on case studies of advanced EMR users in Ontario and forecasting of further potential benefits, this study has demonstrated that Ontario’s investments in EMR to date have yielded noteworthy, tangible benefits. These benefits are diverse and have accrued to patients, providers and the broader health system. Case study participants provided a unique vantage point to convey valuable insights into benefits from their first- hand experience in pioneering and optimizing the use of a variety of EMR tools and functionalities across their patient care teams. While the earliest benefits have been shown to be realized by providers and their patients, modeling and forecasting has demonstrated that continued investment in EMRs across the province can heighten the impact of EMR use on the broader health system, with the potential for EMRs to have very significant direct and indirect impacts.
  • 10. Benefits Realization Study for EMRs in Ontario PwC 6 While it is acknowledged that the potential benefits of EMRs are broad, the current study focused on six categories with key benefits discussed below. 1. Laboratory Management EMRs provide clinicians with more timely access to laboratory information, aiding care decisions and enhancing the patient experience. The ability to receive lab results through direct transmission to EMRs has reduced the time to receive those results by 50% on average (compared to a paper-based environment). Consequently, clinicians are now well enabled to respond to results quickly and effectively. This timely access to test results, while either in their clinics or from remote locations, affords the opportunity to make timely decisions, provide prompt and appropriate care, expedite referrals, and improve patients’ access to care. The value of having comprehensive lab information in a timely manner through all potential sources of lab information provides a comprehensive profile for patients to form a basis for clinical decision-making. In addition, the increasing prevalence of patient portals linked to EMRs, patients and/or proxies can have online access to results, allowing them to review lab results when and where it is convenient for them. These initiatives can improve patients’ experiences with their care, and provide them with some comfort, understanding and ownership in care processes. 2. Communication and Coordination of Care EMRs facilitate improved scheduling and coordination of patient visits, improving access to care. EMRs provide physicians and care team members with the improved ability to schedule patient visits, improving patients’ access to care and efficiency of the care team. Physicians and team members reported that improvements in scheduling and organization facilitate their ability to hold same day appointments open, improving access to care. Ontario’s Health Action Plan (Ministry of Health and Long-Term Care, 2012) identifies improved availability of same-day appointments as a key focus in improving access to primary care. EMRs improve the availability and sharing of information among interdisciplinary team members and enhances quality and efficiency of care. The ability to access patient information at any time and in any place was noted by all as a tremendous asset to providers and their patients. EMRs also improve the ability to communicate within clinics and often with providers outside of clinics, with overall results improving the efficiency of patient care. Quality of care, and the patient experience overall, is improved for patients by ensuring all providers have access to the same patient information. EMRs facilitate the sharing of information with specialists, thereby improving the continuity and efficiency of care. EMRs support the ability to make much more informed and efficient referrals to specialists. With the ease and improved efficiency of making referrals (an approximate reduction in time of 85% to make a referral), the EMR allows primary care clinicians to quickly provide the specialist with key pieces of information needed to understand the patient’s condition. Expedited delivery of hospital reports to EMRs facilitates timely and appropriate care. Case study sites reported a reduction of 85% in time spent waiting for discharge summaries as a result of the transfer of this information to EMRs. The timely access to this information can further reduce patients’ wait times for required post-discharge care, and allow providers to put appropriate follow-up care into place in the out-patient setting. 3. Chronic Disease Management EMRs are a necessary and effective tool to manage the health of defined patient populations. EMRs are being used increasingly by clinicians to manage the health of patient populations, such as patients with diabetes, chronic obstructive pulmonary disease (COPD), and others. Physicians and their care teams are increasingly relying on their EMRs to effectively manage the care of patients with chronic conditions that are costly
  • 11. Benefits Realization Study for EMRs in Ontario PwC 7 to the system overall and have many related co-morbidities constraining health system capacity. It was evident that case study sites were using their EMRs for these purposes as well; for example, clinicians reported the ability to identify patients with specific conditions and plan and monitor their care, and identified this as a very significant benefit of EMRs. Clinicians reported a high degree of willingness and enthusiasm to better manage the health of defined patient populations in their practice through the support of their EMRs, and acknowledged that in the absence of EMRs this undertaking would be quite costly and time consuming. EMRs provide valuable tools to help both care providers and patients with care management and education. Engaging patients through education and care planning activities ensures that they are active participants in their health. Patients that are better informed on what their target values for key indicators (e.g., HbA1c) should be are better able to keep their conditions under control. As data capture and reporting capabilities improve, EMRs will further allow practices to identify a baseline and trend information to ensure their diabetic cohort of patients are monitored for those clinical needs which require careful management and are prone to downstream co-morbid complications. The broad and mature use of EMRs can reduce the costs and burden of illness associated with caring for Ontario’s growing diabetic population. Through the modelling and forecasting exercise, potential savings and a reduction in complications associated with diabetes were identified. These complications and illnesses include foot amputations, diabetic kidney disease, stroke and myocardial infarction, all of which typically require acute care hospitalization. While yet to be realized, EMRs are a contributing factor to potential savings related to diabetic-related illnesses that are in the range of $125 million annually by 2017. Diabetes must be actively managed in the community and primary care settings and with active patient participation, in order to complications such as those forecasted above. Without EMRs and related enablers, it would be very challenging to do so, given the support that EMRs provide to actively identify diabetic patients, keep their conditions under control, and communicate on an ongoing basis. 4. Health Promotion, Screening and Prevention EMRs allow clinicians to survey patients and to proactively arrange screening and prevention activities, while concurrently improving the efficiency of preventative care. While several reporting capabilities related to screening and prevention are in their early stages of development, advanced users of EMRs indicated that they were able to generate information to identify and communicate with patients for preventative purposes. Without the EMR, this type of prevention activity is much more complicated, requiring manual and time-consuming chart reviews. Now, patients are contacted easily and in a timely fashion for preventative care. The potential benefits of increased screening and prevention (including vaccination) activities over time can have a tremendous impact on the health of the population as a whole, and on the sustainability of the health system. The widespread use of EMRs can increase the rates of influenza vaccination and yield potential related health system savings. The forecasted reductions in illness and costs are highly dependent on the mature, proactive use of EMRs on a province-wide basis to identify those at risk, to facilitate communication to encourage those patients to receive the flu shot, and to track compliance. The management of influenza must be actively managed in the community and primary care settings and with active patient participation and willingness to receive vaccination. EMRs enable this challenge, with many interviewees discussing their ability to vaccinate a broader group of patients by leveraging information available in their EMRs. Use of EMRs can support the prevention of colon and other cancers through improved screening rates and other preventative care. Case studies revealed that EMRs can greatly facilitate the ease with which clinicians manage and deliver preventative care for their patients. Although preventive care is a relatively new (and in some cases, advanced) area of EMR use for many clinicians, the ability to identify patients requiring screening
  • 12. Benefits Realization Study for EMRs in Ontario PwC 8 and in turn receive and act upon results is possible with EMR use today, and many provincial organizations (e.g., Cancer Care Ontario) recognize the power and potential of EMRs to support preventive care. Modeling and forecasting activities suggested that the potential costs that can be realized through the prevention of four stages of colon cancer through advanced EMR use is in the range of $38 million annually by 2017. By ensuring that screening protocols and alerts are incorporated in all EMR specifications, there will be an increased ability to identify those patients requiring screening on a widespread basis. 5. Efficiency EMRs facilitate clinical transformation in the primary care setting, improving the effectiveness and efficiency of clinical and administrative activities. The introduction of EMRs into the primary care setting has the potential to be transformative in nature. EMRs enable changes in workflow and clinical decision-making, and can greatly improve the way that clinicians interact with their patients on a regular basis. Benefits associated with the effectiveness and efficiency of patient encounters and the general flow of patient activity allow greater throughput and access to care, including same day visits. These benefits have tremendous impact on patients, providers and the system as a whole, by reducing wait times for primary care, allowing providers to see more patients daily, and reducing the number of unnecessary visits to hospitals. EMRs improve the productivity of administrative staff. It is evident that there is some productivity lost in the early stages of adoption of EMRs with many clinicians and administrators, with many reporting that there is a period of approximately one year while all grow accustomed to working with electronic records. However, interviewees and the forecasting model both suggest that there are significant opportunities to improve the overall capacity among clinic staff, particularly among administrative staff and/or nurses who were previously spending time doing administrative tasks. The greatest value in this increased capacity lies in the opportunities and potential benefits associated with redirecting time from non-value added tasks, to those that improve efficiency, allow for direct patient interaction, and improve the overall patient experience. The reductions in the time required for administrative tasks through the use of EMRs, estimated at 50%, was used to model the potential benefits if EMRs were broadly and maturely used across Ontario. The exercise estimated that approximately 1.8 million hours or a possible $40 million could be saved annually by 2017, providing increased capacity for clinic staff. 6. Medication Management EMRs support the ability to rapidly identify impacted patients when drug warnings are issued, improving patient safety. A tremendous benefit of EMR use identified by many providers through the course of the study was the ability to identify large numbers of patients to whom certain drugs have been prescribed. These patients can be very quickly identified, and alternate means of treatment can quickly be administered, preventing any downstream implications. As such, EMRs can have a tremendous impact on quality of care, and most importantly, patient safety. The ability to quickly target these patients and act is almost impossible in a paper- based clinic environment. Access to complete medication profiles in the EMR increases efficiency and improves the accuracy of medication management. With accurate and complete medication profiles in EMRs, clinicians are able to quickly and accurately manage care. Prescriptions are easily monitored, and patient concerns can be addressed. There are also potential costs avoided by the health system due to unnecessary hospitalization from adverse medication effects. Electronic prescribing and renewals via EMRs have improved medication management efficiencies and patient safety. The sharing of medication information across the care team has resulted in greater efficiencies in the patient care process and improved patient safety. Perhaps the most significant change noted by physicians has been the generation of a printed prescription, eliminating error-prone handwritten prescriptions
  • 13. Benefits Realization Study for EMRs in Ontario PwC 9 and reducing the need for delays arising from “call backs” from pharmacists to physicians seeking clarification on prescriptions. With EMRs, this risk to patient safety and disruption in workflow is avoided. Recommendations Findings from this study provide compelling evidence to continue to advance EMR adoption and maturity across Ontario. The benefits that have been demonstrated by EMR use in the selected case study settings and the accompanying forecasting of province-wide benefits demonstrate the “art of the possible” for Ontario. With the implementation of a number of focused recommendations, the potential for wide ranging and transformative benefits of EMR use can be further realized by providers, patients and the health care system as a whole. The following five recommendations are presented to policy-makers, funders, implementers and adopters of EMRs in Ontario in support of continued benefits realization. 1. As an essential enabler and one of many important health information technology tools to improve care delivery and related patient outcomes in primary care settings and beyond, continued investments in EMRs should be made to ensure broad adoption and realization of benefits across Ontario. Through the course of the study it was widely acknowledged that without the use of EMRs, the ability to realize the identified benefits is compromised. Indirectly and directly, EMRs are critical enablers of enhanced patient care. Continued investment in EMRs and increased physician participation in the EMR Adoption Program are essential. 2. Continue to support increased maturity of use among current and future adopters of EMRs. The effective realization of benefits is highly supported by EMR maturity (defined as the level of adoption and use of the EMR in the practice setting). A continued focus in advancing EMR maturity among users will contribute to a greater diversity of benefits with system-wide impact. 3. Continue to invest in effective change management strategies and user support that extends beyond the initial period of EMR implementation. Mature use of the EMR requires access to training when and where it is needed by all types of users, and should be available well beyond the initial implementation phase as users transition through the “adoption curve”. For example, following an initial period of use that allows clinicians to master essential EMR functions, training could be further made available to address and support more sophisticated needs associated with reporting and analytics for population-based planning and care. 4. Improve the management of information within and across patient care settings through focused efforts related to interoperability of systems, improved quality of data, and the flow of data across care settings. The ability to achieve advanced use and benefits of EMRs will be supported by increased systems integration, improved quality of data, and improved sharing of data across care providers. Improvements in information management through initiatives such as OntarioMD’s Hospital Report Manager and CIHI’s Voluntary Reporting System are showing promising benefits, and similar initiatives should be encouraged. 5. Continue to invest in focused benefits realization studies. Focused studies will afford the opportunity to measure more of the indicators that were defined in this study as part of the Benefits Realization Framework and have a greater understanding of the full scope of current and potential benefits realized by EMRs in Ontario.
  • 14. Benefits Realization Study for EMRs in Ontario PwC 10 2. Introduction Health care systems across Canada and around the world are undergoing transformations to improve quality of care, access to care, value for money, and the patient experience (Commonwealth Fund, 2011). In Ontario, there is an increasing emphasis on ensuring that these attributes are all central to the delivery of primary care, providing the foundation of a strong health care system. Ontario’s Action Plan for Health Care (Ministry of Health and Long-Term Care, 2012) defines “faster access to stronger family health care” as a key imperative, and also focuses on improving the overall health of the population while ensuring that the right care is delivered at the right time, in the right place. In addition, the government’s transformative Excellent Care for All Act (S.O. 2010, c14) will ensure that Ontarians receive health care of the highest possible quality and value, and puts the needs of patients first, placing greater accountability on providers to ensure that the best available evidence is used to make decisions about patient care. Most recently in December 2012, the government announced two very impactful changes to the health system, including:  the Physician Services Agreement reached between the Ontario Medical Association and the government in December 2012 (Ministry of Health and Long-Term Care, 2012), supporting the realization of significant cost savings and efficiencies across the health system and promoting the use of electronic communications and consultations to increase access to care; and  the creation of Health Links (Ministry of Health and Long-Term Care, 2012), placing primary care providers at the centre of the health system and ensuring that patients receive faster care, spend less time waiting for services and are supported by a team of health care providers at all levels of the health care system. With these and other initiatives underway, it is clear that Ontario’s health system will be undergoing significant change, and changes in the delivery of primary care figure prominently in the government’s strategic initiatives. With 80% of health care encounters occurring in primary care settings (Canadian Medical Association, 2011), the vast majority of patient data is collected and managed at the primary care level, and the transformative changes to be undertaken will be reliant on information management programs and tools. The broad and mature adoption of electronic medical records (EMRs) by primary care providers and their staff will support the required transformation and the realization of benefits such as improvements in patient care, positive health outcomes at individual and population levels, efficiencies for providers and system-wide cost reductions. The strategic and ongoing focus on advancing the availability and use of electronic medical records (EMRs) across the province is essential. In supporting, promoting and accelerating the adoption of EMRs across the province, eHealth Ontario’s Physician eHealth Program (PeHP) is a key initiative in advancing the province’s transformative agenda and Ontario’s eHealth strategy and primary goal of establishing and maintaining electronic health records (EHRs) for all of Ontario’s 13 million residents (eHealth Ontario, 2013). EMRs are a partial and necessary component of EHRs, which hold all relevant health information about a person over his/her lifetime (Hodge, 2011). Since 2009, the PeHP has invested in community-based physician offices to support the adoption and use of EMRs. Together with its delivery agent OntarioMD, the PeHP has achieved several outcomes, including the enrolment of over 9,000 physicians in the EMR Adoption Program. The program is currently focused on:  Further equipping and enabling community-based physicians with tools to enhance the use of EMRs;  Maximizing clinical and business value of EMRs;  Providing change management support and promoting the adoption of best practices; and  Fostering the evolution and sustainment of EMRs and related benefits.
  • 15. Benefits Realization Study for EMRs in Ontario PwC 11 Having achieved a critical mass of EMR adoption across the province, the PeHP has further developed its focus on benefits realization in 2012, opting to undertake a study to identify the nature of the benefits realized through program investments to date, and those that can potentially be realized with continued investment and broader adoption of EMRs. Specifically, the current and potential impacts of the use of EMRs on patients, providers and the health system are of interest. To further understand and assess the current and potential benefits to be realized, PeHP engaged PwC to conduct a benefits realization study with a focus on:  The benefits that a select number of advanced users of EMRs in primary care practice settings sites have realized, directly and indirectly through EMR adoption; and  The potential benefits that may be realized through widespread, mature adoption of EMR use across Ontario in the future. In collaboration with the PeHP team, PwC conducted this study through a two-phased approach from August to December 2012. This report presents the study in its entirety as follows:  Section 3: Methodology, outlining the methodology applied to Phases 1 and 2  Section 4: Results, presenting the findings arising from the studies of six primary care clinics considered high performing adopters of EMRs  Section 5: Modeling of Benefits, identifying potential system-wide benefits from EMR use  Section 6: Discussion, reporting on the themes and insights emerging from the results  Section 7: Recommendations, outlining key actions to be taken in order to realize potential benefits across Ontario  Section 8: Concluding Remarks
  • 16. Benefits Realization Study for EMRs in Ontario PwC 12 3. Methodology The methodology for the study was designed by PwC in consultation with the PeHP team to articulate current and potential benefits, and to develop recommendations for widespread EMR benefits realization across Ontario. Two phases of activities from August to December 2012 were undertaken to complete this study:  Phase 1 entailed the validation and evolution of a comprehensive framework for assessing the benefits realized in Ontario as a result of EMR use in primary care settings; and  Phase 2 focused on the analysis of current and potential benefits realized across Ontario through increased and sustained use of EMRs. 3.1 Phase 1: Validation and Evolution of EMR Benefits Realization Framework The work completed during Phase 1 set the stage for successful evaluation while simultaneously building the necessary support among stakeholders through validation of the existing benefits realization (BR) framework. The approach emphasized the purpose of the framework and the effective positioning of potential benefits that can be realized by patients, providers and the health system. 3.1.1 External Validation of the Framework The PeHP program had developed an initial draft of a BR framework, inclusive of several categories of measurement, hypotheses and sixty indicators. This framework was used as the basis for consultations to obtain feedback and was the subject of further refinement and validation. A total of 21 consultations were conducted with a variety of representatives including physicians, family health teams, the Ministry of Health and Long Term Care, government agencies, Local Health Integration Networks (LHINs), and other jurisdictional EMR programs (Alberta and British Columbia), (see Appendix A for a complete list of interviewees). Interviewees were provided with a copy of the interview guide and the draft EMR BR Framework for review prior to the interview (see Appendix B, Consultation Guide). Stakeholders were asked to select indicators that could describe whether or not EMRs have had an overall impact on patients, providers and the health system. A formal system for scoring indicators was not used, however stakeholders reviewed the indicator list and based on their experience and perspective, were able to identify those indicators they felt would be meaningful and useful to measure. 3.1.2 Development of BR Framework and Indicators A workshop was held in September 2012 to review all feedback and define the preferred BR framework and indicators for the case studies and any future benefits realization studies. Workshop discussions were informed by insights from consultations, a preliminary BR framework assessment, benefits literature and PwC’s experience with related benefits realization studies. At the workshop, project team members from the PeHP, Ontario MD and PwC assessed potential indicators based on the following criteria:
  • 17. Benefits Realization Study for EMRs in Ontario PwC 13  Relevance: Assessment of the importance of the indicator for the objective/hypothesis question e.g. Does the indicator support an understanding of the impact of the EMR on quality of care from the patient’s perspective?  Feasibility: Assessment of the ease and cost of measurement e.g. is the data currently available?  Ease of interpretation: Assessment of the ease of interpreting the indicator, e.g. Is the indicator well understood by multiple stakeholders, without requiring extensive explanation?  Traceability to the EMR: Assessment of whether the indicator can be directly attributed to EMR use e.g. is there a clear and demonstrable correlation between the use of EMRs and the indicator? Of the original indicators, 21 were identified as priority indicators, laying the foundation for a longer term BR Framework and measurements that may be used for future analysis. The indicators were also classified into nine categories of benefits (see Appendix C, Revised EMR Benefits Realization Framework). Priority indicators were identified, specifically for assessment in Phase 2 of the Benefits Realization study based on known available EMR functionality, maturity of use and feasibility of measurement at case study sites. A total of 14 indicators were initially selected for the EMR BR Framework - Case Study Indicator Subset (see Appendix D), with study questions in related categories including the following: Table 1 : Case Study Questions Category Study Questions Laboratory Management Does EMR use reduce lab result turnaround time? Communication and Coordination of Care Does EMR use improve access to information between settings? Does EMR use facilitate referral to specialists? Does EMR use facilitate interdisciplinary/team care? Do physicians access the EMR remotely to provide patient care? Chronic Disease Management Does EMR use improve chronic disease management? Health Promotion, Screening and Prevention Does EMR use improve preventative services provided? Efficiency Does EMR use improve efficiency of care?
  • 18. Benefits Realization Study for EMRs in Ontario PwC 14 3.2 Phase 2: Analysis of Benefits Realization Phase 2 focused on assessing both current and potential benefits realized through EMR use. In order to undertake this assessment, a series of case studies were undertaken within six primary care settings. The information collected within these sites together with peer reviewed studies served as inputs to a modelling exercise that defined potential benefits that could be realized with advanced EMR use and adoption across Ontario. 3.2.1 Case Studies At the core of the second phase were the case studies, designed to understand the benefits realized and key lessons learned through the use of EMRs in selected clinics across Ontario that were identified as high performing users of EMRs. While the emphasis of the study was to understand both quantitative and qualitative processes and outcomes as per the indicators defined in the BR framework, due consideration was given to identifying relevant insights, experiences and lessons learned that supported benefits realization and demonstrate the role of EMRs (direct and indirect) in realizing these benefits. Case Study Site Selection Process In order to select case study sites, a preliminary list of ten primary care clinics was compiled by the PeHP and PwC with input from stakeholders, including OntarioMD. Each potential site was assessed based on the following preferred criteria.  EMR Maturity/Experience  Location / Geography / Site Type  Vendor System and Integration  Available and Accessible Documentation  Innovation  Willingness to Participate 1. EMR Maturity/Experience: The case study site will ideally have an EMR in use by clinicians for at least three years and demonstrate mature use of the EMR. The OntarioMD EMR Maturity Model (EMM) illustrated in Figure 1 was used as a guide to provide information on EMR effectiveness and to identify sites that were considered “high performing”, with the ability to measure the impacts on the various indicators. See Appendix E for a more comprehensive description of the EMM.
  • 19. Benefits Realization Study for EMRs in Ontario PwC 15 Figure 1: OntarioMD EMR Maturity Model 2. Location / Geography / Site Type: To ensure representation from across Ontario and different clinic types, geographic factors were considered e.g. urban and rural, in addition to different primary care clinic types, e.g. small practice groups, large groups. 3. Vendor System and Integration: Representation from clinics using a variety of Ontario-certified vendor systems was considered. Another site consideration was to select those with various degrees of connectivity and interoperability (connectivity with OLIS, and access to electronic receipt of hospital reports). 4. Available and Accessible Documentation: The ability of the sites to provide access to documentation that would facilitate measurement of the indicators in the framework was an important criterion. 5. Innovation: Practices that were seen as being innovative and leading edge in the way they use their EMR were rated more highly. 6. Willingness to Participate: The willingness of sites to participate in all aspects of the study was an important criterion. Candidate sites were contacted and asked to participate. Six practices agreed and were the subjects of the case studies. Case Study Tools and Assessment Case studies were conducted at the selected six practices. The case studies focused on addressing the following high level study questions as a means of gathering information related to the subset of indicators, advanced application of EMRs within the clinical setting and lessons learned:  How are EMRs used in high performing clinical practices?  What are the best practices and lessons learned with regards to adoption and use of EMRs?
  • 20. Benefits Realization Study for EMRs in Ontario PwC 16 Evidence for the case studies was collected from the following four sources. Descriptions of the tools and methods are described below:  Surveys (Organizational and Provider)  Interviews  Direct observation  Data extraction Surveys Prior to the case study site visits, two surveys were administered by PwC: 1. Organizational Survey: This survey focussed on gathering general background information related to context, organizational resources and EMR users. It was completed by the Clinical Lead or designate at the site. 2. Provider Survey: Physicians, physician assistants and nurse practitioners were invited to complete an online provider survey prior to the site visit. This survey focussed on gathering information on EMR adoption and current use. All providers were given the option of completing the tool; however, those who had been selected for an interview were required to complete it. a. Provider Survey Part 1: EMR Benefits- Part 1 of the survey focused on providers’ perception of benefits realized from EMR use. b. Provider Survey Part 2: OntarioMD Progress Survey- Part 2 of the survey focused on the measurement of maturity across key functional areas, and was representative of OntarioMD’s EMR progress survey. The survey responses provided valuable information prior to the site visits about each practice, as well as: EMR use and adoption; EMR maturity; and, general perceptions around EMR use. The information obtained was used to tailor the interview guide for each individual on-site visit. The Organizational and Provider Surveys are presented in Appendices F and G respectively. Interviews On-site interviews were conducted with clinicians and team members (including administrative staff) whose responsibilities involved interaction with some aspect of the EMR. For the physicians, the Clinical Lead along with other physicians (a representative number relative to the size of the practice) were interviewed. A total of thirty- nine physicians and clinical staff were interviewed across the six sites. The interviews contained open-ended and closed questions to expand the data gathering and to increase the number of sources of information. A structured interview guide was developed and emailed to each interviewee prior to the interview (see Appendix H). As well, the case study subset of EMR indicators was also provided to the site prior to the site visit (Appendix D). Qualitative data from the interviews were used as a proxy for indicator measurement where EMR date extraction was not feasible. Direct Observation Where possible, the study team observed clinician and staff use of the EMR while on-site. This supported the information gathering related to workflow and specific use of EMRs in the practice setting. In addition, it served to validate the information obtained through the surveys and interviews. Because of Personal Health Information restrictions, the study team did not partake in any activities related to direct patient care or where unique patient identifiers were visible to the study team.
  • 21. Benefits Realization Study for EMRs in Ontario PwC 17 Data Extraction With assistance from Clinical Leads, IT leads, or data analysts, EMR reports were extracted from the EMR systems in each site, where possible, in order to measure benefits related to specific indicators. 3.2.2 Final Revisions to Benefits Realization Framework Following the completion of all case study site visits, the framework and indicators were reviewed and discussed with members of the Steering Committee. Based on data obtained and feedback received, the framework was revised for analysis and reporting purposes. The Steering Committee agreed to focus case study analysis on indicators that would be representative of the BR framework and were most readily accessed and quantified and/or qualified. The revised BR framework with five categories and eleven indicators is presented in Table 2: Table 2: Final BR Framework for Case Study Reporting Purposes Category Indicator Reference Number Indicator Laboratory Management LM2 Average time between laboratory time of service and test results available in EMR Communication and Coordination of Care CC1 Average time to receive discharge summary following inpatient discharge CC3 Average time from referral decision to when the referral is sent CC4 % of practices where the care team has access to and uses the EMR system CC5 % of physicians who have remote access to EMR and use it for patient care Chronic Disease Management CDM1 % of PHC clients/patients, 18 years and over, with diabetes mellitus in whom the last HbA1c was 7.0% or less in the last 15 months CDM2 % of patient population, age 18 and older, with diabetes mellitus who received testing for diabetic complications Health Promotion, Screening and Prevention HPSP1 % of practice population, age 65 and older, who received an influenza immunization HPSP2 % of practice population, age 50 to 74, who had a screening test ordered for colon cancer HPSP3 % of practice population, age 18 and older, who have had their blood pressure measured by their primary health care provider within last 15 months Efficiency E1 % change in time to complete clinical/admin documentation
  • 22. Benefits Realization Study for EMRs in Ontario PwC 18 The values collected from case study sites for the indicators in this framework are referenced and discussed in Section 4, Case Study Findings. 3.2.3 Modeling and Forecasting of Benefits An important activity undertaken in Phase 2 was the modeling and forecasting of the potential benefits associated with broad, mature use of EMRs in Ontario in five to ten years. A fundamental assumption is that this timeframe will allow widespread and more mature use of EMRs, similar to the advanced users of EMRs today, as represented by the case study sites. Benefits were extrapolated from indicators collected through the EMRs during the site visits and further validated with peer reviewed literature. It is to be noted that two studies (Hillestad et al, 2005 and Manitoba Health, 2012) have been referenced on multiple occasions for forecasting purposes, as the scope of these studies are in alignment with this benefits realization study. As outlined in Figure 2, development of each of the benefit estimates was an iterative approach. The EMR BR Framework was used to guide the development of potential EMR benefits. Indicators from the framework were selected for forecasting based on the availability of data for each indicator at the various sites as well as the availability of supporting evidence for the extrapolation of these indicators to province-wide benefits. Figure 2: Approach to Developing Benefit Estimates Comparative Indicators The first step consisted of comparing current values of indicators for leading practices and those for other practices in Ontario. For each indicator selected, minimum, maximum and average values were calculated based on data collected on site. In order to compare this with the current standard in Ontario, a search of the literature and data collection agencies (e.g. Statistics Canada and Canadian Institute for Health Information) was conducted. Where possible, Ontario estimates were used, although in certain instances Canadian or other provincial estimates had to be used as a proxy. This comparative estimate is an average value of the indicator for the entire province. The maximum value for each indicator collected on site was used as the projected indicator for Ontario practices in five and ten years. This was done in order to assess the full potential if all Ontario practices realized benefits similar to the high performing EMR-enabled practices, as represented by the case study sites. Associated Relative Benefit The second step involved assessing the relative benefit for the high-performing practices compared to other practices in Ontario. Although indicators provide some sense of the expected benefit, they are not realized benefits per se. For example, the share of the population that is immunized suggests improved health outcomes and avoided health care costs, however the immunization rate must be extrapolated in order to estimate these actual benefits. A search of the literature was conducted to attribute a benefit to each indicator. The search prioritized impacts on resource use or costs. Indicator estimate from site visits Comparative indicator for average Ontario practice Associated Relative Benefit OntarioExtrapolation OntarioBenefit Estimate Benefit Indicator
  • 23. Benefits Realization Study for EMRs in Ontario PwC Extrapolation to Ontario Population In the third step, relative benefits were extrapolated to the entire p collected for each benefit such as the number of benefit recipients in the province and the costs of various resources. In order to estimate the benefit projected to 2017 and 2022. The Ontario data was estimated using a number of sources including Ministry of Finance population projections and Cancer Care Ontario estimated using various techniques to extrapolate to provincial levels. Ontario Benefit Estimate The obtained benefit estimates provide an indication of the potential avoided costs providers in Ontario adopted an EMR and are calculated using identified formulae and financial impacts are presented in 2012 dollars Selected Indicators The figure below outlines the indicators selected for modelling by benefit category (Effic Screening & Prevention and Chronic Disease Management). As mentioned above, other indica feasibly modeled due to the lack of data extracted interviews conducted. Due to study limitations and the inability to model and forecast benefit values for a that the findings represent a subset of potential province EMR use. In addition, the above indicators themselves populations. For instance, benefits associated with influenza immunization were estimated in the over 65 age group only based on available evidence. 3.3 Study Limitations Limitations of the study approach include the following:  Generalization of Findings: The the representativeness of the findings considered as directional and a foundation for through broad and mature use of • Improved diabetes management Chronic Disease Management • Increased influenza immunization rates • Increased colon cancer screening rates Health Promotion, Screening & Prevention • Staff reduction in time spent on administrative tasks Efficiency Extrapolation to Ontario Population third step, relative benefits were extrapolated to the entire province. In order to do this, additional data was collected for each benefit such as the number of benefit recipients in the province and the costs of various resources. In order to estimate the benefit five and ten years from now, the number of benefit re projected to 2017 and 2022. The Ontario data was estimated using a number of sources including Ministry of Finance population projections and Cancer Care Ontario data. Where local data were not available, proxies were echniques to extrapolate to provincial levels. The obtained benefit estimates provide an indication of the potential avoided costs or quality of care outcomes providers in Ontario adopted an EMR and achieved similar results to the high performing case study sites and financial impacts are presented in 2012 dollars. The figure below outlines the indicators selected for modelling by benefit category (Efficiency, Prevention and Chronic Disease Management). As mentioned above, other indica led due to the lack of data extracted from EMRs or the inability to derive estimates from the Due to study limitations and the inability to model and forecast benefit values for all indicators, it is acknowledg that the findings represent a subset of potential province-wide benefits that can be derived from broad and mature ition, the above indicators themselves capture a subset of benefits and often for specific patient . For instance, benefits associated with influenza immunization were estimated in the over 65 age group imitations of the study approach include the following: : The intentional focus on six practices with advanced use of the representativeness of the findings for the province as a whole. Accordingly, the findings should be a foundation for “the art of the possible” for benefits that can be realized broad and mature use of EMRs in primary care settings across Ontario. Improved diabetes management Chronic Disease Management Increased influenza immunization rates Increased colon cancer screening rates Health Promotion, Screening & Prevention Staff reduction in time spent on administrative tasks 19 rovince. In order to do this, additional data was collected for each benefit such as the number of benefit recipients in the province and the costs of various years from now, the number of benefit recipients was projected to 2017 and 2022. The Ontario data was estimated using a number of sources including Ministry of . Where local data were not available, proxies were or quality of care outcomes, if all case study sites. Values iency, Health Promotion, Prevention and Chronic Disease Management). As mentioned above, other indicators could not be s or the inability to derive estimates from the ll indicators, it is acknowledged wide benefits that can be derived from broad and mature and often for specific patient . For instance, benefits associated with influenza immunization were estimated in the over 65 age group practices with advanced use of EMRs limits , the findings should be benefits that can be realized
  • 24. Benefits Realization Study for EMRs in Ontario PwC 20  Sampling: Case study sites were selected based on identified criteria and in consultation with OntarioMD and the PeHP at eHealth Ontario. Accordingly, the results reflect this sample bias. In addition, survey participants were identified by the clinical leader rather than being randomly selected, which helped ensure that interviewees were advanced EMR users but also introduced a sample bias.  Attribution of Benefits to EMRs: It is difficult to establish a strong correlation between EMR use and measures among the different practice environments because there are many other variables that contribute to outcomes measured by the indicators (e.g., interdisciplinary care, well established workflows, payment models, etc.). Anecdotal evidence was acquired to support findings, and define the nature of the attribution of EMR use to benefits realized. In many instances, the benefits are indirectly attributable to EMR use, but it was widely acknowledged that several benefits cannot be realized as quickly nor as effectively in the absence of EMR use, i.e. in a paper-based clinical setting.  EMR Maturity: While maturity of EMR use is improving, the use of EMRs is still evolving across Ontario. Although the case study sites were identified as advanced users of EMR, there was variability in EMR maturity and connectivity to other systems. Accordingly, interview data was not always comparable across sites as interviewees were not able to provide a consistent perspective on EMR capability and, as a result, provided varying levels of detail regarding EMR use and impact.  Lack of Data Quality, Standardization, and Extraction/Reporting Capabilities: To varying degrees, each site had difficulty extracting high quality, standardized data from its EMR. Accordingly, it was difficult for the sites to report EMR impact on specific indicators.  Lack of Baseline Measures: All case study sites estimated pre-implementation indicator values in the absence of measured baseline information for the selected indicators. Indicator values prior to EMR implementation were typically estimated from respondents’ memories and as a result, may not be accurate.  Ability to Forecast EMR Adoption and Use: The modeling approach assumes that, by 2017, all community-based practices in Ontario will have adopted EMRs and will benefit from EMRs to the same degree as the case study sites. The pace of EMR adoption in Ontario over the next five years is uncertain and will likely not be on a “straight line” basis, i.e., rates of adoption and benefits realization will likely vary on a year-to-year basis. Although there was a large increase in EMR adoption in Canada from 2009 to 2012 with the Commonwealth Fund Survey reporting an increase from 37% to 56%, a linear increase to 2017 and in turn 2022 cannot be assumed.  Limited Evidence for Modeling: The model addresses a limited number of benefits based on data collected by EMR practices to date and available evidence in the literature. For some indicators, practices were not able to extract the data from their EMR. For other indicators, although values could be extracted, there was insufficient evidence regarding the impact of the indicator on cost or quality of care outcomes (e.g., average time to receive charts after emergency department). In addition, some benefits of EMRs are not yet reflected by indicators so the benefits cannot be estimated (e.g., immunization in young children).
  • 25. Benefits Realization Study for EMRs in Ontario PwC 21 4. Case Study Results Findings obtained from the six primary care case study sites are reported in the section below. The findings reflect the quantitative and qualitative benefits that have been measured and realized by the six sites, with data compiled from provider surveys, site visit data collection as per the selected indicators and in-person interviews. This section presents results of the provider surveys, followed by indicator and interview results in alignment with indicator categories. Following the presentation of findings and modelling and forecasting results in Section 5 that follows, a discussion is presented in Section 6 with a focus on the broader potential benefits that can be realized through mature EMR adoption and use across Ontario in primary care settings. 4.1 Provider Survey Results The survey of 28 physicians, physician assistants and nurse practitioners from the six case study sites focused on providers’ assessments of the impact of EMRs on quality, communication and coordination of care, efficiency, and the patient experience in the primary care setting. Results, demonstrating levels of agreement with specific impact statements, were very positive and are presented in the four bar chart figures that follow. Figure 3: EMR Impact on Quality *Statements have been truncated (See Appendix G for complete survey questions). 70.4 70.4 55.6 40.7 48.1 85.2 33.3 55.6 29.6 25.9 37 48.1 37 14.8 51.9 40.7 3.7 7.4 7.4 14.8 7.4 3.73.7 7.4 Enhancesthequalityofcare deliveredtopatients* Improvesthemanagementof chronicdiseases Improvespatientsafetyand proactivemonitoring* Enablesidenitificationof patientsforchangesin management* Improvesthedecision- makinge.g.decisionsupport tools* Supportsremoteaccessand useforpatientcare Enablesthepracticetodo needsplanning* Enablesthepracticetoaudit andimprovethepractice PercentAgreementorDisagreement Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
  • 26. Benefits Realization Study for EMRs in Ontario PwC 22 The eight survey questions posed reflect a very broad definition of “quality”, addressing aspects that include:  Quality of care delivered  Chronic disease management  Patient safety and proactive monitoring of conditions  Clinical decision-making  Remote access  Needs-based planning  Practice improvement Very strong levels of agreement were received in response to all of the questions posed, reflecting the significant range of impact and benefits that providers believe may be directly or indirectly attributed to the use of EMRs. On average, 93.0% of respondents either strongly agreed or agreed with statements presented for EMR benefits related to quality of care. Figure 4: EMR Impact on Communication and Coordination of Care Questions around the impact of EMRs on communication and coordination of care were more narrowly focused on internal sharing of information, supporting interdisciplinary care coordination and overall practice’s ability to coordinate patient care. 100% of survey respondents agreed or strongly agreed that the use of EMRs made a positive impact on those activities related to communication and coordination of care, indicating the tremendous impact that EMRs can have in the clinical practice setting. 92.6 92.6 74.1 7.4 7.4 25.9 Improves the sharing of patient information with providers internal to our practice Supports interdisciplinary care coordination in our practice Enhances our practice’s ability to coordinate patient care PercentAgreementorDisagreement Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
  • 27. Benefits Realization Study for EMRs in Ontario PwC 23 Figure 5: EMR Impact on Efficiency *Statements have been truncated (See Appendix G for complete survey questions). Nine survey questions were posed around efficiency, again reflecting a broad definition of “efficiency”, addressing aspects that include:  Efficiency of ordering lab tests and prescriptions  Practice productivity (increased output and reduction of effort)  Time spent responding to call backs or other pharmacist requests  Availability of test results (turnaround time)  Administrative efficiencies  Preventative care incentives  Claims management processes  Management of overhead costs Overall, 81.5% of survey respondents strongly agreed or agreed with statements presented for EMR benefits related to efficiency of practice. Findings suggest that while the majority of respondents perceive their EMR system to positively impact the efficiency of their practice, there is some level of disagreement around the efficiency of certain aspects of EMR use (vs. a paper-based environment). This is most pronounced around the ability of EMR use to positively impact claims management processes and the management of overhead costs. 63 59.3 44.4 33.3 77.8 70.4 44.4 44.4 14.8 29.6 33.3 33.3 44.4 18.5 18.5 44.4 29.6 29.6 7.4 7.4 18.5 18.5 7.4 3.7 7.4 18.5 3.7 3.7 3.7 7.4 3.7 7.4 18.5 29.6 Enhancestheefficiencyofordering labtests,prescriptions,etc. Improvestheproductivity(output)of mypractice Improvestheefficiency(reductionin effort)ofmypractice Reducesthenumberofcallbacks and/ortimespentrespondingto pharmacistrequests* Reducesthetimefromwhenalab testresultisavailabletowhenthe resultisreceivedbytheEMR Improvesadministrativeefficiencies e.g.useofpre-populatedtemplates Facilitatespreventativecare incentivese.g.throughcohort management Improvestheclaimssubmission processe.g.decreaseinbillingerrors Hasfacilitatedareductioninnet overheadcostse.g.space,office suppliesetc. PercentAgreementorDisagreement Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
  • 28. Benefits Realization Study for EMRs in Ontario PwC 24 Figure 6: EMR Impact on the Patient Experience The two questions posed around patient satisfaction and patient education indicated strong levels of agreement, with 94.4% indicating that EMRs have a positive impact on both. However, the majority of respondents simply agreed (rather than strongly agreed) with these statements. This suggests that the large majority of respondents perceive their EMR improves the overall patient experience, but that there could be other, and perhaps stronger, contributing factors to patient experience. Survey results are further referenced in support of the findings presented on the indicators and interviews below. 4.2 Indicator and Interview Results As described in Section 3, above, the priority indicators that were identified in Phase 1 were assessed for each of the six case study sites. Data to support the priority indicators were obtained from interviews, provider surveys and EMR data extractions. Benefits are assessed and presented as per the five categories and indicators identified in the EMR BR Framework Case Study Subset and in Table 2. In addition, findings and anecdotes from interviews related to each of the five categories are presented, with selected supporting quotes from providers. A sixth category, Medication Management, emerged in discussion with several providers during the interviews. Although there were no indicators studied for Medication Management, the interview findings are presented. 1. Laboratory Management 2. Communication and Coordination of Care 3. Chronic Disease Management 4. Health Promotion, Screening and Prevention 5. Efficiency 6. Medication Management 37 44.4 59.3 48.1 3.7 7.4 Improves patient satisfaction/experience with the care they receive Supports patient education (e.g. trending patterns for BMI, blood sugar levels etc.) PercentAgreementorDisagreement Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
  • 29. Benefits Realization Study for EMRs in Ontario PwC 25 4.2.1 Laboratory Management The management of laboratory information was determined to be a key area of interest and a feasible area to assess in the current EMR environment. Primary care settings are connected (to an extent) to community (private) laboratories through a direct feed enabling them to receive laboratory results electronically into their EMRs. This simulates the benefits that will be obtained as the Ontario Laboratories Information System (OLIS) is rolled out in primary care settings. Some physicians indicated very recent connection to the OLIS but results and/or trends were not available at this time. The impact of EMR use on Laboratory Management was assessed through a combination of provider survey results and interviews for one case study indicator. Results are presented in Table 3 below. Table 3: Laboratory Management Indicator Results ID # Indicator Definition Source Indicator Values LM2 Average time from laboratory time of service and test results available in EMR Interview Pre EMR: Total turnaround ≈12 hours to 5 days Post EMR: Real time – 24 hours Responses from the provider survey indicated that 100% of participants felt that the EMR reduces the time from when a laboratory test result is available to when the result is received by the EMR. However, indicator values varied among case study sites. Some case study sites reported having laboratories physically on site. Their pre-EMR laboratory turn around time averaged 24 hours for routine bloodwork and was generally within 12-24hours with an EMR. As expected, the time for transmission of results was not a significant factor prior to EMR implementation as results were commonly “walked over”. However, sites that previously relied on paper delivery of results from community laboratories and which now receive electronic reports directly into their EMR reported a noticable difference in the turnaround time post-EMR implementation. Paper-based results were reported to take approximately 3-5 days, but those sites now report receiving information in “real time” when the result is ready for distribution. For most study sites, a minimum of 50% decrease in lab result turn around time was reported with results received electronically in their EMR. Physicians also perceived EMRs to improve the efficiency of ordering lab tests. Findings from the provider survey indicated that 63% of respondents strongly agreed that the EMR enhances the efficiency of ordering lab tests, prescriptions etc. contributing to the overall management of laboratory information. Additional Interview Findings Some physicians reported accessing their EMRs remotely in the morning before their clinic day or on weekends to prepare for the day or week. Consequently, test results requiring immediate follow-up were able to be addressed sooner by scheduling the patient for a follow-up visit, making a referral, or sending a prescription to the pharmacy (depending on EMR capability). It was also reported that some physicians are able to access lab reports on their smart phones, making remote access even easier. Physicians are able to access lab results in “real time” throughout the day in many locations. Some physicians indicated that they enjoyed the ability to, throughout the day, go through their “inbox” (the results also were in the patient chart) and sign off results as they come in, demonstrating how some physicians have adopted a change in workflow to manage laboratory information. Previously, results may have been signed off in batches at the end of the day; now, physicians can access their EMR in the clinic room, in their offices or in hallways, enabling them to review and sign laboratory “I access labs and charts from many locations throughout the day. As a result I can make faster decisions related to next steps in patient care”.
  • 30. Benefits Realization Study for EMRs in Ontario PwC 26 results from many locations throughout the day. A few case study sites reported the use of a patient portal in conjunction with their EMRs. Some sites reported that their patients’ experiences were improved through the sharing of laboratory results via a secure patient portal. Physicians reported a reduction in patients’ anxiety by enabling them to check their laboratory results themselves from home, and others indicated that their patient portal allowed them to post comments along with the results they selected for sharing with patients. This helped patients better understand the context of the result. 4.2.2 Communication and Coordination of Care Communication among health care providers and coordination of care were both identified as key areas of interest for the case studies. In particular, the need was identified to better understand how the use of EMRs has impacted both intra-office and inter-office communications. Study sites were more responsive to discussing and identifying benefits related to intra-office communication and coordination, as the use of EMRs in supporting inter-office communication is perceived to be in its very early stages. Case study findings related to five defined indicators regarding communication and coordination of care are presented in Table 4. Table 4: Communication and Coordination of Care Indicator Results ID # Indicator Definition Source Indicator Values CC1 Average time to receive discharge summary following inpatient discharge Interview Pre EMR: 4 to 14 days Post EMR: Real time to 48 hours CC3 Average time from referral decision to when the referral is sent Interview Pre EMR: 1 to 7 days Post EMR: Real time to 1 day CC4 % of practices where the care team has access to and uses the EMR system Provider Survey Interview 100% CC5 % of physicians who have remote access to EMR and use it for patient care Provider Survey Interview 100% For indicators CC1 and CC3, interviewees were able to estimate a pre-implementation value depending on their personal experiences. While those estimates of pre-implementation values ranged significantly among providers, the reduction in times quoted were, on average, significantly lower demonstrating reductions in time spent waiting for discharge information and/or referrals in the range of 85% or more. These reductions in time spent waiting for discharge summaries and referrals expedite and facilitate the coordination of care such that patient needs can be addressed in a timely fashion. For indicator CC1, physicians reported that, prior to EMR implementation, hospital discharge summaries were available within 1-2 weeks of discharge and are now available with tools such as the Hospital Report Manager or direct connectivity, either in real-time or within two days. With respect to referrals and indicator CC3 specifically, physicians were able to estimate the average time to complete a referral letter and send it to the specialist. Physicians widely reported that the referral process1 is much more efficient with EMRs because of the pre-populated data in the referral forms. Some physicians also reported using macros or templates to further support the process. This finding was also supported in the provider survey 1 Herein refers to as “from referral decision being made to the reference request letter completed/faxed.”
  • 31. Benefits Realization Study for EMRs in Ontario PwC 27 with 70.4% of respondents reporting that EMRs improve administrative efficiencies through the use of features such as pre-populated templates, forms and stamps. Following the development of the referral letter, there was a mix of workflow processes reported. Some physicians reported they sent out the referral letter themselves through the EMR with attachments while other physicians reported consolidating the letter and supporting documentation to send to a referral clerk/ nurse for submission and tracking. Overall, it was observed that the EMR improves physicians’ ability to make timely referrals, expediting the care process and facilitating inter-office communications. For indicator CC4, 100% of practices reported that their entire care team has access to the EMR. Survey responses further emphasized the access and importance of EMRs in providing interdisciplinary care and intra-office communication; 92.6% of survey respondents strongly agreed that the EMR system supports interdisciplinary care coordination in their practice. In addition, 92% of respondents strongly agreed that the EMR system improves the sharing of patient information among providers internal to the practice. For indicator CC5, it was evident at all sites that providers are accessing the EMR and using it remotely for patient care purposes. 85.2% of survey respondents strongly agreed that EMRs support remote access and use of information by providers for patient care. The ability to access patient information at any time and in any place was noted by all as a tremendous asset to providers and their patients. EMRs also improve the ability to communicate within clinics and often with providers outside of clinics, with overall results improving the efficiency of patient care. Additional Interview Findings Interviews with clinic staff revealed many additional benefits of care coordination from EMRs. For example, EMRs facilitate the ability of physicians to set aside dedicated time for same/next day appointments for patients with acute needs. Physicians and team members reported that improvements in scheduling and practice organization facilitate their ability to hold same/next day appointments open. In addition, EMRs provide physicians and care team members with the ability to view everyone’s schedule to better coordinate care across team members. With respect to communication across the team, it was overwhelmingly reported that the illegibility of hand-written clinical notes was formerly a challenge when sharing information among team members. Interviewees reported that the sharing of legible information in the EMR has improved the quality of care delivered. Team members are now able to communicate with one another in real time, ask questions and assign tasks among the team. The patient chart is now accessible to many multiple care team members at the same time, a feature that was not possible in paper. Through the use of EMRs, most physicians reported that the information they provided in referral letters to specialists was more comprehensive, especially for complex cases. It was also reported that specialists appreciate the extensive clinical information provided as part of the referral and facilitated by EMRs. Some physicians reported that they were able to quickly select attachments e.g. diagnostic test results, to send along with the referral letter. “EMRs improve scheduling, resulting in increased capacity and organization of our practice, all of which results in increased access for patients”. “EMRs improve transfer of information to specialists - it is easier to provide consultations with lots of relevant information”.
  • 32. Benefits Realization Study for EMRs in Ontario PwC 28 Lastly, there was consensus among users of the Hospital Report Manager (HRM) (whereby dictated and transcribed hospital reports are transformed into a standards-based data schema that can be directly updated into the patient’s chart in an electronic format) that it has decreased the average time to receive hospital reports. The implementation of HRM has reduced transcription time and significantly decreased hospital report turnaround time. Some practices reported receiving hospital reports directly into their EMRs within thirty minutes of transcription. As a result, proper clinical decisions can be made more effectively to improve the quality of care for patients. 4.2.3 Chronic Disease Management Chronic disease management is an emerging area of interest related to EMR use. Case study sites reported that EMRs facilitate significant and improved changes to the way chronic conditions are managed. Both quantitative and qualitative findings were significant, with physicians expressing a desire to expand their use of EMRs for this purpose. The impact of EMRs on the management of patients with chronic conditions was assessed through a combination of provider survey results, interviews and EMR data extraction. Findings related to the three defined indicators are presented in Table 5. Table 5: Summary Chronic Disease Management Indicator Results ID # Indicator Definition Source Indicator Values CDM1 % of PHC clients/patients, 18 years and over, with diabetes mellitus in whom the last HbA1c was 7.0% or less in the last 15 months EMR Data Extraction Range: 41 - 70% CDM2 % of patient population, age 18 and older, with diabetes mellitus who received testing for all of the following:  Hemoglobin A1c  Full fasting lipid profile screening  Nephropathy screening  Foot examination  Blood pressure measurement  Obesity/overweight screening EMR Data Extraction 34% Overall, provider survey respondents agreed that EMRs improve the management of chronic diseases. Specifically, 70.4% strongly agreed with the statement and 25.9% moderately agreed. 92.5% of survey respondents also reported that the EMR system supports patient education. For example, the EMR provides access to handouts, references or tools to trend patters for BMI, blood sugar, and other labs. Collectively, the educational materials and tools help patients better understand and manage their chronic conditions. CDM1 and CDM2 were measured across all sites, with values ranging from 41-70% and 34% respectively. These results demonstrate the ability to identify a target diabetic population and deliver appropriate, evidence-based care. Physicians and care teams are better able to monitor specific clinical indicators such as HbA1c, and develop care plans tailored to populations. 85.2% of provider survey respondents agreed that EMRs enable practices to perform “Through HRM and my EMR, I receive reports from the local hospital as soon as they are prepared. No more calling the hospital for reports. I have the information immediately available to review”.
  • 33. Benefits Realization Study for EMRs in Ontario PwC 29 needs-based planning e.g. plan for a specific patient population/community. Without EMRs, this is very time consuming, costly and a nearly impossible task. Additional Interview Findings It was evident in discussions with physicians and care team members that they were all motivated to better manage the health of various patient populations. There appears to be a movement away from “chair to chair care” with an increasing emphasis on patient populations. Sites reported that because they are easily able to identify patient populations, they are able to proactively follow up with patients, schedule recall appointments or send them for further testing as needed. Nurse practitioners reported that their practices had many patients with chronic diseases, and they found that the use of EMR tools, e.g. diabetes template/ flow sheet, was very useful when seeing diabetic patients. Relevant information was quickly accessible and consolidated, and the flow sheet ensures that best practice guidelines are incorporated into care delivery. Care team members are aware of what needs to be attended to during the visit, improving efficiency and what they should be planning for in the coming months in order to manage the condition. Most physicians and care team members reported using trending or graphing tools as patient education materials e.g. blood pressure, weight, HbA1C were commonly trended. It was reported that the ability to review consult notes, labs etc with patients builds confidence of care and enhances communication”. In addition, some physicians noted that they were beginning to communicate with patients (especially those with chronic conditions) via secure email. 4.2.4 Health Promotion, Screening and Prevention Keeping Ontarians healthy through health promotion, prevention and screening initiatives is an area of benefits to be explored through the use of EMRs. This aspect of the study is complementary to that of chronic disease management, with a focus on targeting at risk populations to improve quality of care. The impact of EMRs on health promotion, prevention and screening was assessed through a combination of provider survey results, interviews and EMR data extraction. Findings related to the three defined indicators are presented in Table 6. Table 6: Health Promotion, Screening and Prevention Indicator Results ID # Indicator Definition Source Indicator Values HPSP1 % practice population, age 65 and older, who received an influenza immunization* EMR Data Extract Range: 52 - 80% HPSP2 % of practice population, age 50 to 74, who had a screening test ordered for colon cancer EMR Data Extract Range: 51 - 60% HPSP3 % of practice population, age 18 and older, who have had their blood pressure measured by their primary health care provider within last 15 months EMR Data Extract Range: 61 - 82% “For diabetes care I can trend A1C for several years – you can discuss and show the patient how they are doing”.
  • 34. Benefits Realization Study for EMRs in Ontario PwC 30 Overall, EMR data extraction was quite feasible for the above selection of indicators, suggesting that case study sites have been working towards standardizing their data input in order to use the data in a meaningful way to maintain a healthy population. For each of the three indicators, identified patients who had received appropriate preventative treatments (as per the indicators) were greater than 50% and as high as 82%. The role of the EMR in facilitating the identification of patients, communication with those patients, and ordering and/or documentation of the required testing is very important. Providers indicated that these tasks are greatly enabled by EMRs, and much more difficult to accomplish in a paper-based environment. Provider survey results further emphasize the role of the EMR system in promotion, prevention and screening activities, with 92.6% of survey respondents agreeing that EMRs improve patient safety and the proactive monitoring of overdue tests/ exams. Enabling case study sites to track their patient populations through the EMR and compare their indicator values to provincial benchmarks gives them an opportunity to identify areas for improvement in health promotion, screening and prevention activities. Sites are in the early stages of generating reports to proactively screen their patient populations, and priority areas of focus can be linked to financial incentives for preventative care. It should be noted that the provincial target for colorectal cancer screening is 40% (Health Quality Ontario, 2012) and that the values reported by all the case study sites exceeded the provincial average. In addition, 44.4% of provider survey respondents strongly agree, while 44.4% moderately agree that EMRs facilitate preventative care incentives e.g. through cohort management for influenza immunizations. Additional Interview Findings All of the case study sites reported that they were motivated to use EMRs for population health purposes. Physicians and care teams performed simple searches with minimal parameters to identify target populations for disease prevention. For example, as part of its family health team quality improvement plan, one site reported pulling lists of eligible/ not eligible patients for activities such as cervical cancer screening, breast cancer screening and colon cancer screening, and sharing the list among physicians to coordinate screening tests. Another common example discussed with interviewees was the EMR’s ability to support efforts to increase influenza immunization rates. Not only have practices adopted processes for targeting populations and sending reminder letters (or emails where available) but they have also developed and tested new ways of measuring their flu vaccination rates, including a dashboard that presents vaccination rates among patients of physician peers. Some physicians and care team members reported that they were better able to track preventative health tests and recalls, with the EMR generating reminders for the proactive scheduling of wellness visits. Sites also reported that they engage patients in community wellness talks through identification of those in target populations, and speak to them about prevention and promotion topics relevant to their cohort. EMRs afford providers the ability to be very proactive and interactive with patients in managing their own health. “EMRs provide the ability to survey your practice to find people who are due for preventative procedures or care, including finding all the people who have not had a pneumovax.”
  • 35. Benefits Realization Study for EMRs in Ontario PwC 31 4.2.5 Efficiency Obtaining a better understanding of the overall practice efficiencies realized through the use of EMRs was a focus for the case studies. As discussions evolved, it became evident that practice and provider efficiency was a recurring theme that was applicable to several other discussion categories as well. Consequently, results are presented for one indicator only in this section, but findings about efficiencies created are also referenced elsewhere in this report. Table 7: Summary Efficiency Indicator Findings ID # Indicator Definition Source Indicator Values E1 % change in time to complete clinical/admin documentation by using pre-populated templates, forms and stamps Interview ~ 50% decrease Overall, 77.7% of survey respondents agreed with the statement that EMRs “improve the efficiency (reduction in effort) of my practice” and of those 44.4% strongly agreed with the statement while 33.3% moderately agreed. These survey responses mirror the comments collected during the interviews on overall practice efficiencies. With respect to clinical and administrative documentation (indicator E1), physicians were able to report overall that there have been efficiency changes related to certain activities. For example, 92.6% of survey respondents agreed with the statement that EMRs “enhance the efficiency of ordering lab tests, prescriptions, etc” (63.0% strongly agree and 29.6% moderately agree). During interviews, physicians reported that the auto-population of templates (where appropriate, e.g. lab requisition form, s.o.a.p. format encounter notes and others) was an added benefit and saved time for themselves and their administrative staff in completing fields such as demographics. For indicator E1, forms (including templates with the ability to auto-populate existing EMR data) and stamps (e.g. s.o.a.p note or diabetic assessment template used to capture information) were acknowledged as a benefit to clinical and administrative documentation. However, physicians and care team members found it difficult to quantitatively assess the impact, with most estimating the time savings at 50% when compared to a paper-based environment. Survey results support the benefit of EMRs in administrative efficiencies with 70.4% of providers perceiving EMRs to improve administrative efficiencies such as through the use of pre-populated templates, forms and stamps. Additional Interview Findings In addition to the discussions around the above indicator, there was significant discussion on the efficiencies gained (or transferred to other activities) related to EMR use. There was a general consensus from interviewees that EMR features such as reminders allow for more efficient clinical encounters (e.g. patient physical exams), and not surprisingly, physicians and care team members reported that their time is not spent “flipping through papers” or searching for misfiled results. Instead, clinicians reported being able to address their list of planned activities such as reminding patients they are due for certain tests, procedures, medication refills, etc. “I have more efficient clinical encounters. The time is spent on more important things rather than writing out prescription renewals etc.”