2. Defining the New Blueprint
• Background and Context
• Process
• Information sources overview
• Candidates, the Blueprint & Test
Specifications
• Next Steps
2
3. ARTF Recommendations
3
1. LMCC becomes ultimate credential
(legislation issue)
2. Validate and update the blueprints for
all MCC examinations
3. More frequent scheduling of the exams and
associated automation (and harmonization
of MCCQE II)
4. IMG assessment enhancement and
national standardization (NAC & Practice
Ready Assessment)
5. Revalidation of physicians (FMRAC will
lead this one)
6. Implementation oversight, including the
R&D Committee priorities and R&D Budget
Recommendations to focus on MCC’s
reassessment and realignment of exams:
4. Core vs. Discipline-Specific
Competencies
Assessment Review Task Force:
• There is general consensus that the current
MCC examinations should concentrate on the
assessment of those core competencies,
including knowledge, skills, attitudes and
behaviours, required of every physician entering
independent practice
4
6. Purpose of the Blueprinting
• … is to assure the public that physicians
licensed to practice medicine have the
required knowledge, skills and attitudes for
safe and effective patient care.
• Only those who meet this standard are
qualified to enter professional practice
6
7. MCC’s Present Blueprints
• MCCQE Part I
– Equal distribution of questions based on discipline
• Medicine, Pediatrics, PHELO, Psychiatry, Obstetrics/
Gynecology, Surgery
• MCCQE Part II
– Distributed by discipline
– Also by skill
• History, Physical
Examination, Management, Counseling/Education, P
atient Interaction
7
8. Project Objectives
• ensure that critical core
competencies, knowledge, skills and behaviors
required of a physician entering supervised and
unsupervised practice are being appropriately
assessed
The process will
• ensure that MCC assessments continue to fulfill all the
requirements and standards for credentialing examinations
• provide a clearly documented and deliberate process to
• update exam specifications
• respond to ongoing developments in the profession
10
9. This afternoon…
• “Competency-based Assessment:
The Good, The Bad, and The Puzzling”
Dr. Kevin Eva
• “Defining the New MCC Blueprint”
Dr. Claire Touchie
• “MCC Blueprint: Building Consensus”
Blueprint Project Team
11
10. Defining the New Blueprint
Claire Touchie,
MD, FRCPC
for the Blueprint Project team
11. Purpose of this session
• Provide information about the process
• Review the blueprint and test specs
• Provide an opportunity for consultation over
the next two days
13
12. Purpose & Format
1. To engage Council members in a consultation about
the proposed Blueprint and Test Specifications
2. Format
a. This afternoon
• Gather initial feedback on the blueprint and test
specifications
b. Tomorrow
• Linkages with CanMEDS
• Workshop #1 – explore feedback from afternoon
• Workshop #2 – explore opportunities & next steps
14
13. Defining the Blueprint
Blueprint and Test Specifications
Defined 2 candidates Common Blueprint
Test Specifications for
each decision point
Subject Matter Experts
12 Experts 3 day meeting
4 Reports
Current Issues
PGY-1
supervision
Incidence and
Prevalence
National Survey
15
14. Documents used
Current Issues in Health
Professional and Health
Professional Trainee
Assessment
Supervising PGY-1
Residents
Incidence and Prevalence
National Survey
16
15. Supervising New PGY-1 Residents:
A Case Study of Supervisors expectations
vs.
Residents’ perceptions
16. Ten EPAs defined
1. Recognition and initial management of a critically ill
patient
2. Disclosure of medical errors
3. Interpretation of investigations
(laboratory, ECG, radiographs) with proper
communication of results to patients
4. Management of intravenous fluids
5. Handover of patient care to colleagues/other service
18
17. Ten EPAs defined
6. Discharge prescription writing including medication
reconciliation
7. Coordination of patient discharge/transfer (including
counseling of patient, organizing follow-up and
completing discharge summary)
8. Completion of admission and/or post-operative orders
9. Obtaining informed consent
10. Obtaining advanced directives/goals of care (code
status)
19
18. PGY1 CS PGY1 CS PGY1 CS
IVF Informed consent Goals of care
No supervision
Indirect supervision
Direct supervision
Not performed
EPAs that varied between
SUPERVISORS and RESIDENTS
20
19. EPAs Resident Responses:
DAYTIME vs. NIGHTTIME
Day Night Day Night Day Night
Critically ill Critically ill Handover Handover Patient D/C Patient D/C
No supervision
Indirect supervision
Direct supervision
Not performed
21
20. How does this help in
setting the blueprint?
• Helps to define who is the candidate at
Decision point 1 – entry to supervised practice
• Use this information to ensure that the
knowledge, skills and attitudes associated
with the EPAs are assessed prior to entering
residency
22
22. Incidence and Prevalence Data
• Determine what Canadian physicians see in
their practice
• E.g.: Frequency of clinical presentations
• Review certain areas of importance for the
practice of medicine in Canada
• Other needed specific competencies
• Care of Elderly
• Population Health
• Care of Aboriginal people
• Patient safety
24
23. Inpatient Hospitalization
(Excluding Maternal Cases)
Emergency Department Outpatient care/Clinics
1 Appendicitis Abdominal/ Pelvic Pain Anxiety
2 Gallstones Chest/ Throat Pain Supervision of Normal Pregnancy
3
Fracture of Lower Leg, Including
Ankle
Open Wound, Wrist/ Hand Depressive Disorder
4 Abdominal/ Pelvic Pain Back Pain Backache
5 Schizophrenia Other Medical Care Contraception Counsel/ Advice
6
Excessive and Irregular
Menstruation
Urinary Tract Infection Abdominal Pain
7 Convalescence Sore Throat Upper Respiratory Infection
8
Mental/ Behavioural Disorder
Due to Alcohol
Upper Respiratory Infection Urinary Tract Infection
9 Complications of Procedures Diarrhea and Gastroenteritis Essential Hypertension
10 Pancreatitis Sprain/ Strain of Ankle/ Foot Acute Pharyngitis
Main Diagnosis – 19 to 44
25
24. Main Diagnosis – 65+
26
Hosp. Inpt Emergency Ambulatory (AB) Ambulatory (NS)
COPD Chest/throat pain UTI HTN
Heart failure Other med care Chemotx DM II
ACS UTI Chest pain COPD
Pneumonia Abdo/pelvic pain Surg dressing Backache
Femur # COPD Other med care UTI
Knee arthrosis Pneumonia HTN Anxiety
Other med care Cellulitis Repeat prescription Pneumonia
Atrial flutter/fib. Heart failure Pneumonia Dementia
25. National Survey of the
Physicians, Pharmacists, Nurses
and Public
in Canada: 2013
26. • Provide the judged importance of the
knowledge, skills and attitudes (KSAs)
• Different stakeholders: Physicians,
Pharmacists, Nurses, and the public
28
Purpose of the
National Survey
27. • Based on the MCC Objectives
◦ Medical expert: expert
◦ Non-medical expert: communicator, collaborator, health
advocate, manager, scholar, and professional
• Two decision points
◦ Supervised: for a physician starting residency training who
is assessing a patient at the initial presentation.
◦ Unsupervised: for any newly licensed physician entering
unsupervised practice who is assessing a patient at the
initial presentation.
29
National Survey Design-
Physicians
32. Public – open ended question
As a person who used services
provided by the Canadian health care
system, what are the most important
competencies that a physician should
have?
34
33. Public Survey
0 500 1000 1500
Concerns around time
Centered on the patient
Knowledge/Credibility
Doctor Characteristics
Public Survey
35
34. Conclusions
1. Complete overlap of survey questions
“Very/Extremely Important” across supervised and
unsupervised decision points
2. Non-medical expert questions were proportionally as
important as the medical expert questions
(i.e., collaborator, communicator, professionalism) at
the first decision point
3. At supervised level
– Determine Cause slightly more important
4. At unsupervised level
– Initiate Management slightly more important
36
36. Defining the Blueprint
Blueprint and Test Specifications
Defined 2 candidates Common Blueprint
Test Specifications for
each decision point
Subject Matter Experts
12 Experts 3 day meeting
4 Reports
Current Issues
PGY-1
supervision
Incidence and
Prevalence
National Survey
38
37. Who were the SMEs?
Blueprint
MRA Rep of
Council
Central
Examination
Committee
Objectives
Committee
Test
Committees
RCPSCCFPC
UGME
Deans
PGME
Deans
University
Rep of
Council
39
38. SME Panel Meeting
– Defining the Proposal
Candidate
Descriptions
(D1 & D2)
Blueprint
Test
Specifications
(D1 & D2)
40
41. Proposed Common Blueprint
43
Dimensions of Care
Health Promotion
and Illness
Prevention
Acute Chronic Psychosocial
Aspects
PhysicianActivities
Assessment/
Diagnosis
Management
Communication
Professional
Behaviors
42. Definitions
Dimensions of
Care
Focus of care for the patient, family, community
and/or population.
Health Promotion
and Illness
Prevention
The process of enabling people to increase control over their health and its
determinants, and thereby improve their health. Illness prevention covers
measures not only to prevent the occurrence of illness such as risk factor
reduction but also to arrest its progress and reduce its consequences once
established. This includes but is not limited to screening, periodic health exam,
health maintenance, patient education and advocacy, and community and
population health.
Acute
Brief episode of illness, within the time span defined by initial presentation through
to transition of care. This dimension includes but is not limited to urgent,
emergent, and life-threatening conditions, new conditions, and exacerbation of
underlying conditions.
Chronic
Illness of long duration that includes but is not limited to illnesses with slow
progression.
Psychosocial
Aspects
Presentations rooted in the social and psychological determinants of health that
include but are not limited to life challenges, income, culture, and the impact of the
patient’s social and physical environment.
44
43. Definitions
Physician
Activities
Reflects the scope of practice and behaviors of a
physician practicing in Canada
Assessment/
Diagnosis
Exploration of illness and disease through gathering, interpreting and synthesizing
relevant information that includes but is not limited to history taking, physical
examination and investigation.
Management
Process that includes but is not limited to generating, planning, organizing care in
collaboration with patients, families, communities, populations, and health care
professionals (e.g., finding common ground, agreeing on problems and goals of
care, time and resource management, roles to arrive at mutual decisions for
treatment).
Communication
Interactions with patients, families, caregivers, other professionals, communities and
populations. Elements include but are not limited to active listening, relationship
development, education, verbal, non-verbal and written communication (e.g. patient
centered interview, disclosure of error, informed consent).
Professional
Behaviors
Attitudes, knowledge, and skills based on clinical and/or medical administrative
competence, communication skills, ethics, societal, and legal duties resulting in the
wise application of behaviors that demonstrate a commitment to excellence, respect,
integrity, empathy, accountability and altruism. (e.g., self- awareness, reflection, life-
long learning, scholarly habits and physician health for sustainable practice).
45
44. Consultations to date
• General comfort with and support of the
proposed Blueprint
• Varied responses for Psychosocial Aspects
– Name itself may be seen as pejorative
– Having it explicit may socialize it
– Should be incorporated in the other 3 Dimensions
of Care
• Initial considerations regarding weightings
between decision points
• Patient-safety is not explicit
46
47. Practice Poll
• Übermeetings tool
– Web: mcc.ubermeetings.com
– Text: 613-519-1313
• Who would you have liked to meet?
49
48. Proposed Common Blueprint
50
Dimensions of Care
Health Promotion
and Illness
Prevention
Acute Chronic Psychosocial
Aspects
PhysicianActivities
Assessment/
Diagnosis
Management
Communication
Professional
Behaviors
50. Consultation
• How well do the dimensions and titles
resonate with you?
– Do the Dimensions of Care resonate positively
with you?
– Do the Physician Activities resonate positively
with you?
– Should Psychosocial Aspects of care be used as
a title?
52
51. Consultation
• When you think of being a physician, what
key words are missing in the definitions?
53
53. Proposed Common Blueprint
55
Dimensions of Care
Health Promotion
and Illness
Prevention
Acute Chronic Psychosocial
Aspects
PhysicianActivities
Assessment/
Diagnosis
Management
Communication
Professional
Behaviors
54. Dimensions of Care
Health Promotion
and Illness
Prevention
Acute Chronic Psychosocial
Aspects
Row
Percent
PhysicianActivities
Assessment/
Diagnosis 30±5
Management 20±5
Communication 30±5
Professional
Behaviors 20±5
Column Percent 20±5 30±5 30±5 20±5 100
Assessment leading up to Decision 1:
Entry into Supervised Practice
56
55. Constraints
Decision 1 – Entry into Supervised Practice
Specification 1 – Constraints
CONSTRAINT
CATEGORY
DESCRIPTION CONDITION
Complexity multiple morbidities at least 10%
Age
Neonate, infant/child, adolescent, adult,
adult women of childbearing age, frail
elderly
sample across the age
categories including adult
woman of childbearing age
and the frail elderly
Gender male, female
balance evenly
(minimum of 40% each)
Special
populations
Included but not limited to immigrant,
LGBT, rural, disabled and First Nation
populations; end of life patients;
refugees; inner city poor, the addicted
and the homeless
sample across categories
Setting
Included but not limited to rural or remote
settings, long term care institutions and
home visits
sample across categories
57
57. Dimensions of Care
Health
Promotion and
Illness
Prevention
Acute Chronic Psychosocial
Aspects
Row
Percent
PhysicianActivities
Assessment/
Diagnosis 30±5
Management 20±5
Communication 30±5
Professional
Behaviors 20±5
Column Percent 20±5 30±5 30±5 20±5 100
Assessment leading up to Decision 1:
Entry into Supervised Practice
59
58. Dimensions of Care
Health
Promotion and
Illness
Prevention
Acute Chronic Psychosocial
Aspects
Row
Percent
PhysicianActivities
Assessment/
Diagnosis 25±5
Management 35±5
Communication 20±5
Professional
Behaviors 20±5
Column Percent 20±5 25±5 35±5 20±5 100
Assessment leading up to Decision 2:
Entry into Unsupervised Practice
60
59. Constraints
Decision 2 – Entry into Unsupervised Practice
Specification 2 – Constraints
CONSTRAINT
CATEGORY
DESCRIPTION CONDITION
Complexity multiple morbidities at least 20%
Age
Neonate, infant/child, adolescent, adult,
adult women of childbearing age, frail
elderly
sample across the age
categories including adult
woman of childbearing age
and the frail elderly
Gender male, female
balance evenly
(minimum of 40% each)
Special
populations
Included but not limited to immigrant,
LGBT, rural, disabled and First Nation
populations; end of life patients;
refugees; inner city poor, the addicted
and the homeless
sample across categories
Setting
Included but not limited to rural or remote
settings, long term care institutions and
home visits
sample across categories
61
61. Dimensions of Care
Health
Promotion
and Illness
Prevention
Acute Chronic Psychosocial
Aspects
Row
Percent
PhysicianActivities
Assessment/
Diagnosis 25±5
Management 35±5
Communication 20±5
Professional
Behaviors 20±5
Column Percent 20±5 25±5 35±5 20±5 100
Assessment leading up to Decision 2:
Entry into Unsupervised Practice
63
62. Comparison between Two Decision
points – Dimensions of Care
20
30 30
2020
25
35
20
0
5
10
15
20
25
30
35
40
Health Promotion & Illness
Prevention
Acute Chronic Pyschosocial Aspects
Weighting%
Dimensions of Care
Test Specification Weightings between
Decision 1 and 2
D1-Entry into Supervised Practice D2-Entry into Unsupervised Practice
64
63. Comparison between Two Decision
points – Physician Activities
30
20
30
20
25
35
20 20
0
5
10
15
20
25
30
35
40
Assessment/Diagnosis Management Communication Professional Behaviors
Weighting%
Physician Activities
Test Specification Weightings between
Decision 1 and 2
D1-Entry into Supervised Practice D2-Entry into Unsupervised Practice
65
65. Consensus Scale 1
Activity
1. Think about how comfortable you are with
the blueprint & test specifications?
2. Identify where you are on the scale?
1 2 3 4 5
I hate it!
I don’t
like it
I can live with
& support it
I like it I love it!
1. Source: Facilitators Guide to Participatory Decision Making by Sam Kaner
Do we have consensus
to move forward?
67
67. Next Steps for MCC
Blueprint &
Specifications
Stakeholder
consultations
Input & Impact Approval
Current QEs Gap Analysis Changes
New content
Structures
Future View
Research &
Analysis
Assessment
strategies & tools
Partnership
opportunities
Jan. 2014
Future
2016-2017
69
68. Next Steps
• Planning for implementation underway – pending
approved Blueprint and Specifications
• Preliminary impacts identified
– Transition QEI and QEII to the blueprint, with a focus on
• Content
– Complex cases with multi-morbidity; frail elderly cases;
psychosocial cases
• Assessment Tools Adjustments
– Unfolding CDM cases
– New OSCE item formats etc.
– Assessment Evolution will be required to meet the
blueprint envisioned
70
69. TransitionQEI&QEIItoBP-Content
TransitionQEI&QEIItoBP-
AssessmentTools
Timeline
July 2013 May 2017
Oct 2013 Jan 2014 Apr 2014 Jul 2014 Oct 2014 Jan 2015 Apr 2015 Jul 2015 Oct 2015 Jan 2016 Apr 2016 Jul 2016 Oct 2016 Jan 2017 Apr 2017
Classification finalized with CEC
Configure Item
Bank
(report &
structure)
Reclassify,
migrate & clean
QEI content
Analyze
Pool
Pilot
Define pilot
requirements /
strategy
Translate content
Implementation
Identify potential item type
innovation (tweak & improve)
Investigate
Unfolding cases in
CDM
Item formats (i.e.
new OSCE stations) Design
Establish Test Specs for QEI & QEII
(interim targets for current exams)
Test Committees to develop content for known gaps Test Committees to develop content based on pool analysis
TC Committee Structure & Content
Development Review
(i.e. committees, process etc.)
Other
Develop
Reclassify, migrate & clean QEII content
Transition QEI and QEII
to the blueprint
71
75. Proposed Common Blueprint
Dimensions of Care
Health Promotion
and Illness
Prevention
Acute Chronic Psychosocial
Aspects
PhysicianActivities
Assessment/
Diagnosis
Management
Communication
Professional
Behaviors
77
76. Assessment leading up to Decision 1:
Entry into Supervised Practice
Dimensions of Care
Health
Promotion and
Illness
Prevention
Acute Chronic Psychosocial
Aspects
Row
Percent
PhysicianActivities
Assessment/
Diagnosis 30±5
Management 20±5
Communication 30±5
Professional
Behaviors 20±5
Column Percent 20±5 30±5 30±5 20±5 100
78
77. Assessment leading up to Decision 2:
Entry into Unsupervised Practice
Dimensions of Care
Health
Promotion and
Illness
Prevention
Acute Chronic Psychosocial
Aspects
Row
Percent
PhysicianActivities
Assessment/
Diagnosis 25±5
Management 35±5
Communication 20±5
Professional
Behaviors 20±5
Column Percent 20±5 25±5 35±5 20±5 100
79
78. Workshop
Mapping CanMEDs
• Based on consultations-to-date, there has been a
view to map the MCC Assessment Blueprint to
CanMEDs roles to ensure alignment considering a
comprehensive view of physician assessment
• Activity
– Map CanMEDs roles to the Blueprint components
• Dimensions of Care
• Physician Activities
80
79. Proposed Common Blueprint
Dimensions of Care
Health Promotion
and Illness
Prevention
Acute Chronic Psychosocial
Aspects
PhysicianActivities
Assessment/
Diagnosis
Management
Communication
Professional
Behaviors
81
80. Mapping CanMEDs Roles
• For the specific dimension select all the
CanMEDS roles that can be mapped
82
81. Mapping CanMEDs
Physician
Activities
Reflects the scope of practice and behaviors
of a physician practicing in Canada
Assessment/
Diagnosis
Exploration of illness and disease through
gathering, interpreting and synthesizing
relevant information that includes but is not
limited to history taking, physical
examination and investigation.
83
82. Physician
Activities
Reflects the scope of practice and behaviors
of a physician practicing in Canada
Management
Process that includes but is not limited to
generating, planning, organizing care in
collaboration with patients, families,
communities, populations, and health care
professionals (e.g., finding common ground,
agreeing on problems and goals of care,
time and resource management, roles to
arrive at mutual decisions for treatment).
Mapping CanMEDs
84
83. Physician
Activities
Reflects the scope of practice and behaviors
of a physician practicing in Canada
Communication
Interactions with patients, families,
caregivers, other professionals,
communities and populations. Elements
include but are not limited to active
listening, relationship development,
education, verbal, non-verbal and written
communication (e.g. patient centered
interview, disclosure of error, informed
consent).
Mapping CanMEDs
85
84. Physician
Activities
Reflects the scope of practice and behaviors
of a physician practicing in Canada
Professional
Behaviors
Attitudes, knowledge, and skills based on
clinical and/or medical administrative
competence, communication skills, ethics,
societal, and legal duties resulting in the
wise application of behaviors that
demonstrate a commitment to excellence,
respect, integrity, empathy, accountability
and altruism. (e.g., self- awareness,
reflection, life-long learning, scholarly habits
and physician health for sustainable
practice).
Mapping CanMEDs
86
85. Dimensions
of Care
Focus of care for the patient, family,
community and/or population.
Health
Promotion and
Illness
Prevention
The process of enabling people to increase
control over their health and its determinants,
and thereby improve their health. Illness
prevention covers measures not only to
prevent the occurrence of illness such as risk
factor reduction but also to arrest its progress
and reduce its consequences once
established. This includes but is not limited to
screening, periodic health exam, health
maintenance, patient education and
advocacy, and community and population
health.
Mapping CanMEDs
87
86. Dimensions
of Care
Focus of care for the patient, family,
community and/or population.
Acute
Brief episode of illness, within the time span
defined by initial presentation through to
transition of care. This dimension includes
but is not limited to urgent, emergent, and
life-threatening conditions, new conditions,
and exacerbation of underlying conditions.
Mapping CanMEDs
88
87. Dimensions
of Care
Focus of care for the patient, family,
community and/or population.
Chronic
Illness of long duration that includes
but is not limited to illnesses with slow
progression.
Mapping CanMEDs
89
88. Dimensions
of Care
Focus of care for the patient, family,
community and/or population.
Psychosocial
Aspects
Presentations rooted in the social and
psychological determinants of health that
include but are not limited to life challenges,
income, culture, and the impact of the
patient’s social and physical environment.
Mapping CanMEDs
90
92. Workshop #1
Gaps & Issues
• Explore in more detail feedback from
yesterday’s initial consultation to establish
key take-aways in consultation with
Council members
94
93. Proposed Common Blueprint
Dimensions of Care
Health Promotion
and Illness
Prevention
Acute Chronic Psychosocial
Aspects
PhysicianActivities
Assessment/
Diagnosis
Management
Communication
Professional
Behaviors
95
94. Overall Blueprint - Feedback
• What key words are missing?
– Add key words on Sunday pm
96
95. Overall Blueprint - Feedback
• Insert responses from Consensus scale if
not all 3-5’s
97
96. Overall Blueprint
• What are the critical take-aways we need to
consider as part of this consultation from
your perspective?
98
97. Definitions
Dimensions
of Care
Focus of care for the patient, family,
community and/or population.
Health
Promotion and
Illness
Prevention
The process of enabling people to increase control over their health and
its determinants, and thereby improve their health. Illness prevention
covers measures not only to prevent the occurrence of illness such as
risk factor reduction but also to arrest its progress and reduce its
consequences once established. This includes but is not limited to
screening, periodic health exam, health maintenance, patient education
and advocacy, and community and population health.
Acute
Brief episode of illness, within the time span defined by initial
presentation through to transition of care. This dimension includes but is
not limited to urgent, emergent, and life-threatening conditions, new
conditions, and exacerbation of underlying conditions.
Chronic
Illness of long duration that includes but is not limited to illnesses with
slow progression.
Psychosocial
Aspects
Presentations rooted in the social and psychological determinants of
health that include but are not limited to life challenges, income, culture,
and the impact of the patient’s social and physical environment.
99
98. Blueprint – Dimensions of Care
• What are the critical take-aways we need to
consider as part of this consultation?
100
99. Definitions
Physician
Activities
Reflects the scope of practice and behaviors of a
physician practicing in Canada
Assessment/
Diagnosis
Exploration of illness and disease through gathering, interpreting and
synthesizing relevant information that includes but is not limited to history
taking, physical examination and investigation.
Management
Process that includes but is not limited to generating, planning, organizing
care in collaboration with patients, families, communities, populations, and
health care professionals (e.g., finding common ground, agreeing on
problems and goals of care, time and resource management, roles to
arrive at mutual decisions for treatment).
Communication
Interactions with patients, families, caregivers, other professionals,
communities and populations. Elements include but are not limited to
active listening, relationship development, education, verbal, non-verbal
and written communication (e.g. patient centered interview, disclosure of
error, informed consent).
Professional
Behaviors
Attitudes, knowledge, and skills based on clinical and/or medical
administrative competence, communication skills, ethics, societal, and
legal duties resulting in the wise application of behaviors that demonstrate
a commitment to excellence, respect, integrity, empathy, accountability
and altruism. (e.g., self- awareness, reflection, life-long learning, scholarly
habits and physician health for sustainable practice). 101
100. Blueprint – Physician Activities
• What are the critical take-aways we need to
consider as part of this consultation?
102
101. Comparison between Two Decision
points – Dimensions of Care
20
30 30
2020
25
35
20
0
5
10
15
20
25
30
35
40
Health Promotion & Illness
Prevention
Acute Chronic Pyschosocial Aspects
Weighting%
Dimensions of Care
Test Specification Weightings between
Decision 1 and 2
D1-Entry into Supervised Practice D2-Entry into Unsupervised Practice
103
102. Comparison between Two Decision
points – Physician Activities
30
20
30
20
25
35
20 20
0
5
10
15
20
25
30
35
40
Assessment/Diagnosis Management Communication Professional Behaviors
Weighting%
Physician Activities
Test Specification Weightings between
Decision 1 and 2
D1-Entry into Supervised Practice D2-Entry into Unsupervised Practice
104
103. Blueprint – Test Specifications
• Do the differences between the two
specifications make sense?
– Why did it resonate? Or why not?
– If you where unsure has anything to date
changed your mind?
105
104. Blueprint – Test Specifications
• What are the critical take-aways we need to
consider as part of this consultation?
106
106. Workshop #2
Challenges & Opportunities
• To explore the impacts, challenges and
opportunities of the proposed Blueprint
and Test Specifications for an
undifferentiated physician
108
107. Blueprint – Challenges
• Based on your understanding of MCC qualifying
examinations what will be some of the challenges
for MCC to implement this proposed blueprint?
– Is it possible to meet the proposed blueprint with the tools
we have for Decision Point 1 and Decision point 2?
– No challenges? How do you propose that we fulfill 50%
communication/professional behavior blueprint
requirement for decision point 1 with a written exam? Can
we cover the entire BP with our present OSCE for
decision point 2?
109
109. Blueprint – Challenges
• Can you see the BP fit along the
Assessment Continuum? What are the
challenges for using this Blueprint in the
Assessment Continuum?
111
111. Blueprint – Opportunities
• What other opportunities exist to collaborate in
assessment leading to the two decision points?
– Standardize faculty of medicine OSCEs
– Standardize mini-CEX across schools
– Common technical skills assessment tools
– Harmonize with specialty exams of RC
– Standardize ITER/FITER to include as assessment
tool
– Other suggestions…
113
112. Blueprint – Opportunities
• What would you consider to be your
personal “top 3” collaboration opportunities
for what ever reason?
114
Thank you for the opportunity to highlight some of the work we have completed over the past few months,We’ll provide a refresher on the background and context, describe the process usedhighlight the information sources used to define the propose MCC assessment blueprint and test specifications and then talk about next steps in preparation for our workshops tomorrow
From a project perspective our main objective is to ensure that the critical core competencies, knowledge, skills and behaviors required of a physician entering supervised and unsupervised practice are being appropriately assessedand in doing so will ensure that MCC assessment will fulfill all the requirements and standards for credentialing examinations and provide a clear and deliberate process to update the exam specifications while responding to ongoing developments in the profession
To engage Council members in a consultation about the proposed Blueprint and Test SpecificationsFormatThis afternoonGather initial feedback on the blueprint and test specificationsTomorrowProposed Blueprint Linkages with CanMEDSWorkshop #1 – explore feedback from afternoon, gaps and issues – healthy discussion on the proposed blueprintWorkshop #2 – explore opportunities & next steps
As you may be aware, we held a subject matter expert panel meeting to use the information sources to draft a blueprint and test specifications in mid-May. It was a 3 day meeting and involve 12 experts from various stakeholder groups and MCC staff from examinations and research.We’ll take a little bit of time to highlight three of the reports that were used to inform the blueprint and test specifications, but throughout the 3 day workshop we obtained a common understanding of candidates at two decisions points – entry into supervised and unsupervised practices, developed a common blueprint and then test specifications for assessment leading up to those two decision points.
In addition to their expert judgment that the panel brought to the discussionwe’ll be highlight some three of the reports that were instrumental in guiding the development of the proposed blueprint and test specifications
We have highlighted the 4 Reports used during a subject matter expert panel meeting to draft a blueprint and test specifications in mid-May. It was a 3 day meeting and involve 12 experts from various stakeholder groups and MCC staff from examinations and research.but throughout the 3 day workshop we obtained a common understanding of candidates at two decisions points – entry into supervised and unsupervised practices, developed a common blueprint and then test specifications for assessment leading up to those two decision points.
As with any development work, we engaged a wide range of physicians as subject matter experts and many played multiple roles from various stakeholder organizations.Internal and external views represented on the panel.From an internal MCC perspective, we had representatives from University and MRA Council, the Central Examination Committee, Objectives and Test CommitteesExternalRCPSC, CFPCAnd UGME and PGME DeansRepresented a diverse group who effectively came together to collaborate.
As a reminder, our key outputs from our workshop in mid-May were candidates clearly identifieda common blueprint and test specifications for assessments leading up to the two decision points.
We defined an undifferentiated candidate (regardless of specialty) at two decisions points, entering into supervised practice and entering into unsupervised practice. This was more logistical as we broke the group into smaller groups to define test specifications and needed a common understanding of who were thinking about
The common blueprint was developed that we then used to propose test specifications.Dimensions of Care - Focus of care for the patient, family, community and/or population. Physician Activities - Reflects the scope of practice and behaviors of a physician practicing in CanadaI’ll walk through some of the key points in the definitions.
Health Promotion and Illness Preventionprocess of enabling people to increase control over their health and its determinants, and thereby improve their health. Illness prevention covers measures not only to prevent the occurrence of illness such as risk factor reduction but also to arrest its progress and reduce its consequences once established. Concepts include screening, periodic health exam, health maintenance, patient education and advocacy, and community and population health.Acute careBrief episode of illness, within the time span defined by initial presentation through to transition of care. Concepts includes urgent, emergent, and life-threatening conditions, new conditions, and exacerbation of underlying conditions. ChronicIllness of long duration that includes but is not limited to illnesses with slow progression.Psychosocial AspectsPresentations rooted in the social and psychological determinants of health Concepts include life challenges, income, culture, and the impact of the patient’s social and physical environment.
Assessment/ DiagnosisA physicians exploration of illness and disease through gathering, interpreting and synthesizing relevant information Concepts include history taking, physical examination and investigation.ManagementThe processes used to generating, planning, organizing care in collaboration with patients, families, communities, populations, and health care professionalsConcepts include finding common ground, agreeing on problems and goals of care, time and resource management, roles to arrive at mutual decisions for treatmentCommunicationInteractions with patients, families, caregivers, other professionals, communities and populations. Elements include active listening, relationship development, education, verbal, non-verbal and written communication Concepts include patient centered interview, disclosure of error, informed consent).Professional BehaviorsAttitudes, knowledge, and skills based on clinical and/or medical administrative competence, communication skills, ethics, societal, and legal duties resulting in the wise application of behaviors that demonstrate a commitment to excellence, respect, integrity, empathy, accountability and altruism.Concepts include self- awareness, reflection, life-long learning, scholarly habits and physician health for sustainable practice).
Not sure where to put this slide.
Take a 15 m break and then resume with a consultation on the topic
Before go into thisTablets or phonesTwo ways to do this – text or you can go online at xxxxxxx
Dimensions of Care - Focus of care for the patient, family, community and/or population. Physician Activities - Reflects the scope of practice and behaviors of a physician practicing in Canada
Explore how well the different dimensions resonate positively with you
Based on dimensions of care and physician activities, what key words are missing?
Once we had obtained consensus on the propose blueprint structure and general description of the dimensions, we then tasked the group with defining test specifications at
Once we had obtained consensus on the propose blueprint structure and general description of the dimensions, we then tasked the group with defining test specifications for assessment leading up to two decision points
Decision 1 – entry into supervised practiceWalkthrough areasPhysician activitiesDimensions of careFrom a dimensions of care, there was more of a weighting on acute and chronic areas, but still a clear indicator that health promotion and illness prevention and psychosocial aspects were important. In part this represents some of the data from the Incidence and prevalance of disease in Canada
Identified constraints that underpin the blueprint.The constraints are common across the two decision pointsComplexity – read slide
Decision 1 – entry into supervised practiceWalkthrough areasPhysician activitiesDimensions of careFrom a dimensions of care, there was more of a weighting on acute and chronic areas, but still a clear indicator that health promotion and illness prevention and psychosocial aspects were important. In part this represents some of the data from the Incidence and prevalance of disease in Canada
For assessments leading up to decision point 2For dimensions of care, there was a slight shift between acute and chronic care with chronic being weighted as slightly more focus would be required at this stageFor physician activities, there is about a 60% weighting on the medical expert, and 40% on communications and professionalism, not that it isn’t important but the focus for unsupervised practice was on the expert role.
Across the two specifications, we have consistent test specification constraints with them being equal with the exception of complexity (mutiple morbidities) at D1 at least 10% at D2 at least 20%.The other constraints were sampling across age including adult women of childbearing age and the frail elderly, an even balance between men and womenSampling across special populations, immigrant, LGBTrans-gender, rural, disabled etc. and also settings, remote, long term care / home visits.The general theme was to keep the constraints fairly broad to enable test committees to do what they do best and enable some evolution of the examinations based on changing populations.
Placeholder for their response
When we compare the two decisions points for Dimensions of care there was a Minor decrease of emphasis on Acute care with a corresponding increase in Chronic care in assessments leading up to entry into unsupervised practice as that would be more of the situations faced in a practice environment for an undifferentiated physician
When we compare the two decisions points for Physician activities, there was a little bit more movementMinor decrease of emphasis on assessment and diagnosis, less emphasis on communication, and a greater focus on Management in assessments leading up to entry into unsupervised practice, again, as that would be more of the situations faced in a practice environment for an undifferentiated physician
Considering the bluprint, associated test specifications and constraints is there consensus on the proposal moving forward with consultations.
And initial discussions have identified some opportunities along the physician assessment continuum – from undergraduate education to Physician performance enhancement – revalidationWalkthrough slideOpportunity to come back once we know what we are doing for potential business/assessment opportunities and making use of existing MCC structures like TAC to help move these agendas forward.What role would you see your committee playing?
Open ended question for thinking about tomorrow
Thank you for your time, and we will be using some of the material and information gathered this afternoon to enrich our workshop discussions tomorrow.
The common blueprint was developed that we then used to propose test specifications that we highlighted yesterday included….Dimensions of Care - Focus of care for the patient, family, community and/or population. Physician Activities - Reflects the scope of practice and behaviors of a physician practicing in Canada
Decision 1 – entry into supervised practiceFrom a dimensions of care, there was more of a weighting on acute and chronic areas, but still a clear indicator that health promotion and illness prevention and psychosocial aspects were important. In part this represents some of the data from the Incidence and prevalance of disease in CanadaThere was general equal weighting across the two types of physician activities medical expert with assessment/diagnosis & management at 50% communication and professional behaviours at 50 %. This reflected the experts view that both broad categories were equally important entering supervised practice.
For assessments leading up to decision point 2For dimensions of care, there was a slight shift between acute and chronic care with chronic being weighted as slightly more focus would be required at this stageFor physician activities, there is about a 60% weighting on the medical expert, and 40% on communications and professionalism, not that it isn’t important but the focus for unsupervised practice was on the expert role.
On table there is a reference to the RC CanMEDS roles for reference purposesWe will project the definition on xx screen and the question / activity will be on xx screen.
MCC objectives organized thru CanMEDS
Cindy to take notes and type inShout out and then vote – some of the activities we will be doing during our workshops
The common blueprint was developed that we then used to propose test specifications.Dimensions of Care - Focus of care for the patient, family, community and/or population. Physician Activities - Reflects the scope of practice and behaviors of a physician practicing in Canada
Health Promotion and Illness Preventionprocess of enabling people to increase control over their health and its determinants, and thereby improve their health. Illness prevention covers measures not only to prevent the occurrence of illness such as risk factor reduction but also to arrest its progress and reduce its consequences once established. Concepts include screening, periodic health exam, health maintenance, patient education and advocacy, and community and population health.Acute careBrief episode of illness, within the time span defined by initial presentation through to transition of care. Concepts includes urgent, emergent, and life-threatening conditions, new conditions, and exacerbation of underlying conditions. ChronicIllness of long duration that includes but is not limited to illnesses with slow progression.Psychosocial AspectsPresentations rooted in the social and psychological determinants of health Concepts include life challenges, income, culture, and the impact of the patient’s social and physical environment.
Assessment/ DiagnosisA physicians exploration of illness and disease through gathering, interpreting and synthesizing relevant information Concepts include history taking, physical examination and investigation.ManagementThe processes used to generating, planning, organizing care in collaboration with patients, families, communities, populations, and health care professionalsConcepts include finding common ground, agreeing on problems and goals of care, time and resource management, roles to arrive at mutual decisions for treatmentCommunicationInteractions with patients, families, caregivers, other professionals, communities and populations. Elements include active listening, relationship development, education, verbal, non-verbal and written communication Concepts include patient centered interview, disclosure of error, informed consent).Professional BehaviorsAttitudes, knowledge, and skills based on clinical and/or medical administrative competence, communication skills, ethics, societal, and legal duties resulting in the wise application of behaviors that demonstrate a commitment to excellence, respect, integrity, empathy, accountability and altruism.Concepts include self- awareness, reflection, life-long learning, scholarly habits and physician health for sustainable practice).
When we compare the two decisions points for Dimensions of care there was a Minor decrease of emphasis on Acute care with a corresponding increase in Chronic care in assessments leading up to entry into unsupervised practice as that would be more of the situations faced in a practice environment for an undifferentiated physician
When we compare the two decisions points for Physician activities, there was a little bit more movementMinor decrease of emphasis on assessment and diagnosis, less emphasis on communication, and a greater focus on Management in assessments leading up to entry into unsupervised practice, again, as that would be more of the situations faced in a practice environment for an undifferentiated physician
In thinking about the proposed blueprint in context of the assessment continuum.Progression from Undergraduate education, MCC QE Part IPost graduate training, Part IICertification through RC, CFPC or CMQLicensureConsidering Physician Performance Enhancement - revalidation
And initial discussions have identified some opportunities along the physician assessment continuum – from undergraduate education to Physician performance enhancement – revalidationWalkthrough slideOpportunity to come back once we know what we are doing for potential business/assessment opportunities and making use of existing MCC structures like TAC to help move these agendas forward.What role would you see your committee playing?