Welcome!
• This webinar will be recorded.
• Your microphone and camera will be turned off for the duration of the
webinar.
• To ensure accessibility, live captions can be enabled from the control
panel.

October 6, 2022
Presenters: Emily Belita
Facilitator: Karlene Stoby
Evidence Informed Decision Making (EIDM)
Competence Measure

Housekeeping
• Connection issues
• We recommend using a wired Internet
Connection
• If you are experiencing technical issues please
send a private message to Alanna Miller
• Use the Q&A and chat to post questions
and/or comments throughout the webinar
• Post your questions in the Q&A
• Send questions about technical difficulties in a
private chat to Alanna Miller
• Polling

After Today
After the webinar, access the recording (in English) at
www.youtube.com/nccmt and slides in English and French at
www.slideshare.net/NCCMT/presentations.

Pre-webinar Polling Questions
1.How many people are watching today’s session with you?
A) Just Me
B) 2-3
C) 4-5
D) 6-10
E) >10
2. Have you visited the National Collaborating Centre for Methods and Tools’ website or used its resources before?
A) Yes
B) No
3. If you stated YES on the previous question, how many times have you used the NCCMT’s resources?
A) Once
B) 2-3 times
C) 4-10 times
D) 10+ times

NCCMT Products and Services
Registry of Methods and Tools
Online Learning
Opportunities
Workshops
Video Series
Public Health+
Networking and
Outreach

Presenter
Emily Belita, RN, PhD
Assistant Professor, School of Nursing,
McMaster University

mcmaster.ca |
mcmaster.ca
Evidence-Informed
Decision-Making
Competence Measure
Emily Belita, RN, PhD
Assistant Professor, School of Nursing
McMaster University
NCCMT Spotlight
Webinar
October 2022

mcmaster.ca |
Land Acknowledgment
2
October 9, 2022
I would like to begin by acknowledging that we meet
on the traditional territories of the Mississauga and
Haudenosaunee nations, and within the lands
protected by the “Dish with One Spoon” wampum
agreement.

mcmaster.ca |
Agenda
3
October 9, 2022
 Background – The Why?
 Development Process
 Online Tool
 Practice and Research Uses

mcmaster.ca | 4
June 2022
Background
• General EIDM expectations but ↓ specificity1
• Unclear EIDM roles and expectations2, 3
• Lack of formal, standard methods & tools to guide EIDM2, 3
• Lack of EIDM competence indicators & tools in public health
Issue to be addressed:
Importance of EIDM Low Implementation Barriers & Facilitators

mcmaster.ca | 5
June 2022
“If you can’t measure it,
you can’t manage it.”
-Peter Drucker

mcmaster.ca |
Overview: Evidence-Informed Decision-Making Competence Measure
6
October 9, 2022
Purpose: To assess EIDM knowledge, skills, attitudes/beliefs, and
behaviors and explore broad needs and strengths for the
individual and group
Structure: 27-item self-report tool (rated on 7-point scales)
• 7 knowledge items
• 5 skills items
• 5 attitudes/beliefs items
• 10 behaviour items

mcmaster.ca | 7
June 2022
Stage 1:
Psychometric
Systematic
Review
Stage 2:
Item Development
& Content
Validation
Stage 3:
Validity, Reliability,
& Acceptability
Testing
Tool Development Process

mcmaster.ca |
Conceptual Definitions
8
June 2022
• Evidence-informed decision making (EIDM) involves identifying, sharing,
and applying the best available evidence into public health practice4

mcmaster.ca | 9
June 2022
Stage 1: Psychometric Systematic Review - Methods
Component Description
Search strategy
• Online databases (n=6)
• Grey literature (e.g., nursing organization websites, conference
proceedings)
• Handsearching relevant journals
• Content experts
Study selection • Independent, duplicate by two reviewers
Data extraction
• Study/measure characteristics: Extracted by one reviewer, checked
by second reviewer for accuracy
• Psychometric outcomes: Independent, duplicate extraction by two
reviewers
• Outcomes: Acceptability, reliability, validity evidence
Data analysis
• Narrative synthesis of psychometric outcomes and measure
characteristics across practice settings

mcmaster.ca | 10
June 2022
Stage 1: Psychometric Systematic Review - Findings
Key Findings
• 103 studies and 35 unique measures
Settings:
• Acute care (n=31)
• Limited tools for public health (n=4), primary care
(n=4), long-term care (n=1)
Attributes:
• Tools assessed primarily one EIDM competence attribute
• Majority measured only one attribute (n=19)
• Measures assessing all four EIDM competence attributes (n=3)
Psychometric properties:
• Limited reporting of acceptability and validity evidence
• Primarily reliability reported (internal consistency)

mcmaster.ca | 11
October 9, 2022
Belita et al. (2021). Measures of evidence-informed decision-making competence
attribute: A psychometric systematic review. BMC Nursing, 19:44.

mcmaster.ca | 12
June 2022
Stage 2: Item Development and Content Validation
• 63 self-report items generated
• Subscales of:
1. Knowledge
2. Skills
3. Attitudes/Beliefs
4. Behaviours

mcmaster.ca | 13
June 2022
Stage 2: Item Development and Content Validation
Validity Based on
Content
Validity Based on
Response Process
• Item relevance rated by
11 EIDM/public health
experts on 4-point scale
• Item content validity
indices ranged from 0.64-
1.00
• 28 item deletions, 23
modified, 5 newly
added, 12 in original
form
• Cognitive interviews with
9 public health nurses
• Comprehension,
ease/difficulty of
completion assessed
• 8 item modifications to
improve clarity
Belita et al. (2021). Development and content validation of a
measure to assess evidence-informed decision-making competence
in public health nursing. PLoS ONE, 16, e0248330.

mcmaster.ca | 14
June 2022
Stage 3: Validity, Reliability & Acceptability Testing
Methods
Design • Cross-sectional
Sample • Registered nurses in Ontario health units
• Any role or department
Data collection • One-time self-report online survey
• 40-item EIDM Competence Measure
Data analysis
Validity evidence
• Exploratory factor analysis
• Correlations, t-test, ANOVAs
Reliability
• Internal consistency: Cronbach’s alpha
Acceptability
• Completion time
• Missing data
Belita et al. (2022). Validity, reliability, and acceptability of the Evidence-Informed Decision-Making (EIDM) competence
measure. PLoS ONE, 17, e0272699.

mcmaster.ca | 15
June 2022
Stage 3: Validity, Reliability & Acceptability Testing
• Item-subscale correlations – reduced from 4027 items
• Four factor model (loadings >0.4); 27 items
• Significant relationship between competence attributes
and education, EIDM training, project involvement,
organizational culture
Sample
Validity
Reliability
Acceptability
• 16 Ontario health units
• 201 (35.8%) RNs completed survey
• Cronbach’s alphas for all subscales ranged from 0.80-0.96
• 7% of participants had missing data (at least one item)
• Completion time was 7 minutes and 20 seconds

mcmaster.ca |
27-item EIDM Competence Measure
16
June 2022
Subscale1 Number of Items Rating Scale
Knowledge 7 1=Poor to 7=Excellent
Skills 5 1=Beginner to 7=Expert
Attitudes/Beliefs 5 1=Strongly disagree to
7=Strongly agree
Behaviours 10 1=Not competent to
7=Competent

mcmaster.ca |
Online EIDM Competence Measure
17
October 9, 2022

mcmaster.ca |

mcmaster.ca |
EIDM Knowledge Items
1. Knowledge of what is involved in the 'define' step of EIDM.
2. Knowledge of what is involved in the 'search' step of EIDM.
3. Knowledge of what is involved in the 'appraise' step of EIDM.
4. Knowledge of what is involved in the 'synthesize' step of EIDM.
5. Knowledge of what is involved in the 'adapt' step of EIDM.
6. Knowledge of what is involved in the 'implement' step of EIDM.
7. Knowledge of what is involved in the 'evaluate' step of EIDM.
19
October 9, 2022

mcmaster.ca |
EIDM Skills Items
1. Ability to develop an answerable practice question.
2. Ability to use online databases that house research
evidence.
3. Ability to use critical appraisal tools to appraise the quality
of research evidence (e.g., AGREE II tool, CASP)
4. Ability to assess applicability of research evidence to the
local public health context.
5. Ability to participate in the development of evaluation
indicators to assess outcomes of evidence-informed decision
or practice changes.
20
October 9, 2022

mcmaster.ca |
EIDM Attitudes/Beliefs
1. I believe that I can implement EIDM in a time efficient way.
2. I believe I can engage others in implementing strategies to
address barriers (e.g., personal, organizational, community)
when implementing EIDM
3. I believe that evaluating outcomes of an evidence-informed
decision or practice change is an important component of
EIDM.
4. I believe that implementing EIDM can improve public
health services, programs, and policies.
5. I believe that critically appraising evidence is an important
step in the EIDM process.
21
October 9, 2022

mcmaster.ca |
EIDM Behaviours Items
22
October 9, 2022

mcmaster.ca |
EIDM Behaviours Items
23
October 9, 2022
1. I question public health practices for the purpose of improving the quality of care/service delivery.
2. I describe public health practice issues using client assessment data (i.e., community, individuals, families, populations).
3. I participate in the formulation of public health practice questions.
4. I search for research evidence to answer public health practice questions.
5. I participate in the synthesis and interpretation of a body of research evidence gathered to formulate recommendations for public health
practice.
6. I integrate evidence gathered from public health expertise, client or community preferences, and local context with research evidence to
plan evidence-informed practice changes.
7. I participate in the assessment of barriers and facilitators (related to resources, organization, evidence/guidelines, clients'
preferences/values) when implementing a practice change.
8. I participate in the process of stakeholder analyses (i.e., collecting and analyzing information on stakeholders' importance and
influence) when implementing a practice change.
9. I participate in the development of an action plan to implement a practice change.
10. I participate in evaluating outcomes of evidence-informed decisions or practice changes.

mcmaster.ca | 24
October 9, 2022

mcmaster.ca | 25
October 9, 2022

mcmaster.ca | 26
October 9, 2022

mcmaster.ca | 27
October 9, 2022

mcmaster.ca |
Practice and Research Use
• Guide professional development in practice
• Individual assessment
• Group assessment (teams/departments)
• Annual reviews
• Integrate as indicators in performance appraisals
• Intervention assessment
• Effectiveness of EIDM capacity building interventions on
knowledge, skills, attitudes/beliefs, behaviours
28
October 9, 2022

mcmaster.ca | 29
June 2022
Future areas of research in EIDM competence assessment
Multi-method approach to EIDM competence
assessment
Testing across national public health workforce
Psychometric performance in intervention
research and education

mcmaster.ca |
Summary
30
June 2022
• New EIDM Competence Measure has demonstrated:
• Validity (content, response process, internal structure and relationships to other variables)
• Reliability
• Acceptability
• Can be used in public health nursing practice for:
• Organizational planning
• Personal self-reflection and professional development
• Potential for use in nursing education and intervention research

mcmaster.ca |
Team Acknowledgments
• Dr. Maureen Dobbins, NCCMT, McMaster University
• Dr. Kathryn Fisher, McMaster University
• Dr. Rebecca Ganann, McMaster University
• Dr. Janet E. Squires, University of Ottawa
• Dr. Jennifer Yost, Villanova University
31
October 9, 2022

mcmaster.ca |
Thank you!
Questions:
Emily Belita
belitae@mcmaster.ca
32
June 2022

mcmaster.ca |
References
33
June 2022
1Peirson, L., Ciliska, D., Dobbins, M., & Mowat, D. (2012). Building capacity
for evidence informed decision making in public health: A case study of
organizational change. BMC Public Health, 12, 137. doi:10.1186/1471-2458-
12-137
2Melnyk, B. M., Gallagher-Ford, L., Long, L. E., & Fineout-Overholt, E. (2014).
The establishment of evidence-based practice competencies for practicing
registered nurses and advanced practice nurses in real-world clinical settings:
Proficiencies to improve healthcare quality, reliability, patient outcomes, and
costs.Worldviews on Evidence-Based Nursing, 11(1), 5-15.
3Brownson, R. C., Fielding, J. E., & Green, L. W. (2018). Building Capacity for
Evidence-Based Public Health: Reconciling the Pulls of Practice and the Push
of Research. Annual Review of Public Health, 39, 27-53.
4National Collaborating Centre for Methods and Tools. (2020a). Evidence-
informed public health Retrieved from http://www.nccmt.ca/professional-
development/eiph

Share your story!
• Are you using EIDM in your practice? We want to hear about it!
• Email us: nccmt@mcmaster.ca
• Need support for EIDM? Contact us for help!
• Email us: nccmt@mcmaster.ca
• We typically respond within 24 business hours
9

Webinar Feedback
Your responses will be kept anonymous.
Please indicate your level of agreement with the following:
1. Participating in the webinar increased my knowledge and understanding of evidence-informed decision
making.
2. I will use the information from today’s webinar in my own practice.
3. Which of the following statements apply to your experience with the webinar today (check all that
apply):
□ The webinar was relevant to me and my public health practice
□ The webinar was effectively facilitated
□ The webinar had opportunities to participate
□ The webinar was easy to follow along
□ The webinar met my expectations
Strongly agree Agree Undecided Disagree Strongly Disagree
Strongly agree Agree Undecided Disagree Strongly Disagree

Webinar Feedback
Your responses will be kept anonymous.
4. Can we contact you in the future to discuss how the NCCMT can
improve its webinar series?
□ Yes
□ No

Webinars from the NCCMT
Learn more about our webinars:
http://www.nccmt.ca/capacity-development/webinars
12

For more information:
NCCMT website: www.nccmt.ca
Contact: nccmt@mcmaster.ca