Join Dr. Emily Belita, PhD, as she describes the launch of the Evidence-Informed Decision-Making (EIDM) Competence Measure. This self-report tool has 27 questions to help public health professionals assess knowledge, skills, attitudes/beliefs, and behaviours related to EIDM. Using this comprehensive assessment will help to highlight individual strengths and areas for development related to EIDM

1. Welcome!
• This webinar will be recorded.
• Your microphone and camera will be turned off for the duration of the
webinar.
• To ensure accessibility, live captions can be enabled from the control
panel.

2. October 6, 2022
Presenters: Emily Belita
Facilitator: Karlene Stoby
Evidence Informed Decision Making (EIDM)
Competence Measure

3. Housekeeping
• Connection issues
• We recommend using a wired Internet
Connection
• If you are experiencing technical issues please
send a private message to Alanna Miller
• Use the Q&A and chat to post questions
and/or comments throughout the webinar
• Post your questions in the Q&A
• Send questions about technical difficulties in a
private chat to Alanna Miller
• Polling

4. After Today
After the webinar, access the recording (in English) at
www.youtube.com/nccmt and slides in English and French at
www.slideshare.net/NCCMT/presentations.

5. Pre-webinar Polling Questions
1.How many people are watching today’s session with you?
A) Just Me
B) 2-3
C) 4-5
D) 6-10
E) >10
2. Have you visited the National Collaborating Centre for Methods and Tools’ website or used its resources before?
A) Yes
B) No
3. If you stated YES on the previous question, how many times have you used the NCCMT’s resources?
A) Once
B) 2-3 times
C) 4-10 times
D) 10+ times

7. NCCMT Products and Services
Registry of Methods and Tools
Online Learning
Opportunities
Workshops
Video Series
Public Health+
Networking and
Outreach

8. Presenter
Emily Belita, RN, PhD
Assistant Professor, School of Nursing,
McMaster University

9. mcmaster.ca |
mcmaster.ca
Evidence-Informed
Decision-Making
Competence Measure
Emily Belita, RN, PhD
Assistant Professor, School of Nursing
McMaster University
NCCMT Spotlight
Webinar
October 2022

10. mcmaster.ca |
Land Acknowledgment
2
October 9, 2022
I would like to begin by acknowledging that we meet
on the traditional territories of the Mississauga and
Haudenosaunee nations, and within the lands
protected by the “Dish with One Spoon” wampum
agreement.

11. mcmaster.ca |
Agenda
3
October 9, 2022
 Background – The Why?
 Development Process
 Online Tool
 Practice and Research Uses

12. mcmaster.ca | 4
June 2022
Background
• General EIDM expectations but ↓ specificity1
• Unclear EIDM roles and expectations2, 3
• Lack of formal, standard methods & tools to guide EIDM2, 3
• Lack of EIDM competence indicators & tools in public health
Issue to be addressed:
Importance of EIDM Low Implementation Barriers & Facilitators

13. mcmaster.ca | 5
June 2022
“If you can’t measure it,
you can’t manage it.”
-Peter Drucker

14. mcmaster.ca |
Overview: Evidence-Informed Decision-Making Competence Measure
6
October 9, 2022
Purpose: To assess EIDM knowledge, skills, attitudes/beliefs, and
behaviors and explore broad needs and strengths for the
individual and group
Structure: 27-item self-report tool (rated on 7-point scales)
• 7 knowledge items
• 5 skills items
• 5 attitudes/beliefs items
• 10 behaviour items

15. mcmaster.ca | 7
June 2022
Stage 1:
Psychometric
Systematic
Review
Stage 2:
Item Development
& Content
Validation
Stage 3:
Validity, Reliability,
& Acceptability
Testing
Tool Development Process

16. mcmaster.ca |
Conceptual Definitions
8
June 2022
• Evidence-informed decision making (EIDM) involves identifying, sharing,
and applying the best available evidence into public health practice4

17. mcmaster.ca | 9
June 2022
Stage 1: Psychometric Systematic Review - Methods
Component Description
Search strategy
• Online databases (n=6)
• Grey literature (e.g., nursing organization websites, conference
proceedings)
• Handsearching relevant journals
• Content experts
Study selection • Independent, duplicate by two reviewers
Data extraction
• Study/measure characteristics: Extracted by one reviewer, checked
by second reviewer for accuracy
• Psychometric outcomes: Independent, duplicate extraction by two
reviewers
• Outcomes: Acceptability, reliability, validity evidence
Data analysis
• Narrative synthesis of psychometric outcomes and measure
characteristics across practice settings

18. mcmaster.ca | 10
June 2022
Stage 1: Psychometric Systematic Review - Findings
Key Findings
• 103 studies and 35 unique measures
Settings:
• Acute care (n=31)
• Limited tools for public health (n=4), primary care
(n=4), long-term care (n=1)
Attributes:
• Tools assessed primarily one EIDM competence attribute
• Majority measured only one attribute (n=19)
• Measures assessing all four EIDM competence attributes (n=3)
Psychometric properties:
• Limited reporting of acceptability and validity evidence
• Primarily reliability reported (internal consistency)

19. mcmaster.ca | 11
October 9, 2022
Belita et al. (2021). Measures of evidence-informed decision-making competence
attribute: A psychometric systematic review. BMC Nursing, 19:44.

20. mcmaster.ca | 12
June 2022
Stage 2: Item Development and Content Validation
• 63 self-report items generated
• Subscales of:
1. Knowledge
2. Skills
3. Attitudes/Beliefs
4. Behaviours

21. mcmaster.ca | 13
June 2022
Stage 2: Item Development and Content Validation
Validity Based on
Content
Validity Based on
Response Process
• Item relevance rated by
11 EIDM/public health
experts on 4-point scale
• Item content validity
indices ranged from 0.64-
1.00
• 28 item deletions, 23
modified, 5 newly
added, 12 in original
form
• Cognitive interviews with
9 public health nurses
• Comprehension,
ease/difficulty of
completion assessed
• 8 item modifications to
improve clarity
Belita et al. (2021). Development and content validation of a
measure to assess evidence-informed decision-making competence
in public health nursing. PLoS ONE, 16, e0248330.

22. mcmaster.ca | 14
June 2022
Stage 3: Validity, Reliability & Acceptability Testing
Methods
Design • Cross-sectional
Sample • Registered nurses in Ontario health units
• Any role or department
Data collection • One-time self-report online survey
• 40-item EIDM Competence Measure
Data analysis
Validity evidence
• Exploratory factor analysis
• Correlations, t-test, ANOVAs
Reliability
• Internal consistency: Cronbach’s alpha
Acceptability
• Completion time
• Missing data
Belita et al. (2022). Validity, reliability, and acceptability of the Evidence-Informed Decision-Making (EIDM) competence
measure. PLoS ONE, 17, e0272699.

23. mcmaster.ca | 15
June 2022
Stage 3: Validity, Reliability & Acceptability Testing
• Item-subscale correlations – reduced from 4027 items
• Four factor model (loadings >0.4); 27 items
• Significant relationship between competence attributes
and education, EIDM training, project involvement,
organizational culture
Sample
Validity
Reliability
Acceptability
• 16 Ontario health units
• 201 (35.8%) RNs completed survey
• Cronbach’s alphas for all subscales ranged from 0.80-0.96
• 7% of participants had missing data (at least one item)
• Completion time was 7 minutes and 20 seconds

24. mcmaster.ca |
27-item EIDM Competence Measure
16
June 2022
Subscale1 Number of Items Rating Scale
Knowledge 7 1=Poor to 7=Excellent
Skills 5 1=Beginner to 7=Expert
Attitudes/Beliefs 5 1=Strongly disagree to
7=Strongly agree
Behaviours 10 1=Not competent to
7=Competent

25. mcmaster.ca |
Online EIDM Competence Measure
17
October 9, 2022

26. mcmaster.ca |

27. mcmaster.ca |
EIDM Knowledge Items
1. Knowledge of what is involved in the 'define' step of EIDM.
2. Knowledge of what is involved in the 'search' step of EIDM.
3. Knowledge of what is involved in the 'appraise' step of EIDM.
4. Knowledge of what is involved in the 'synthesize' step of EIDM.
5. Knowledge of what is involved in the 'adapt' step of EIDM.
6. Knowledge of what is involved in the 'implement' step of EIDM.
7. Knowledge of what is involved in the 'evaluate' step of EIDM.
19
October 9, 2022

28. mcmaster.ca |
EIDM Skills Items
1. Ability to develop an answerable practice question.
2. Ability to use online databases that house research
evidence.
3. Ability to use critical appraisal tools to appraise the quality
of research evidence (e.g., AGREE II tool, CASP)
4. Ability to assess applicability of research evidence to the
local public health context.
5. Ability to participate in the development of evaluation
indicators to assess outcomes of evidence-informed decision
or practice changes.
20
October 9, 2022

29. mcmaster.ca |
EIDM Attitudes/Beliefs
1. I believe that I can implement EIDM in a time efficient way.
2. I believe I can engage others in implementing strategies to
address barriers (e.g., personal, organizational, community)
when implementing EIDM
3. I believe that evaluating outcomes of an evidence-informed
decision or practice change is an important component of
EIDM.
4. I believe that implementing EIDM can improve public
health services, programs, and policies.
5. I believe that critically appraising evidence is an important
step in the EIDM process.
21
October 9, 2022

30. mcmaster.ca |
EIDM Behaviours Items
22
October 9, 2022

31. mcmaster.ca |
EIDM Behaviours Items
23
October 9, 2022
1. I question public health practices for the purpose of improving the quality of care/service delivery.
2. I describe public health practice issues using client assessment data (i.e., community, individuals, families, populations).
3. I participate in the formulation of public health practice questions.
4. I search for research evidence to answer public health practice questions.
5. I participate in the synthesis and interpretation of a body of research evidence gathered to formulate recommendations for public health
practice.
6. I integrate evidence gathered from public health expertise, client or community preferences, and local context with research evidence to
plan evidence-informed practice changes.
7. I participate in the assessment of barriers and facilitators (related to resources, organization, evidence/guidelines, clients'
preferences/values) when implementing a practice change.
8. I participate in the process of stakeholder analyses (i.e., collecting and analyzing information on stakeholders' importance and
influence) when implementing a practice change.
9. I participate in the development of an action plan to implement a practice change.
10. I participate in evaluating outcomes of evidence-informed decisions or practice changes.

32. mcmaster.ca | 24
October 9, 2022

33. mcmaster.ca | 25
October 9, 2022

34. mcmaster.ca | 26
October 9, 2022

35. mcmaster.ca | 27
October 9, 2022

36. mcmaster.ca |
Practice and Research Use
• Guide professional development in practice
• Individual assessment
• Group assessment (teams/departments)
• Annual reviews
• Integrate as indicators in performance appraisals
• Intervention assessment
• Effectiveness of EIDM capacity building interventions on
knowledge, skills, attitudes/beliefs, behaviours
28
October 9, 2022

37. mcmaster.ca | 29
June 2022
Future areas of research in EIDM competence assessment
Multi-method approach to EIDM competence
assessment
Testing across national public health workforce
Psychometric performance in intervention
research and education

38. mcmaster.ca |
Summary
30
June 2022
• New EIDM Competence Measure has demonstrated:
• Validity (content, response process, internal structure and relationships to other variables)
• Reliability
• Acceptability
• Can be used in public health nursing practice for:
• Organizational planning
• Personal self-reflection and professional development
• Potential for use in nursing education and intervention research

39. mcmaster.ca |
Team Acknowledgments
• Dr. Maureen Dobbins, NCCMT, McMaster University
• Dr. Kathryn Fisher, McMaster University
• Dr. Rebecca Ganann, McMaster University
• Dr. Janet E. Squires, University of Ottawa
• Dr. Jennifer Yost, Villanova University
31
October 9, 2022

40. mcmaster.ca |
Thank you!
Questions:
Emily Belita
belitae@mcmaster.ca
32
June 2022

41. mcmaster.ca |
References
33
June 2022
1Peirson, L., Ciliska, D., Dobbins, M., & Mowat, D. (2012). Building capacity
for evidence informed decision making in public health: A case study of
organizational change. BMC Public Health, 12, 137. doi:10.1186/1471-2458-
12-137
2Melnyk, B. M., Gallagher-Ford, L., Long, L. E., & Fineout-Overholt, E. (2014).
The establishment of evidence-based practice competencies for practicing
registered nurses and advanced practice nurses in real-world clinical settings:
Proficiencies to improve healthcare quality, reliability, patient outcomes, and
costs.Worldviews on Evidence-Based Nursing, 11(1), 5-15.
3Brownson, R. C., Fielding, J. E., & Green, L. W. (2018). Building Capacity for
Evidence-Based Public Health: Reconciling the Pulls of Practice and the Push
of Research. Annual Review of Public Health, 39, 27-53.
4National Collaborating Centre for Methods and Tools. (2020a). Evidence-
informed public health Retrieved from http://www.nccmt.ca/professional-
development/eiph

42. Share your story!
• Are you using EIDM in your practice? We want to hear about it!
• Email us: nccmt@mcmaster.ca
• Need support for EIDM? Contact us for help!
• Email us: nccmt@mcmaster.ca
• We typically respond within 24 business hours
9

43. Webinar Feedback
Your responses will be kept anonymous.
Please indicate your level of agreement with the following:
1. Participating in the webinar increased my knowledge and understanding of evidence-informed decision
making.
2. I will use the information from today’s webinar in my own practice.
3. Which of the following statements apply to your experience with the webinar today (check all that
apply):
□ The webinar was relevant to me and my public health practice
□ The webinar was effectively facilitated
□ The webinar had opportunities to participate
□ The webinar was easy to follow along
□ The webinar met my expectations
Strongly agree Agree Undecided Disagree Strongly Disagree
Strongly agree Agree Undecided Disagree Strongly Disagree

44. Webinar Feedback
Your responses will be kept anonymous.
4. Can we contact you in the future to discuss how the NCCMT can
improve its webinar series?
□ Yes
□ No

45. Webinars from the NCCMT
Learn more about our webinars:
http://www.nccmt.ca/capacity-development/webinars
12

46. For more information:
NCCMT website: www.nccmt.ca
Contact: nccmt@mcmaster.ca