Survey Analyses for Implementing an Electronic Information System to Enhance ...
Research method ch14 selected research papers
1. Research Methods in HealthChapter 14. Selected Research Papers-Purpose, Framework, Results -
Young Moon Chae, Ph.D.
Graduate School of Public Health
Yonsei University, Korea
ymchae@yuhs.ac
2. Table of Contents
•Online psychological service for health professional ---------------------------3
•Evaluation of CDSS for Drug Prescription Based on Success Measures --12
•Implementing Health Management Information Systems: Measuring Success in Korea’s Health Centers -------------------------------------------------20
•Risk Factors for Cervical Cancer amongKorean Women from the Korean Cancer Prevention Study --------------------------------------------------------------30
•Evaluation of Mobile Phone-based Diet Game for Weight Control ----------51
•Management Issues in Health Information Systems ----------------------------58
•Competency-based Learning for Distance Education in System Analysis and Design --------------------------------------------------------------------------------62
2
3. Burnout, job stress and job satisfaction: A study among doctors and nurses of Ulaanbaatar, Mongolia
3
Ariunsanaa B, SugarmaaM, KhuderchuluunN, SaranchuluunO, Chae YM
4. Introduction
•Background
–Mongolia is considered to be one of the countries with a sufficient supply of health professionals in terms of proportion to the size of the population, especially in urban areas.
–However, the problem of dense population in urban areas and internal migration has deteriorated the quality, accessibility, and load of healthcare services.
–In this situation, it is claimed that the stress and depression driven by over workload of health professionals has somewhat negatively influenced their attitude towards work and job satisfaction.
–Other factors, such as the number of the patients, duration of medical checkups, procedures, consultations, and other additional functions, like serving patients with special needs, can lead to significant job stress.
•Significance
–There was no study dealing with job stress and job satisfaction for health professionals in Mongolia
4
5. Research Objectives
•General
–To study job stress, burnout and some depressive symptoms of medical doctors and nurses of tertiary health care settings of Ulaanbaatar
•Specific Objectives
–To determine the prevalence of job stress and burnout among medical doctors and nurses,
–To determine the prevalence of job dissatisfaction and some depressive symptoms of medical doctors and nurses
–To analyze the association between job stress, burnout and depressive symptoms
5
6. Methods
•Study design
–Study design is cross sectional and pre-experimental as it is social science research.
•Sampling
–In this study 400 of medical doctors and nurses will be included.
–Stratified sampling by using hospital size as strata
6
7. Research Framework
Burn out
Job stressJob satisfaction
Age
Sex
EducationJob characteristics
Multiple regression
Chi square or t-testCorrelation orChi square test
7
8. Analytic Methods
•All variables are ordinal scale (5 points)
•Simple regression vscorrelation (direction)
•Correlation vsChi-square test
•T-test vsChi-square test
•T-test vspaired t-test
•Parametric test vsnon-parametric test
•Multiple regression vslogistic regression
8
12. Evaluation of CDSS for Drug Prescription Based on Success MeasuresJinwooPark, Young Moon Chae, Young TaekLee, Koungwon Cho, JungheeKim, Byung HwaLee(Published in Journal of Korean Society of Medical Informatics. 2009; 15(3): 293-301)
12
13. Introduction
•Background
–Medication errors are increasing
–The Clinical Decision Support System (CDSS) for drug prescriptions was developed by integrating the computerized physician order entry (CPOE) system to support doctors and pharmacists in making correct decisions on prescribing drugs in line with the prescription guidelines by the Health Insurance Review Agency (HIRA)
•Significance
–This is the first attempt to evaluate effectiveness of CDSS for drug prescription in Korea
–Use of DeLoneand McFarlanmodel for performance evaluation of CDSS
13
14. Objectives
•The purpose of this paper is to analyze the performance of the CDSS with respect to system quality, information quality, and user satisfaction to reduce prescription error
•Specific objectives are:
–To identify the factors influencing CDSS performance
–To analyze association between quality measures and user satisfaction
14
15. Subjects
•A questionnaire was designed by structuring primary evaluation areas first and then more detailed sub-areas in each primary area.
•The survey was conducted by mail or by visit for 6 weeks from April 20 to May 29, 2009, for the pharmacists from 38 hospitals using the CDSS.
•Hospitals were selected by stratified random sampling
•Research design is cross-sectional design
•A total of 84 questionnaires were returned from 22 users (response rate of 58%). Of them, 77 were used for the analysis, excluding 7 that were found unusable for statistical analysis.
15
16. Model and Statistical Analysis
•Based on the Deloneand McLean framework, we evaluated the success of the CDSS by using two quality measures (system quality and information quality) as independent variables; and three performance measures (user satisfaction and organizational impact) as dependent variables for the multiple regression analysis (Figure 1).
•Chi-square analysis with cross-tabulation was performed to analyze the specific associations among two quality measures and three measures of user satisfaction (information satisfaction, system satisfaction, and willingness to recommend to others).
•We also analyzed the associations between two quality measures and improvements in drug safety as a performance measure of organizational impact by using Chi-square
16
18. The effect of the information quality measures on user Satisfaction
Factors
Regressioncoefficient
Standard error
t-value
P-value
Informationaccuracy
0.060
0.252
0.24
0.8116
Informationtimelines
0.502
0.285
1.76
0.0823
Informationreliability
0.462
0.275
1.68
0.0975
Informationup-to-datedness
0.252
0.265
0.95
0.3438
Decisionsupportingfunction
0.771
0.263
2.94
0.0045
18
19. The association of quality measures and information satisfaction
Description
Information satisfaction
No
In-between
Yes
Total
Chisq
p- value
System
Quality
Easeofsystemuse
No
1(25.0)
3(75.0)
0( 0.0)
4( 5.2)
18.0
0.002
In-between
0( 0.0)
19(59.4)
13(40.6)
32(41.6)
Yes
1( 2.4)
12(29.3)
28(68.3)
41(53.2)
Resultsunderstanding
No
1(100.0)
0( 0.0)
0( 0.0)
1( 1.3)
58.7
<.001
In-between
1( 3.2)
23(74.2)
7(22.6)
31(40.3)
Yes
0( 0.0)
11(24.4)
34(75.6)
45(58.4)
Terminologyunderstanding
No
1(33.3)
0( 0.0)
2(66.7)
3( 3.9)
32.7
<.001
In-between
1( 2.6)
27(69.2)
11(28.2)
39(50.6)
Yes
0( 0.0)
7(20.0)
28(80.0)
35(45.5)
19
20. Implementing Health Management Information Systems: Measuring Success in Korea’s Health CentersYoung Moon Chae, Suck Il Kim, Byung HwaLee, Sung HaeChoi, In SookKim(Published in International Journal of Health Planning and Management, 1994; 9: 341-8)
20
21. Introduction
•Background
–Quality of health statistics produced at the district level is still very low so that health status of the district cannot be accurately determined.
–Management data needed for planning, organizing, and evaluating health programs are not readily available for district managers.
–Productivity and morale of most health workers are low because they suffer from heavy workloads of tedious paper works and simple tabulation.
–Health centers and health subcentersare not responding to the changing needs of primary health services, which has been changed from family planning or MCH to chronic degenerative diseases (e.g. hypertension, diabetes)
•Significances
–HMIS can be regarded as management innovation within health center to meet the unrelenting challenge of providing and assuring health services to the community.
–Relatively little research, however, has been done on measuring the processes of introducing a management innovation into a health center.
21
22. Purposes
•To assess the extent to which a management innovation, in this case a HMIS, is perceived as successful to health center personnel
•Specifically,
–To examine the three key measures of successful implementation of HMIS: productivity of health center staffs, adoption process, and the satisfaction of visitors to the health center.
–To compare user satisfaction between computerized center and not-computerized center
22
23. Methods
•Research design –longitudinal design with control
–Three surveys were conducted to measure the changes in productivity and adoption process (knowledge, persuasion, decision, implementation, and confirmation) of health workers over time during the period of 20 months.
–The effects of the Health Management Information System (HMIS) on the quality of services to the visitors were also measured 7 months after the 3rd survey by comparing the quality of services between the study health center and the similar health center as a control group
23
26. Statistical Analysis
•The changes in three adoption variables (knowledge, persuasion, and decision) were compared between the survey 1 and survey 2 (before and after the implementation of the HMIS) using Wilcoxsonsigned-rank test.
•The changes in job processing time, which is a surrogate measure for the productivity of health center staffs, between the three surveys were compared using the Bonferronimethod because there were significant differences in variances among the three groups.
•Three satisfaction variables (waiting time, credibility, and convenience) of visitors between the two health centers were compared with Chi-square test, and average score of these variables were compared with t-test(1 point for the dissatisfied and 3 point for the satisfied).
26
30. Association between Smoking, Screening, and Death Caused by Cervical Cancer inKorean Women
Odongua N, Chae YM, Kim MR, Yun JE, JeeSH
(Published in Yonsei Medical Journal, 2007; 48(2): 192-200)
31. Introduction
Cervical Cancer: 2004
1. Worldwide
– 510.000 new cases
– 288.000 deaths
2. United States
– 10,520 new cases
– 3,900 deaths
3. Korea
– … new cases
– … deaths
Developed
countries
17.0%
Developing
countries
83.0%
Risk Factors:
•Failure to obtain regular Pap smear
test
•HPV infection
•Cigarette smoking
•Sex at an early age
•Multiple sexual partners
•Late menarche
•Early age at first delivery
•High parity
33. Proportion of Cancer Cases among
Korean Women (1999-2001)
stomach
16.4%
breast
13.7%
colorectal
10.4%
cervix
9.8%
lung
8.0%
liver
7.3%
thyroid
6.7%
gallbladder
3.5%
ovary
3.0%
pancreas
2.5%
others
18.7%
Source: National Cancer Center, Korea
34. Objectives
The purpose of this study is to explore and investigate the risk factors for cervical cancer among Korean women from KCPS. Specifically:
1.To estimate the age-adjusted standardized incidence and mortality rates of cervical cancer among Korean women aged 30 years and older.
2.To investigate the relationship between cervical cancer and reproductive factors.
3.To investigate the relationship between cervical cancer and smoking and other behavioral factors.
39. Number of cervical cancer cases by age group (n=2,520)
Incidence40526314598340318283121022633023521410200400600800100030-3940-4950-5960-69≥70Age group Number of cases Women with abnormal resultWomen with normal resultUnscreened women
40. Age-adjusted standard incidence rates by age group
Incidence rate13.612.610.67.91.313.612.29.07.41.10.09.07.64.50.80.031.118.113.62.501020304030-3940-4950-5960-69≥70Age group Rate per 100,000 All women (overall rate 45.9) Unscreened women (43.2) Women with normal result (21.9) Women with abnormal result (65.2)
41. Number of cervical cancer deaths by age group (n=209)
Mortality1219161819012241290112227802040608030-3940-4950-5960-69≥70Age group Number of deaths Women with abnormal resultWomen with normal resultUnscreened women
42. Age-adjusted standard mortality rates by age group
Mortality rate0.40.70.91.00.50.40.91.01.40.60.00.30.60.40.40.01.51.21.60.500.511.5230-3940-4950-5960-69≥70Age group Rate per 100,000 All women (overall rate 3.4) Unscreened women (4.2) Women with normal result (1.8) Women with abnormal result (4.7)
48. Incidence and mortality rate of cervical cancer among all women
Cervical cancer:
–Incidence rate: 45.9 per 100,000.
•The highest incidence rate: women aged 30-39 years old
–Mortality rate: 3.4 per 100,000
•The highest mortality rate: women aged 50 or older
Pap smear test:
–Women who had never been screened by Pap smears:
•1.7-fold high risk of incidence for cervical cancer
•2-fold high risk of mortality for cervical cancer
49. Discussion
•The Healthy People 2010target for cervical cancer is a reduction in mortality to 2.0 deaths per 100,000 women.
•Since 1998, the rate remains near 3.0 deaths per 100,000 women. The U.S. Preventive Services Task Force strongly recommends screening for cervical cancer in women who have been sexually active and have a cervix.
•Pap smear screening should be targeted for younger women. As a result, we can also reduce the overall burden of cervical cancer among Korean women.
•Future studies are needed.
50. Discussion
•This large prospective cohort study of Korean women showed that several reproductive factors including age at menarche, age at first delivery, menopause status and long-term estrogen exposure were independent risk factors for development of cervical cancer.
•Death due to cervical cancer among Korean women was associated with high level of fasting serum glucose (FSG) (RR=1.73, 95%CI=1.03-2.92), late stage of hypertension (RR=2.00, 95%CI=1.06-3.59) and smoking.
51. Evaluation of Mobile Phone-based Diet Game for Weight Control
WonbokLee, Young Moon Chae, Sukil Kim,, Seung Hee Ho, Inyoung Choi
(Published in Journal of Telemedicine and Telehealth, 2009; 16: 270-275)
51
52. Introduction
•Background
–3rdNational Health and Nutrition Survey conducted in 2005 reported that 32% of Korean adults were obese, based on the same criteria. Obesity is related to a metabolic syndrome and therefore obese people are at a greater risk of developing cardiovascular disease
–web-based therapy management system with mobile phone access has been developed and distributed to obese patients to support weight management.
•Significance
–We developed the SmartDiet, a mobile phone-based weight control system that not only closely tracks an obese patient’s daily nutrition intake, but also has games that users can play to learn about weight control.
–The SmartDietis different from the previous dieting applications in a sense that users could download and implement personalized dietary information onto their personal mobile phones
52
53. Objectives
•This study evaluated the effectiveness of the mobile phone application SmartDietwith respect to acquiring dietary information, weight control, and user satisfaction.
•Specific objectives are:
–To describe demographic characteristics and lifestyle of study population
–To analyze the effects of SmartDietby comparing body composition between case and control group
–To analyze user satisfaction
53
54. Research Design
•Subjects
–The effectiveness of the SmartDietwas evaluated based on the pretest-posttest with control design.
–19 subjects were assigned to the case or intervention group, and 17 subjects were assigned to the control group.
•Statistical Analysis
–Differences in demographic characteristics and lifestyle between the intervention group and the control group were analyzed by the Chi-square test.
–The changes in body composition before and after the intervention for both groups were compared by the paired t test.
54
58. Management Issues in Health Information Systems
58
Kwak EA, Chae YM, Ho SH, Kim GK. Management issues in hospital information systems.
(Published in Journal of Korea Society of Medical Informatics.
2007; 13 (1): 9-18)
59. •Purpose
–This study was conducted to identify management issues in hospital information systems
–Specifically,
•To identify management issues in 2007 and compare them with 1999
•Classify management issues by using McFarlan’sframework
•Methods
–Sample survey of 50 managers from 28 hospital information centers.
–Two rounds of interview surveys were conducted by using the Delphi method.
59
60. Changes in Management Issues
60
Key Issues
Rank
7-year
Change
2007
1999
TopManagementSupport
1
1
0
CooperationwithinISOrganization
2
4
+2
PACS
3
18
+15
DisasterRecovery
4
-
NEW
ImprovingISStrategicPlanning
5
17
+12
Standardization
6
-
NEW
LegalizationofElectronicMedicalRecord(EMR)
7
6
-1
ImprovingInformationSecurity&Control
8
3
-5
EMR
9
12
+3
EducatingSystemEngineers
10
7
-3
Reference: KwakEA, Chae YM, Ho SH, Kim GK. Management issues in hospital information systems. Journal of Korea Society of Medical Informatics. 2007; 13 (1): 9-18
61. Classification of Management Issues Based on McFarlan’sFramework
61
High
Present
Low
KeyOperational
Strategic
EducatingSystemEngineers
TopManagementSupport
EMR
AligningtheISOrganizationwithintheEnterprise
PACS
Standardization
DisasterRecovery
ImprovingISStrategicPlanning
Support
HighPotential
ImprovingInformationSecurity&Control
IntegratedHIS
MakingEffectiveUseofDataResource
IS/ITStrategy&Planning
BuildingPatientInformationSystem
InfrastructureforU-Health
Recruiting&DevelopingISHumanResource
ActivationofUbiquitousApplicationSystem
CollaborativeITSystemOrganization& Hospital
DevelopingLaboratoryInformationSystem(LIS)
ImprovingtheEffectivenessofSoftware
Development
LowHigh
Future
62. Competency-based Learning for Distance Education in System Analysis and Design
Byung HwaLee, Young Moon Chae, Tomiko Hokama, Seok Kim
(Published in Asia-Pacific Journal of Public Health.
2010; 22(3): 299-309)
62
63. Purposes
•To assess how well SAD competencies could be taught via a CBL-based e-learning course that we developed for students registered at the ICUH.
•Specifically,
–To develop a CBL-based curriculum by linking various competencies to the learning goals of SAD.
–To develop a CBL model for SAD based on the curriculum
–To measures the competency scores of students three times (before, during, and after the class) by using a competency diary, in order to assess the students’ progress.
–To identify the factors affecting learning effectiveness.
63
64. Research Framework for CBL Course Evaluation
64
Learning process
Case and problem
Tutor’s role
Student’s role
Learning
effectivenessMultiple regression
