This paper is presented in Proceedings of the 2020 International Conference on Big Data in Management organized by the University of Manchester UK.

1. USABILITY ANALYSIS OF EDUCATIONAL INFORMATION
SYSTEMS FROM STUDENT’S PERSPECTIVE
Dr. Nadeem Ahmad Ch.
Professor/ Pro Vice- Chancellor
• The University of Sialkot
• Dual PhD in HCI (Politecnico di Torino, Italy
& University of Potsdam, Germany)
• 17 Years Teaching and Research Experience
• 24 International Conference and Journal
Publications
• Head of People Centered Technology
Research Group
• Former Head of CS & IT department, The
University of Lahore
Presenter:

2.  INTRODUCTION
 RESEARCH QUESTIONS
 LITERATURE REVIEW
 PROPOSED METHDOLOGY
 DATA COLLECTION & SAMPLING
 RESULTS AND ANAYYSIS
 CONCLUSION AND DISCUSSION
CONTENTS

3.  At the height of nation-wide lockdowns in COVID-19 during April 2020, approximately 91
percent of the world's students in more than 194 countries were out of school [1].
 This research evaluates educational information systems, online teaching,
teaching modes, teacher’s interaction with students, course readiness and
institutional support from a student's perspective that are primary stakeholders in this
process
 User Centered Design methodology is used to evaluate the trust and satisfaction
level of students upon online teaching
 The 200 students participated in a comprehensive survey from the United Kingdom,
Germany, China, India and Pakistan
 Afterwards interviews were conducted in focus groups in which 30 students and 30
instructors participated.
 This study provides a baseline for institutes, policy makers, instructors and
developers to rethink how student satisfaction rate could be increased by their
active involvement.
INTRODUCTION

4. RQ 1: Whether students are satisfied from educational information systems?
RQ 2: Whether students are satisfied from contents being delivered and assessments
being evaluated?
RQ 3: Whether the faculty members have appropriate training for online courses
delivery?
RQ 4: Whether appropriate technical support is available in case of any difficulty?
RESEARCH QUESTIONS

5.  Multiple modalities are considered in e-learning including blended learning, virtual
learning, mobile learning, distance learning, online learning, simulated based
learning, but each of these has different functionalities and operating modes [2]
 E-learning composed of three key components e.g. (i) pedagogic models (ii) instructive
and learning plans and (iii) the educational tools and technologies [3].
 Issues in educational information systems from student’s perspective fall in two
categories: students’ attitude and satisfaction [4]
 Information quality, system navigation, system learnability, virtual design,
responsiveness, instructional assessment and system interactivity for usage of online
information systems were found critical factors for improving usability of mobile
learning [5].
LITERATURE REVIEW

6.  The challenges found in online education are classified into four major categories
that were: technological, individual, cultural and course challenges [6].
 In a research performed in 2020, the researchers performed study analysis in Covid-19
pandemic to address the feasibility of academic courses virtualization, impact on the
student learning process and designed four steps pipeline e.g. (I) Curricula
classification and analysis, (II) Evaluation of student’s condition and needs, (III)
Continuing education for faculty and (IV) Activation of virtual educational platforms [7].
 In another study performed in 2020, identified four themes of challenges that were
poor motivation and expectation, resource intensive, not suitable for all
disciplines/contents and lack of IT skills are the main barriers that impede the information
systems to make it successful in health sciences education [8]
LITERATURE REVIEW

7. Our methodology is primarily based on three steps:
 Identified existing approaches used for remote teaching
 we conduct a survey questionnaire in which IT related students are involved to explore the
challenges of online education. Questionnaires and Interviews methods are used to inquire
about student’s learning. The questionnaire has three parts:
 Evaluates the student rating regarding educational information system and course
development by asking six different questions
 Five different questions are asked about interaction and engagement with the instructor
 Asks students about support provided by their institute
 In the last step on the basis of usability questionnaire results we analyzed the results and
proposed our guidelines to make on-line teaching more effective.
METHODOLOGY

8. Usability Questionnaires for Evaluating Online Teaching
Q1. Are you satisfied with the educational information system being used for teaching in your institute?
Q2. Whether the courses are well communicated by using educational information system?
Q3. Do you feel that educational information system is enhancing your learning?
Q4. Whether you can easily learn your lab work/ practical’s in online classes?
Q5. Whether the online activities are designed to get the best out of students?
Q6. Whether you found the online courses are visually appealing and attractive?
Q7. Whether the workload for the online courses are too heavy? (reversed)
Q8. Whether you are able to solve your assignments with the help of instructors?
Q9. Whether your assessments are properly marked and results are provided to you in online classes?
Q10. I didn’t receive enough helpful online feedback from my teachers. (reversed)
Q11. Does the instructors encourage you for online discussions?
Q12. In case of technical problems, do you get help from your institution?
TABLE 1. STUDENT PERCEPTION ON USABILITY QUESTIONNAIRE

9. A. Interview Questions for Students
Q1. Which educational information system is being used by your institute for online lectures delivery?
Q2. How do you usually access the Internet?
Q3. Whether the educational information system is easily configurable or accessible on your system?
B. Interview Question for Teachers
Q1. Whether you have done any kind of training before preparing online lectures?
Q2. Whether you have tailored contents according to online teaching
Q3. What steps are you taking for better student interactivity?
INTERVIEW QUESTIONS

10.  we sent an email request to 543 students for questionnaire
 Around 253 students have filled an online questionnaire
 we found 53 students submitted incomplete surveys, so their information was discarded
SAMPLING
TABLE 2. PARTICIPATING STUDENTS FROM FIVE DIFFERENT COUNTRIES
COUNTRIES MALE
STUDENTS
FEMALE
STUDENTS
TOTAL
United Kingdom 20 10 30
Germany 30 11 41
China 35 10 45
India 20 9 29
Pakistan 40 15 55

11.  The results show that overall satisfaction of the students upon educational information
system remained satisfactory which was on average 3.8 (S.D = 1.1) and 95% confidence
interval was 0.2.
RESULTS AND ANALYSIS
FIGURE 1. STUDENT’S SATISFACTION UPON EDUCATIONAL INFORMATION SYSTEM AND
ENGAGEMENT WITH COURSES

12. RESULTS AND ANALYSIS
TABLE 3. COUNTRIES WISE AVERAGE SATISFACTION ON INFORMATION
SYSTEMS
COUNTRIES MALE FEMALE OVERALL
MEAN S.D MEAN S.D MEAN S.D
United Kingdom 4.6 0.5 4.6 0.5 4.6 0.5
Germany 4.6 0.5 4.5 0.5 4.5 0.5
China 4.1 0.7 4.2 0.6 4.1 0.7
India 3.5 0.8 3.7 1.1 3.6 0.9
Pakistan 2.6 1.0 2.1 1.0 2.5 1.0

13. RESULTS AND ANALYSIS
FIGURE 2. STUDENT’S ENGAGEMENT AND INTERACTION WITH INSTRUCTOR

14. RESULTS AND ANALYSIS
FIGURE 3. TECHNICAL HELP AVAILABLE IN DIFFERENT COUNTRIES BY
INSTITUTES

15.  In covid-19 both teachers and students around the world are facing hardships due to lack of
exposure from a student perspective and lack of training from an instructor's perspective
 Video lectures without sufficient interaction with students are not achieving required goals and
unable to fulfil course objectives
 In some low middle countries, students are getting depressed with
 Insufficient support
 High charges in tuition fee
 Force learning in online environment where teachers are not trained for (how to deliver
course contents, how to conduct assessments or lab work, how to engage the students)
usage of educational information systems
 Resulting in shape of anger and fear of failure in students
CONCLUSION AND DISCUSSION

16.  The findings of this research are based on usability questionnaires and interviews which
identified important factors:
 Students are getting less assistance in solving their problems
 Insufficient training of tools used for virtual lab simulations
 Low regular feedback by instructors
 Less encouragement in classes
 Overburden activities are a source of dissatisfaction
 This research is useful for all stakeholders, policy makers, educationalists to understand
the hardships which students are facing during the use of educational information
systems and provide analysis for improvements
CONCLUSION AND DISCUSSION

17. [1] Education: From disruption to recovery. UNESCO. 2020 [cited 20 November 2020]. Available from:
https://en.unesco.org/covid19/educationresponse
[2] Frehywot, S., et al., E-learning in medical education in resource constrained low-and middle-income
countries. 2013. 11(1): p. 4.
[3] Dabbagh, N.J.I.j.o.t.i.t. and learning, Pedagogical models for E-Learning: A theory-based design
framework. 2005. 1(1): p. 25-44.
[4] Nariman, D. Impact of the Interactive e-Learning Instructions on Effectiveness of a Programming Course.
in Conference on Complex, Intelligent, and Software Intensive Systems. 2020. Springer.
[5] Vlasenko, K., et al. Usability analysis of on-line educational courses on the platform “Higher school
mathematics teacher”. in E3S Web of Conferences. 2020. EDP Sciences.
[6] Martin, F., K. Budhrani, and C.J.O.L. Wang, Examining Faculty Perception of Their Readiness to Teach
Online. 2019. 23(3): p. 97-119.
[7] Chavarría-Bolaños, D., et al., E-Learning in Dental Schools in the Times of COVID-19: A Review and
Analysis of an Educational Resource in Times of the COVID-19 Pandemic. 2020: p. 207-224.
[8] Regmi, K. and L.J.B.m.e. Jones, A systematic review of the factors–enablers and barriers–affecting e-
learning in health sciences education. 2020. 20: p. 1-18.
REFERENCES

18. THANK YOU!