SlideShare une entreprise Scribd logo
1  sur  34
GCompris Usability Testing   1




GCompris Usability Testing Research Report




       Shakarah McCrae - 06004107

         AR3X0 – Research Paper

              May 15, 2009




        Supervisor:      David Bain

     Course Moderator:      Dr. Ramsay




          Table of Contents
GCompris Usability Testing   2




Executive Summary………………………………………………………………… ...2

Introduction…………………………………………………………………………… 3

    Research Question…………………………………………………………......3

    Null Hypothesis………………………………………………………………..3

    Background…………………………………………………………………….3

    Worth of Study………………………………………………………………....6

    Objectives………………………………………………………………………6

Literature Review………………………………………………………………………8

Methodology…………………………………………………………………………...11

    Procedure………………………………………………………………………12

    Participant Summary…………………………………………………………..13

    Gcompris Set-up……………………………………………………………….14

Data Presentation………………………………………………………………………16

Analysis………………………………………………………………………………..23

Conclusions and Recommendations ………………………………………………….25

    Limitations……………………………………………………………………

References……………………………………………………………………………..27

Appendices………………………………………………………………………….....29




                       Executive Summary
GCompris Usability Testing       3


This study is in line with the requirements of the University of the West Indies where a research

project has to be done. This topic was chosen in accordance with my field of study and to

discover how influential or effective this medium is in the classroom.


Gcompris was chosen because it is free and can be used as a tool for teaching basic math to

students. The research is aimed at showing how user friendly the software is and how the

students will react to using it.


Usability testing is used to measure the appropriateness of software for its user. It is aimed at

assessing the effectiveness, efficiency and satisfaction with which specified users can achieve

specified goals in particular environments or contexts of use.


The chosen open source software for this purpose is GCompris, which is an educational software

suite comprising of numerous activities for children, aged 2 to 10. Some of the activities are

game orientated, but nonetheless educational. More specifically, I will be looking at the extent to

which G-Compris, as a tool for teaching basic mathematics skills can improve or aid learning in

mathematics for grade one students at August Town Primary School.




                                          Introduction

Research Question:
GCompris Usability Testing          4


How effective is Gcompris- an open source software- for teaching basic mathematics skills to

grade one students at August Town Primary school?


Null Hypotheses:


Gcompris as an educational tool is not beneficial in improving the performance of grade one (1)

students in school.


Background


          The disappointing performance of Jamaican students in the mathematics subject area

continues to be an increasing concern to the wider Jamaican society. According to Francis

(2006):

          “New statistics from the CXC show that of the 35,428 students who sat the 2006

          mathematics exam, only 32 per cent managed to achieve grade one to three. This

          represents a decrease over last year's 36 per cent. This figure is also three per cent below

          the regional average” (para. 2).

In fact, Wesley Barrett Pro-Registrar for the Caribbean Examination Council states quot;These

results are disappointing and it is still a very worrying problem,quot; ” (Francis, 2006, para. 2). Poor

mathematics performance is not only at the Caribbean Secondary Education Level but also at the

Primary Level.



          With this in mind and out of the need to stem the problem from a tender age, I have

decided to carry out a usability study of an open source software for teaching basic mathematics

skills to grade one students in Jamaican primary schools. Usability testing “is a technique for
GCompris Usability Testing         5


ensuring that the intended users of a system can carry out the intended tasks efficiently,

effectively and satisfactorily.” (Gaffney, 1999)



       The chosen open source software for this purpose is GCompris, which is an educational

software suite comprising of numerous activities for children, aged 2 to 10. Some of the

activities are game orientated, but nonetheless educational. (http://gcompris.net/-en- )More

specifically, I will be looking at the extent to which G-Compris, as a tool for teaching basic

mathematics skills can improve or aid learning in mathematics for grade one students at August

Town Primary school.




Worth of Study


       The worth of this research is multi-facet in nature. If we can identify software that can aid

and improve the performance of children from a tender age, we will not only be facilitating

improvement in the grades but also a greater appreciation and a more fun way of learning the

subject. It is of academic value because it provides information to educators and their

stakeholders as to the value of educational software to students and the implications it can have.

In addition, it provides valuable information to be taken into consideration by developers of

educational software for students.


Objectives of the study include:

    1. To determine whether or not Gcompris is effective and appropriate for children six (6)
       and seven (7) years of age

    2. To use the findings of this research to make recommendations to Educators as to one way
       of improving students performance in school
GCompris Usability Testing       6


   3. To prove whether or not Gcompris as an educational tool is beneficial to children

   4. To evaluate students learning potential based on the activities available on GCompris

   5. To document findings that are as a result of the usability research done.



   Dominant terms that will be used throughout this research are Usability, usability testing,

educational, multimedia, multimedia cognitive theory, activity theory and children. There is an

Independent Variable and a Dependent Variable. The independent variable is GCompris and the

dependent variable is the August Town Primary Grade one (1) students. The main theories that

will be guiding this research are the Multimedia Cognitive theory, Cognitive Load theory, The

Multimedia Principle and the activity theory.



   For this testing, eighteen grade one primary student with little or no computer experience will

be chosen randomly. The eighteen students will be further divided into two groups of nine

and exposed to the software for 20 minutes on two separate occasion. The students will be

observed, recorded, and a post-test evaluation will be done.



       This research report provides background, findings and recommendations of the

Gcompris Usability testing. It documents and discusses theories and methodologies of the testing

between March 17 and April 21, 2009. The report provides participant demographics, test

evaluation methods and detail additional information that is necessary for this report. Our

recommendations are based on the data-driven findings and the user-centered theory.
GCompris Usability Testing     7




                                     Literature Review



       According to Sashoi Green, teacher at Jamaica College (Clarke, 2009, para 6) quot;There is a

phobia of mathematics, but when students relate it to things around them, they connect more.quot; In
GCompris Usability Testing      8


a report in the Jamaica Gleaner, She said students develop a dislike for mathematics at an early

age. (Clarke, 2009, para. 8). This line of thought is in keeping with the train of thought that

exists. In fact, with the current low performance in mathematics within Jamaican Schools, new

policies have been designed and researches are being conducted in an effort to stem the problem.

Another perception of the cause of this low performance is provided by the Government of

Jamaica who has admitted in a statement recently by Opposition spokesman on education,

Senator Basil Waite that “Jamaican students are performing poorly in the area of mathematics

because many primary school teachers did not pass the subject at high school level.” (Douglas,

2009, para. 1)


       This problem has led to developments, research and policies that have surfaced. One

example of such is the New Horizons for Primary Schools (NHP), which was implemented by

the United States Agency for International Development (USAID) in 1999 and lasted until 2004.

“The objective of the program was to improve the language arts and mathematics performance of

72 of Jamaica’s poorer performing schools, through a school-based model of intervention.”

(Marlaine, Harris, Gammill, & Barrow, 2006, p.1 ). “The NHP employs a holistic, student centre

approach to improvement of instruction and learning in mathematics and reading (n.d.)


       Another current example is the National Mathematics Strategy being undertaken by the

Education Transformation Team under the Ministry of Education. National Mathematics

coordinator, Tamika Benjamin, says the launch of this strategy was the “beginning of a quot;long and

difficult journeyquot;. (Anonymous, 2008, para. 9)


“Through this strategy, direction will be given to stem the continued decline in student

performance and to increase national awareness about the importance of
GCompris Usability Testing          9


numeracy.”(Anonymous, 2008 ,Para. 2) the establishment of a National Mathematics and

Numeracy Policy, which is aimed at improving the performance of students’ island wide.



       While there are many theories that speak to learning and the use of multimedia

application to aid the learning, the theories I will be focusing on for the purpose of this research

are The Cognitive Theory of Multimedia, Cognitive Load theory, The Multimedia Principle and

the activity theory. According to Mayer & Moreno (2002) The Cognitive Theory of Multimedia

states that multimedia narration and graphical images produce verbal and visual mental

representations, which integrate with prior knowledge to construct new knowledge.(to cite).



       According to Mayer and Moreno (1998) and Mayer (2003), the Cognitive

Theory of Multimedia Learning is based on several assumptions. First, working memory

includes auditory and visual channels, which are equivocal to the phonological loop and the visu

CognitiveTheory & Multimedia Instruction spatial sketch pad in Baddeley’s (1986) theory of

working memory. Second, each subsystem of working memory has a limited capacity, consistent

with Cognitive Load Theory (Sweller, 1988, 1994). Third, humans are knowledge-constructing

processors who produce meaningful learning when they attend to relevant incoming information,

organize the information in coherent representational structures, and then integrate it with other

existing knowledge (Mayer, 1996, 1999).

Fourth, connections can be made only if corresponding visual and verbal representations are in

working memory at the same time, which is similar to Paivio’s (1986; Clark and Paivio, 1991)

Dual Coding Theory.
GCompris Usability Testing       10


“Cognitive Learning Theory is concerned with the way in which a learner’s cognitive resources

are focused and

used during learning and problem solving, suggesting that for instruction to be effective, care

must be taken to design instruction in a way as to not overload the mind’s capacity for

processing information.”(to cite)



Multimedia Principle

The multimedia principle states that better transfer occurs from animation/pictures and narration/

words than from words alone. When words and pictures are both presented, learners have the

chance to construct verbal and visual cognitive representations and integrate them.

Limitation to this study includes the fact that there have not been much recorded accounts of

similar studies conducted. This limits the resources available to the researcher and the

information for which to base the research on.

The Activity Theory based usability testing methodology offers evaluators an insight into the

natural context of use in an artificial laboratory setting which offers a high degree of control. (to

cite)



                                          Methodology

For the purpose of this research, the group that I have decided to study is grade one (1) students

at the August Town Primary School. Eighteen students were randomly chosen, Nine from each

of the two grade one students classes that were present.


In an effort to effectively choose methods that would reflect the required data needed, two main

usability evaluation methods based on Preece (1993), were considered and chosen. These are
GCompris Usability Testing      11


observational evaluation and survey evaluation. “Observational evaluation implies collecting

data that provide information about what users do when interacting with educational software.”

Preece (1993) stated that two broad categories of data may be obtained: how users tackled the

tasks given, where the major difficulties lie and what can be done; and performance measures

like frequency of correct task completion, task timing, frequency of participant errors.

“Surveys are employed to know users' opinions or to understand their preferences about an

existing or potential product through the use of interviews or questionnaires.”

The techniques that were used include direct observation, video recording, software logging,

verbal protocols and Questionnaire administered by interviews.

The Gcompris usability evaluation was conducted as monitored test sessions. Eighteen (18)

students were tested. While this number is not representative of the general grade one population

in Jamaica for a true experimental design, a minimum of 10 to 12 participants per condition must

be used (Spyridakis, 1992). On the other hand, new research has shown that four to five

participants can reveal up to 80% of usability insufficiencies in products, which often times tend

to be most of the major problems (Virzi, 1990).

The test sessions were held in a computer lab on the computer science department of the

University of the West Indies, Mona campus.

Procedure

The eighteen students were divided into two groups of nine and the tests carried out in four

sessions, each group being exposed to two. Each testing lasted for twenty minutes per session.

From henceforth on, the two groups will be referred to as Group A and Group B.

For the first session, Group A had three persons monitoring the test and Group B had five

persons. This was due to the availability of persons to participate as monitors. The students were
GCompris Usability Testing         12


transported from their school to the lab. On arrival, the participants were lead to the computer lab

where there were each placed in front of a computer which had the software on screen. The

students were then given one minute without instructions to see if they would know what to do.

After the initial minute, the participants were first introduced to the computer keyboard aspect of

the software where they were allowed to do one (1) activity of falling letters to allow them to

familiarize themselves with the keyboard. After the basic introduction with the computer, the

students were then exposed to basic addition and subtraction mathematics activities. All 9

students were exposed to the mouse where they were allowed to do an activity that required them

using the mouse to click the correct answer.


Two activities were chosen for this.


   1. where they were allowed to connect the numbers to draw an object. This not only aided

       their mouse using skill but also tested their skills in counting.


   2. Others were tested based on their ability to read and identify objects where they were

       shown different objects and they were required to click the letter that was missing.


This first session was used as an introductory session. The students were monitored and

comments and expressions were monitored.


The second session however was of a different format, as the students were first exposed to

mathematics addition activities for the first 10 minutes then subtraction activities for the other

ten minutes. Their individual score was documented, after which a simple questionnaire was

administered to the participants individually.
GCompris Usability Testing          13


Participant Summary

Eighteen participants participated in the Gcompris usability study. Participants were randomly

selected based on their availability to participate in the study from the two grade one classes at

August Town Primary. The selected participants were mixed in their performance in

mathematics. That means some participants were exceptional performers, some needed extra

help and the others were rather slow. The participants were each given permission letters to give

to their parents/guardians, which were signed and returned to the researcher.

Participant     Age    Group      Sex    Previous Computer         Used other       Previous
                                         Usage(1-Expert, 5-        educational      Gcompris
                                         Novice)                   software         Usage

      1        6           1      F      5                              None              None

      2        6           1      M      5                              None              None

      3        6           1      M      5                              None              None

      4        7           1      F      5                              None              None

      5        7           1      F      5                              None              None

      6        6           1      F      5                              None              None

      7        7           1      M      4                              None              None

      8        7           1      M      5                              None              None

      9        6           1      F      4                              None              None

     10        7           2      F      5                              None              None

     11        7           2      M      5                              None              None

     12        6           2      M      4                              None              None

     13        7           2      M      3                              None              None

     14        6           2      M      5                              None              None

     15        7           2      M      5                              None              None

     16        6           2      F      5                              None              None
GCompris Usability Testing           14




     17         6          2       F      5                              None              None

     18         6          2       F      5                              None              None




Gcompris Set-up

Gcompris was pre-installed for each participant on the linux operating system at the computer

science computer lab. During the pretesting familiarizing section of the test, the participants were

introduced to the welcome screen where no instruction was given at first.


The participatory observation as an instrument of data collection is more appropriate in this case

than other instruments that exist. In that , while dealing with the children it is much better to be

able to interact with them while observation their reaction and behavior towards the software.

Furthermore, this is best suited as a usability study consists of and generally involves measuring

how well test subjects respond in four areas: efficiency, accuracy, recall, and emotional response,

which will be useful in what we are trying to achieve.



The aim of the evaluation was to validate the usability of the activities specifically the addition

and subtraction mathematical activities which, are features of Gcompris. The testers were given

activities to complete and their effectiveness, efficiency, reaction and satisfaction of the software

were noted.
GCompris Usability Testing       15




                                      Data Presentation


Of the eighteen participant respondents, nine were female and the remaining nine male. Giving a

fifty-fifty percentage. However, group one had five females and four males while group two had

four females and five females. The respondents are all between the age of six and seven. In fact

of the ten of the participants are six years of age while the remaining eight are seven years of

age.




                          Usability Problems            Number of
                                                   Participants affected

                         Time given for                      18
                         activities

                         Needs Supervision                   14
GCompris Usability Testing        16




                         Navigation                         14

                         Participants were                  18
                         unsure what to do



Figure 1: Major Usability Problems and participants affected.



Figure one shows the major usability problems and the number of participants affected by it.

The Time given to complete the activities is at the top because it is the greatest concern among

the participants. However, with time and with practice, the respondents will be allowed to

complete the activities within the given time.


Participant    Age    Group     Average Activity            Average Activity Score- Subtraction
                                Score- Addition

    1         6          1      8/10                                         7/10

    2         6          1      5/10                                         3/10

    3         6          1      7/10                                         8/10

    4         7          1      3/10                                         3/10

    5         7          1      6/10                                         5/10

    6         6          1      5/10                                         6/10

    7         7          1      9/10                                         8/10

    8         7          1      10/10                                        9/10

    9         6          1      7/10                                         5/10

    10        7          2      3/10                                         1/10

    11        7          2      2/10                                         2/10

    12        6          2      4/10                                         3/10

    13        7          2      3/10                                         2/10
GCompris Usability Testing       17




    14        6          2      8/10                                         7/10

    15        7          2      6/10                                         5/10

    16        6          2      4/10                                         3/10

    17        6          2      5/10                                         5/10

    18        6          2      6/10                                         6/10
Figure 2: Table showing average performance by the participants in the mathematics activities.

Group one had a stronger set of participants when compared to the average score of group two.

Although, there were a few strong participants in group two, majority of the participants required

extra attention and was much slower in their performance.


The list below represents some findings based on our participant responses to the feedback
survey:




   • Of our 18 participants, 14 indicated that they had no previous computer exposure.
GCompris Usability Testing   18




    • Two of the four with previous experience said they had access to a computer at home.
.




    • Of the 18 participants, none had previous exposure to Gcompris prior to the testing
GCompris Usability Testing        19




10 out of the 18 participants suggested they would only use the software again if they had

someone to help them as these students stated that the math activities were too hard. These 10

students happen to be the ones that scored the lowest on the activities. When asked why they

would not want to use the software again, the most prominent answer was because they did not

know how to do the math. However, some said they would use it again if they were supervised.




Other Important Findings that came out of the Research


   1. Participant 11 Boy is confused when it comes it comes to math, you have to completely

       break it down for him or work with him. He is easily distracted and works better on other

       activities


   2. Participant #12 is unable to do basic addition, she is very slow and loses focus easily

       when she gets the answers incorrect. She s unable to recognize basic mathematics signs

       on her own.
GCompris Usability Testing      20


3. Participant number 10 is very fast, she uses her fingers to assist her in solving problems.

   She works off her own initiative and is twice as fast as the others


4. Participant #1 likes math


   Likes to do it on the computer


   Likes the “thing” that she is using to do the math


5. Participant #7 likes subtraction


   -father encourages him to do math


   -likes the software


   -the math was too hard


   -wanted someone to help him because he did not know what to do


   -would not use it again because it was too hard.




6. Participant #5 – likes it because its nice


           Likes the game with the fish


           It was too hard because he can not do the math


           He would use it again if he had somebody to help him


7. Participant # 3 it was fun


   Fish game is her favourite
GCompris Usability Testing       21


       The math was not hard


       Yes would use it again


       Has a computer at home.


   8. Participant #2 thought some of the math were hard, wanted someone to help with the

       ones that were hard, could do the math


When asked if they would like the teacher to use it in class 90% said yes


Activities the children liked


   1. Letters that fall from the sky - 2


   2. One participant response was her stomach hurts her when she does not know what to do.


   3. Another one likes the addition


Disapproval


   -   The time was too short


   -   They were not familiar with the layout of the computer keyboard


       It is also important to note that the age group used for this exercise was 6-7 so one might

say that the level of work is very easy, but the ability of the students was mixed and so there

were those who were performing at a lower level and those who were excelling. Nevertheless,

the students of both mental capacities were able to use the software with relative ease and

explanation at times.
GCompris Usability Testing       22




                                             Analysis


       From the test conducted and from general observations and posttest evaluation, various

findings from which generalizations can be made were found. With reference to the effectiveness

of the GCompris it was found out that although the majority of the participants enjoyed playing

the other activities they did not like the math activities that were available for more than one

reasons, time availability and their ability perform well at the activities. Although the math

questions being asked in the activities were appropriate, 80% of the participants felt that they
GCompris Usability Testing          23


needed to be guided when participating in the activities. This is due to their ability to calculate

and function on their own as these participants were not as academically inclined as the others.


       While 50% of the participants did not have a problem with the mathematical activities,

the time issues was of a major concern to these participants as they used their fingers to help

them calculate. However based on the researcher’s observations, the participants will eventually

perform better once they are exposed to the software and general computer usage skills on a

weekly basis.


       Twenty percent of the participants managed the mathematical program very well because

they were able to apply the skills taught in basic computation to those that were being tested on

the software. For example, one student was given a computation of 10-6=___, immediately she

held up her ten fingers and subtracted 6 and was able to reel the answer (4) in seconds. Another

option was the reading skills which asked them to fill in the missing letters. This was done with

ease as well because the students already had a grasp of the words and so identifying the missing

letters was pretty easy. Owing to the fact that they were using the computers for the first time the

difficulty came when they had to identify the keys on the keyboard in order to get the right

answer.


       It is also important to note that the age group used for this exercise was 6-7 so one might

say that the level of work is very easy, but the ability of the students was mixed and so there

were those who were performing at a lower level and those who were excelling. Nevertheless,

the students of both mental capacities were able to use the software with relative ease and

explanation at times.
GCompris Usability Testing      24


Overall, the participants were enthusiastic about the software and the activities available on the

software. They enjoyed playing the various games.




                           Conclusions and Recommendations

From this usability testing research on the extent to which G-Compris, as a tool for teaching

basic mathematics skills can improve or aid learning in mathematics for grade one students at

August Town Primary school, numerous recommendations can be made. Some of these

recommendations have been put forward by monitors and participants themselves and the others

will be put forward by the researcher based on the process and findings of the research.

Recommendations put forward by the monitors/evaluators/participants include
GCompris Usability Testing       25


   o The length of the time on the software needs to be slower which will seek to
     accommodate students that have learning disabilities and are not so quick.

   o Students on initial exposure to the software have to be closely monitored in order to fully

       utilise and captivate the benefits that can be had from the software.

Recommendations put forward by the researcher include:


   •   A comparative study to be done between a preparatory and a primary school.

   •   More recorded research including that of usability testing should be done in the

       Caribbean so that Caribbean researchers including students may have readily available

       literature to them when conducting research of this nature and others.

   •   The developers of children software needs to take into consideration children who are not

       tech-savvy hence the researcher is recommending that the developers of GCompris allot

       more time to the completion of the initial activities in order to allow users to get familiar

       and comfortable with the product.

   •   A computer lab should be made available to the students with academic software such as

       Gcompris installed so as to exposure them to alternative methods of learning which will

       be beneficial to the teachers and students alike.

   •   The Ministry of Education should re-visit the syllabus

In conclusion, the researcher discovered that while GCompris can indeed improve or aid the

performance of students in mathematics, the timing available for completion of the activities

needs to be revisited for students that are not that academically inclined and needs more time to

work on the activities. The fact that the most of the participants who were all unfamiliar with

GCompris could learn to use it fairly with just two exposures suggests that GCompris is

generally usable for the intended target population. However the developers of the software
GCompris Usability Testing      26


needs to look in lengthening the time given for completion of each activity as this present timing

is not conducive or receptive of slow learners.
GCompris Usability Testing     27


                                        References

Clarke, O (2009, April 18) Curing the math phobia - Math competition shows problem solving to

       be a good tool. Retrieved January 27, 2009 from

       http://www.jamaica-gleaner.com/gleaner/20090418/lead/lead4.html


Douglas, L (2009, April 5) 'WE HAVE A MATH PROBLEM'. Retrieved January 27, 2009 from

       http://www.jamaicaobserver.com/magazines/Career/html/20090404T220000-0500_1488
       20_OBS__WE_HAVE_A_MATH_PROBLEM__.asp



Francis, P (2006, September 15) Poor CXC results - Jamaican students lag behind region in

       Maths, English. Retrieved January 26, 2009 from

       http://www.jamaica-gleaner.com/gleaner/20060915/lead/lead1.html



Gaffney, G (1999) Information & Design: Usability Testing Series. Retrieved January 26, 2009

       from http://www.infodesign.com.au/ftp/UsabilityTesting.pdf



Marlaine, L., Harris, A, Gammill, P & Barrow, K (2006) IMPACT OF NEW HORIZONS FOR

       PRIMARY SCHOOLS ON LITERACY AND NUMERACY IN JAMAICA 1999-2004.

       Retrieved on February 5, 2008 From

       http://www.equip123.net/JEID/articles/2/NewHorizons.pdf



Preece, J. (1993). A Guide to Usability: human factors in computing. Addison Wesley, the Open

       University.
GCompris Usability Testing   28


Rubin, Jeffrey. (1994). Handbook of Usability Testing: How to Plan, Design and Conduct

       Effective Tests. New York: Wileyl


Spyridakis, J. H. (1992). Conducting Research in Technical Communication: the application of

       true experimental designs. Technical Communications, Fourth Quarter, 607-624.




Virzi, R. A. (1990). Streamling in the Design Process: Running Fewer Subjects. Proceedings of

       the Human Factors Society, 291-294.

Educators count on new maths strategy (2008, June 4) Retrieved on January 27, 2009 from

       http://www.jamaica-gleaner.com/gleaner/20080604/news/news3.htmlv



Mayer, R. E. (1996). Learning strategies for making sense out of expository text: The

       SOI model for guiding three cognitive processes in knowledge construction. Educational

       Psychology Review, 8, 357-371.



Multimedia in Jamaican Schools retrieved on January 31, 2009 from

       http://www.iicd.org/articles/IICDnews.import42/


http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/36/25/e2.

       pdf


http://expanding-educational-horizons.com/Numeracy/NumeracyPolicy.pdf
GCompris Usability Testing                29


                                                     Appendices


                                                                    Caribbean Institute of MEDIA and COMMUNICATION
                                                                                       The University of the West Indies
                                                                                         3 Sherlock Drive, Mona Campus
                                                                                                              Kingston 7
                                                                                                            Jamaica, W.I
                                                                              Tel. (876) 927-1481, 977-2111, 977-0898,
                                                                                         977-1398, 977-9990, 702-3353,
                                                                                                     Fax (876) 977-1597
                                                                                           E-mail carimac@uwimona.edu.jm
Caribbean Institute of MEDIA and COMMUNICATION
                                                                         Web   www.mona.uwi.edu/carimac

          Dear Parent/Guardian,



          Re: Permission for your Child/Ward to participate in a computer study



          This letter seeks permission for your child/ward to participate in a multimedia computer study to
          be conducted at the University of the West Indies Mona Campus. Although we would have
          preferred to conduct the study on the premises of the school, the computer lab is currently
          unavailable for use, hence the need for the study to be done on UWI’s campus. The project will
          run for 2 to 3 weeks commencing on Tuesday March 17, 2009. It will require that your child
          visits the UWI computer lab on two occasions within one of the following sessions (Tuesdays
          and Thursdays 9:00 am – 10:30 am) during duration of the study.



          Shakarah McCrae, a final year student at the Caribbean Institute of Media and Communication
          (CARIMAC), will conduct the study, which is intended to fulfil her research requirement to
          investigate multimedia software and children in the classroom.

          Your child will be required to use a computer with Mathematics learning software for a period of
          20 to 30 minutes.



          Sincerely,



          ----------------------------------------
GCompris Usability Testing            30


David Bain

Multimedia Co-ordinator CARIMAC, UWI Mona




-----------------------------------------

Shakarah McCrae

Final Year Student CARIMAC UWI Mona




-----------------------------------------

Morlton Wilson

Principal, August Town Primary School

--------------------------------------------------------------------------------------------------------------------
-



I ____________________________________ have given permission for

   Parent/Guardian



__________________________ my child/ward to participate in the multimedia computer



study to be held between March 17, 2009 and March 26, 2009 at the UWI campus.
GCompris Usability Testing   31


Screenshots showing sample activities played by participants




                          Addition
GCompris Usability Testing   32




Connect the dots
GCompris Usability Testing   33



Identify the Letters




 Guess the object
GCompris Usability Testing   34




Subtraction

Contenu connexe

Tendances

Lumadue, rick utilizing merlot content builder focus v7 n1 2013 2
Lumadue, rick utilizing merlot content builder focus v7 n1 2013 2Lumadue, rick utilizing merlot content builder focus v7 n1 2013 2
Lumadue, rick utilizing merlot content builder focus v7 n1 2013 2William Kritsonis
 
Comparative And Non Comparative Evaluation In Educational Technology
Comparative And Non Comparative Evaluation In Educational TechnologyComparative And Non Comparative Evaluation In Educational Technology
Comparative And Non Comparative Evaluation In Educational TechnologyYousuf Salim
 
A comparative analysis of i-Ready, IXL, and Prodigy learning software programs
A comparative analysis of i-Ready, IXL, and Prodigy learning software programsA comparative analysis of i-Ready, IXL, and Prodigy learning software programs
A comparative analysis of i-Ready, IXL, and Prodigy learning software programsCristinaRyter
 
Intelligent system for sTudent placement
Intelligent system for sTudent placementIntelligent system for sTudent placement
Intelligent system for sTudent placementFemmy Johnson
 
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...CS, NcState
 
PREDICTING ACADEMIC MAJOR OF STUDENTS USING BAYESIAN NETWORKS TO THE CASE OF ...
PREDICTING ACADEMIC MAJOR OF STUDENTS USING BAYESIAN NETWORKS TO THE CASE OF ...PREDICTING ACADEMIC MAJOR OF STUDENTS USING BAYESIAN NETWORKS TO THE CASE OF ...
PREDICTING ACADEMIC MAJOR OF STUDENTS USING BAYESIAN NETWORKS TO THE CASE OF ...ijcax
 
Fuzzy Association Rule Mining based Model to Predict Students’ Performance
Fuzzy Association Rule Mining based Model to Predict Students’ Performance Fuzzy Association Rule Mining based Model to Predict Students’ Performance
Fuzzy Association Rule Mining based Model to Predict Students’ Performance IJECEIAES
 

Tendances (8)

Feasibility test application of information systems in the media as a learnin...
Feasibility test application of information systems in the media as a learnin...Feasibility test application of information systems in the media as a learnin...
Feasibility test application of information systems in the media as a learnin...
 
Lumadue, rick utilizing merlot content builder focus v7 n1 2013 2
Lumadue, rick utilizing merlot content builder focus v7 n1 2013 2Lumadue, rick utilizing merlot content builder focus v7 n1 2013 2
Lumadue, rick utilizing merlot content builder focus v7 n1 2013 2
 
Comparative And Non Comparative Evaluation In Educational Technology
Comparative And Non Comparative Evaluation In Educational TechnologyComparative And Non Comparative Evaluation In Educational Technology
Comparative And Non Comparative Evaluation In Educational Technology
 
A comparative analysis of i-Ready, IXL, and Prodigy learning software programs
A comparative analysis of i-Ready, IXL, and Prodigy learning software programsA comparative analysis of i-Ready, IXL, and Prodigy learning software programs
A comparative analysis of i-Ready, IXL, and Prodigy learning software programs
 
Intelligent system for sTudent placement
Intelligent system for sTudent placementIntelligent system for sTudent placement
Intelligent system for sTudent placement
 
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...
 
PREDICTING ACADEMIC MAJOR OF STUDENTS USING BAYESIAN NETWORKS TO THE CASE OF ...
PREDICTING ACADEMIC MAJOR OF STUDENTS USING BAYESIAN NETWORKS TO THE CASE OF ...PREDICTING ACADEMIC MAJOR OF STUDENTS USING BAYESIAN NETWORKS TO THE CASE OF ...
PREDICTING ACADEMIC MAJOR OF STUDENTS USING BAYESIAN NETWORKS TO THE CASE OF ...
 
Fuzzy Association Rule Mining based Model to Predict Students’ Performance
Fuzzy Association Rule Mining based Model to Predict Students’ Performance Fuzzy Association Rule Mining based Model to Predict Students’ Performance
Fuzzy Association Rule Mining based Model to Predict Students’ Performance
 

En vedette

Increasing Learner Opportunities With Recent Technologies
Increasing  Learner  Opportunities With  Recent  TechnologiesIncreasing  Learner  Opportunities With  Recent  Technologies
Increasing Learner Opportunities With Recent TechnologiesAlice Bedard-Voorhees, Ph.D.
 
Ar3 X0 Final Research Paper G Compris Usability Research Report
Ar3 X0 Final Research Paper G Compris Usability Research ReportAr3 X0 Final Research Paper G Compris Usability Research Report
Ar3 X0 Final Research Paper G Compris Usability Research ReportShakarah Levy
 
Tools for Developing and Evaluating E-Learning Quality
Tools for Developing and Evaluating  E-Learning Quality  Tools for Developing and Evaluating  E-Learning Quality
Tools for Developing and Evaluating E-Learning Quality Alice Bedard-Voorhees, Ph.D.
 
How Are We Doing? Evaluating Technology-Based Practices in Our Courses
How Are We Doing? Evaluating Technology-Based Practices in Our CoursesHow Are We Doing? Evaluating Technology-Based Practices in Our Courses
How Are We Doing? Evaluating Technology-Based Practices in Our CoursesAlice Bedard-Voorhees, Ph.D.
 
Big data para la toma de decisiones en empresas de Turismo
Big data para la toma de decisiones en empresas de TurismoBig data para la toma de decisiones en empresas de Turismo
Big data para la toma de decisiones en empresas de TurismoEdwin Arley Bernal Holguin
 
Charla factores de exito en websites turisticos
Charla factores de exito en websites turisticosCharla factores de exito en websites turisticos
Charla factores de exito en websites turisticosEdwin Arley Bernal Holguin
 

En vedette (17)

At the qr bar (code).11084
At the qr bar (code).11084At the qr bar (code).11084
At the qr bar (code).11084
 
Free and Lifelong
Free and LifelongFree and Lifelong
Free and Lifelong
 
Increasing Learner Opportunities With Recent Technologies
Increasing  Learner  Opportunities With  Recent  TechnologiesIncreasing  Learner  Opportunities With  Recent  Technologies
Increasing Learner Opportunities With Recent Technologies
 
Start the Video
Start the VideoStart the Video
Start the Video
 
kcWorks nightscapes
kcWorks nightscapeskcWorks nightscapes
kcWorks nightscapes
 
Ar3 X0 Final Research Paper G Compris Usability Research Report
Ar3 X0 Final Research Paper G Compris Usability Research ReportAr3 X0 Final Research Paper G Compris Usability Research Report
Ar3 X0 Final Research Paper G Compris Usability Research Report
 
Tools for Developing and Evaluating E-Learning Quality
Tools for Developing and Evaluating  E-Learning Quality  Tools for Developing and Evaluating  E-Learning Quality
Tools for Developing and Evaluating E-Learning Quality
 
Ple2016
Ple2016Ple2016
Ple2016
 
How Are We Doing? Evaluating Technology-Based Practices in Our Courses
How Are We Doing? Evaluating Technology-Based Practices in Our CoursesHow Are We Doing? Evaluating Technology-Based Practices in Our Courses
How Are We Doing? Evaluating Technology-Based Practices in Our Courses
 
PLE
PLEPLE
PLE
 
Big data para la toma de decisiones en empresas de Turismo
Big data para la toma de decisiones en empresas de TurismoBig data para la toma de decisiones en empresas de Turismo
Big data para la toma de decisiones en empresas de Turismo
 
kcWorks landmarks
kcWorks landmarkskcWorks landmarks
kcWorks landmarks
 
Serious Games for Learning
Serious Games for LearningSerious Games for Learning
Serious Games for Learning
 
Web2.0 Buffet
Web2.0 BuffetWeb2.0 Buffet
Web2.0 Buffet
 
Charla factores de exito en websites turisticos
Charla factores de exito en websites turisticosCharla factores de exito en websites turisticos
Charla factores de exito en websites turisticos
 
ELCCmooc panel
ELCCmooc panelELCCmooc panel
ELCCmooc panel
 
Herramientas digitales para agencias de viaje
Herramientas digitales para agencias de viajeHerramientas digitales para agencias de viaje
Herramientas digitales para agencias de viaje
 

Similaire à Ar3 X0 Final Research Paper G Compris Usability Research Report

Clustering Students of Computer in Terms of Level of Programming
Clustering Students of Computer in Terms of Level of ProgrammingClustering Students of Computer in Terms of Level of Programming
Clustering Students of Computer in Terms of Level of ProgrammingEditor IJCATR
 
A Comparative Analysis of i-Ready, IXL, and Prodigy e-learning software programs
A Comparative Analysis of i-Ready, IXL, and Prodigy e-learning software programsA Comparative Analysis of i-Ready, IXL, and Prodigy e-learning software programs
A Comparative Analysis of i-Ready, IXL, and Prodigy e-learning software programsCristinaRyter
 
scopus journal.pdf
scopus journal.pdfscopus journal.pdf
scopus journal.pdfnareshkotra
 
Journal publications
Journal publicationsJournal publications
Journal publicationsSarita30844
 
The effectiveness of an enrichment program using dynamic geometry software in...
The effectiveness of an enrichment program using dynamic geometry software in...The effectiveness of an enrichment program using dynamic geometry software in...
The effectiveness of an enrichment program using dynamic geometry software in...Mohamed El-Demerdash
 
Technical writing report on teacher burnout and grading solutions
Technical writing report on teacher burnout and grading solutionsTechnical writing report on teacher burnout and grading solutions
Technical writing report on teacher burnout and grading solutionsCarlos
 
IRJET- A Conceptual Framework to Predict Academic Performance of Students usi...
IRJET- A Conceptual Framework to Predict Academic Performance of Students usi...IRJET- A Conceptual Framework to Predict Academic Performance of Students usi...
IRJET- A Conceptual Framework to Predict Academic Performance of Students usi...IRJET Journal
 
SSRN-id3333587.pdf
SSRN-id3333587.pdfSSRN-id3333587.pdf
SSRN-id3333587.pdfjesb3
 
Applicability of Extreme Programming In Educational Environment
Applicability of Extreme Programming In Educational EnvironmentApplicability of Extreme Programming In Educational Environment
Applicability of Extreme Programming In Educational EnvironmentCSCJournals
 
Munassir etec647 e presentation
Munassir etec647 e presentationMunassir etec647 e presentation
Munassir etec647 e presentationMunassir Alhamami
 
UNIVERSITY ADMISSION SYSTEMS USING DATA MINING TECHNIQUES TO PREDICT STUDENT ...
UNIVERSITY ADMISSION SYSTEMS USING DATA MINING TECHNIQUES TO PREDICT STUDENT ...UNIVERSITY ADMISSION SYSTEMS USING DATA MINING TECHNIQUES TO PREDICT STUDENT ...
UNIVERSITY ADMISSION SYSTEMS USING DATA MINING TECHNIQUES TO PREDICT STUDENT ...IRJET Journal
 
A COMPARATIVE ANALYSIS OF SELECTED STUDIES IN STUDENT PERFORMANCE PREDICTION
A COMPARATIVE ANALYSIS OF SELECTED STUDIES IN STUDENT PERFORMANCE PREDICTIONA COMPARATIVE ANALYSIS OF SELECTED STUDIES IN STUDENT PERFORMANCE PREDICTION
A COMPARATIVE ANALYSIS OF SELECTED STUDIES IN STUDENT PERFORMANCE PREDICTIONIJDKP
 
An Experimental Investigation Of Drill-And-Practice Mobile Apps And Young C...
An Experimental Investigation Of  Drill-And-Practice  Mobile Apps And Young C...An Experimental Investigation Of  Drill-And-Practice  Mobile Apps And Young C...
An Experimental Investigation Of Drill-And-Practice Mobile Apps And Young C...Audrey Britton
 
May 6633122 Researc & Evaluation- Gamification in Education.docx
May 6633122 Researc & Evaluation- Gamification in Education.docxMay 6633122 Researc & Evaluation- Gamification in Education.docx
May 6633122 Researc & Evaluation- Gamification in Education.docxMay Saeed
 
May 6633122 Researc & Evaluation- Gamification in Education.docx
May 6633122 Researc & Evaluation- Gamification in Education.docxMay 6633122 Researc & Evaluation- Gamification in Education.docx
May 6633122 Researc & Evaluation- Gamification in Education.docxMaySaeed5
 

Similaire à Ar3 X0 Final Research Paper G Compris Usability Research Report (20)

Clustering Students of Computer in Terms of Level of Programming
Clustering Students of Computer in Terms of Level of ProgrammingClustering Students of Computer in Terms of Level of Programming
Clustering Students of Computer in Terms of Level of Programming
 
Ar cgi don
Ar cgi donAr cgi don
Ar cgi don
 
A Comparative Analysis of i-Ready, IXL, and Prodigy e-learning software programs
A Comparative Analysis of i-Ready, IXL, and Prodigy e-learning software programsA Comparative Analysis of i-Ready, IXL, and Prodigy e-learning software programs
A Comparative Analysis of i-Ready, IXL, and Prodigy e-learning software programs
 
scopus journal.pdf
scopus journal.pdfscopus journal.pdf
scopus journal.pdf
 
Journal publications
Journal publicationsJournal publications
Journal publications
 
IJMERT.pdf
IJMERT.pdfIJMERT.pdf
IJMERT.pdf
 
The effectiveness of an enrichment program using dynamic geometry software in...
The effectiveness of an enrichment program using dynamic geometry software in...The effectiveness of an enrichment program using dynamic geometry software in...
The effectiveness of an enrichment program using dynamic geometry software in...
 
Technical writing report on teacher burnout and grading solutions
Technical writing report on teacher burnout and grading solutionsTechnical writing report on teacher burnout and grading solutions
Technical writing report on teacher burnout and grading solutions
 
Students’ Performance and Satisfaction with the Cisco Academy Networking Pr...
Students’ Performance and Satisfaction  with the Cisco Academy Networking  Pr...Students’ Performance and Satisfaction  with the Cisco Academy Networking  Pr...
Students’ Performance and Satisfaction with the Cisco Academy Networking Pr...
 
IRJET- A Conceptual Framework to Predict Academic Performance of Students usi...
IRJET- A Conceptual Framework to Predict Academic Performance of Students usi...IRJET- A Conceptual Framework to Predict Academic Performance of Students usi...
IRJET- A Conceptual Framework to Predict Academic Performance of Students usi...
 
ID Unit Report 1
ID Unit Report 1ID Unit Report 1
ID Unit Report 1
 
SSRN-id3333587.pdf
SSRN-id3333587.pdfSSRN-id3333587.pdf
SSRN-id3333587.pdf
 
Applicability of Extreme Programming In Educational Environment
Applicability of Extreme Programming In Educational EnvironmentApplicability of Extreme Programming In Educational Environment
Applicability of Extreme Programming In Educational Environment
 
Munassir etec647 e presentation
Munassir etec647 e presentationMunassir etec647 e presentation
Munassir etec647 e presentation
 
A450105.pdf
A450105.pdfA450105.pdf
A450105.pdf
 
UNIVERSITY ADMISSION SYSTEMS USING DATA MINING TECHNIQUES TO PREDICT STUDENT ...
UNIVERSITY ADMISSION SYSTEMS USING DATA MINING TECHNIQUES TO PREDICT STUDENT ...UNIVERSITY ADMISSION SYSTEMS USING DATA MINING TECHNIQUES TO PREDICT STUDENT ...
UNIVERSITY ADMISSION SYSTEMS USING DATA MINING TECHNIQUES TO PREDICT STUDENT ...
 
A COMPARATIVE ANALYSIS OF SELECTED STUDIES IN STUDENT PERFORMANCE PREDICTION
A COMPARATIVE ANALYSIS OF SELECTED STUDIES IN STUDENT PERFORMANCE PREDICTIONA COMPARATIVE ANALYSIS OF SELECTED STUDIES IN STUDENT PERFORMANCE PREDICTION
A COMPARATIVE ANALYSIS OF SELECTED STUDIES IN STUDENT PERFORMANCE PREDICTION
 
An Experimental Investigation Of Drill-And-Practice Mobile Apps And Young C...
An Experimental Investigation Of  Drill-And-Practice  Mobile Apps And Young C...An Experimental Investigation Of  Drill-And-Practice  Mobile Apps And Young C...
An Experimental Investigation Of Drill-And-Practice Mobile Apps And Young C...
 
May 6633122 Researc & Evaluation- Gamification in Education.docx
May 6633122 Researc & Evaluation- Gamification in Education.docxMay 6633122 Researc & Evaluation- Gamification in Education.docx
May 6633122 Researc & Evaluation- Gamification in Education.docx
 
May 6633122 Researc & Evaluation- Gamification in Education.docx
May 6633122 Researc & Evaluation- Gamification in Education.docxMay 6633122 Researc & Evaluation- Gamification in Education.docx
May 6633122 Researc & Evaluation- Gamification in Education.docx
 

Ar3 X0 Final Research Paper G Compris Usability Research Report

  • 1. GCompris Usability Testing 1 GCompris Usability Testing Research Report Shakarah McCrae - 06004107 AR3X0 – Research Paper May 15, 2009 Supervisor: David Bain Course Moderator: Dr. Ramsay Table of Contents
  • 2. GCompris Usability Testing 2 Executive Summary………………………………………………………………… ...2 Introduction…………………………………………………………………………… 3 Research Question…………………………………………………………......3 Null Hypothesis………………………………………………………………..3 Background…………………………………………………………………….3 Worth of Study………………………………………………………………....6 Objectives………………………………………………………………………6 Literature Review………………………………………………………………………8 Methodology…………………………………………………………………………...11 Procedure………………………………………………………………………12 Participant Summary…………………………………………………………..13 Gcompris Set-up……………………………………………………………….14 Data Presentation………………………………………………………………………16 Analysis………………………………………………………………………………..23 Conclusions and Recommendations ………………………………………………….25 Limitations…………………………………………………………………… References……………………………………………………………………………..27 Appendices………………………………………………………………………….....29 Executive Summary
  • 3. GCompris Usability Testing 3 This study is in line with the requirements of the University of the West Indies where a research project has to be done. This topic was chosen in accordance with my field of study and to discover how influential or effective this medium is in the classroom. Gcompris was chosen because it is free and can be used as a tool for teaching basic math to students. The research is aimed at showing how user friendly the software is and how the students will react to using it. Usability testing is used to measure the appropriateness of software for its user. It is aimed at assessing the effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in particular environments or contexts of use. The chosen open source software for this purpose is GCompris, which is an educational software suite comprising of numerous activities for children, aged 2 to 10. Some of the activities are game orientated, but nonetheless educational. More specifically, I will be looking at the extent to which G-Compris, as a tool for teaching basic mathematics skills can improve or aid learning in mathematics for grade one students at August Town Primary School. Introduction Research Question:
  • 4. GCompris Usability Testing 4 How effective is Gcompris- an open source software- for teaching basic mathematics skills to grade one students at August Town Primary school? Null Hypotheses: Gcompris as an educational tool is not beneficial in improving the performance of grade one (1) students in school. Background The disappointing performance of Jamaican students in the mathematics subject area continues to be an increasing concern to the wider Jamaican society. According to Francis (2006): “New statistics from the CXC show that of the 35,428 students who sat the 2006 mathematics exam, only 32 per cent managed to achieve grade one to three. This represents a decrease over last year's 36 per cent. This figure is also three per cent below the regional average” (para. 2). In fact, Wesley Barrett Pro-Registrar for the Caribbean Examination Council states quot;These results are disappointing and it is still a very worrying problem,quot; ” (Francis, 2006, para. 2). Poor mathematics performance is not only at the Caribbean Secondary Education Level but also at the Primary Level. With this in mind and out of the need to stem the problem from a tender age, I have decided to carry out a usability study of an open source software for teaching basic mathematics skills to grade one students in Jamaican primary schools. Usability testing “is a technique for
  • 5. GCompris Usability Testing 5 ensuring that the intended users of a system can carry out the intended tasks efficiently, effectively and satisfactorily.” (Gaffney, 1999) The chosen open source software for this purpose is GCompris, which is an educational software suite comprising of numerous activities for children, aged 2 to 10. Some of the activities are game orientated, but nonetheless educational. (http://gcompris.net/-en- )More specifically, I will be looking at the extent to which G-Compris, as a tool for teaching basic mathematics skills can improve or aid learning in mathematics for grade one students at August Town Primary school. Worth of Study The worth of this research is multi-facet in nature. If we can identify software that can aid and improve the performance of children from a tender age, we will not only be facilitating improvement in the grades but also a greater appreciation and a more fun way of learning the subject. It is of academic value because it provides information to educators and their stakeholders as to the value of educational software to students and the implications it can have. In addition, it provides valuable information to be taken into consideration by developers of educational software for students. Objectives of the study include: 1. To determine whether or not Gcompris is effective and appropriate for children six (6) and seven (7) years of age 2. To use the findings of this research to make recommendations to Educators as to one way of improving students performance in school
  • 6. GCompris Usability Testing 6 3. To prove whether or not Gcompris as an educational tool is beneficial to children 4. To evaluate students learning potential based on the activities available on GCompris 5. To document findings that are as a result of the usability research done. Dominant terms that will be used throughout this research are Usability, usability testing, educational, multimedia, multimedia cognitive theory, activity theory and children. There is an Independent Variable and a Dependent Variable. The independent variable is GCompris and the dependent variable is the August Town Primary Grade one (1) students. The main theories that will be guiding this research are the Multimedia Cognitive theory, Cognitive Load theory, The Multimedia Principle and the activity theory. For this testing, eighteen grade one primary student with little or no computer experience will be chosen randomly. The eighteen students will be further divided into two groups of nine and exposed to the software for 20 minutes on two separate occasion. The students will be observed, recorded, and a post-test evaluation will be done. This research report provides background, findings and recommendations of the Gcompris Usability testing. It documents and discusses theories and methodologies of the testing between March 17 and April 21, 2009. The report provides participant demographics, test evaluation methods and detail additional information that is necessary for this report. Our recommendations are based on the data-driven findings and the user-centered theory.
  • 7. GCompris Usability Testing 7 Literature Review According to Sashoi Green, teacher at Jamaica College (Clarke, 2009, para 6) quot;There is a phobia of mathematics, but when students relate it to things around them, they connect more.quot; In
  • 8. GCompris Usability Testing 8 a report in the Jamaica Gleaner, She said students develop a dislike for mathematics at an early age. (Clarke, 2009, para. 8). This line of thought is in keeping with the train of thought that exists. In fact, with the current low performance in mathematics within Jamaican Schools, new policies have been designed and researches are being conducted in an effort to stem the problem. Another perception of the cause of this low performance is provided by the Government of Jamaica who has admitted in a statement recently by Opposition spokesman on education, Senator Basil Waite that “Jamaican students are performing poorly in the area of mathematics because many primary school teachers did not pass the subject at high school level.” (Douglas, 2009, para. 1) This problem has led to developments, research and policies that have surfaced. One example of such is the New Horizons for Primary Schools (NHP), which was implemented by the United States Agency for International Development (USAID) in 1999 and lasted until 2004. “The objective of the program was to improve the language arts and mathematics performance of 72 of Jamaica’s poorer performing schools, through a school-based model of intervention.” (Marlaine, Harris, Gammill, & Barrow, 2006, p.1 ). “The NHP employs a holistic, student centre approach to improvement of instruction and learning in mathematics and reading (n.d.) Another current example is the National Mathematics Strategy being undertaken by the Education Transformation Team under the Ministry of Education. National Mathematics coordinator, Tamika Benjamin, says the launch of this strategy was the “beginning of a quot;long and difficult journeyquot;. (Anonymous, 2008, para. 9) “Through this strategy, direction will be given to stem the continued decline in student performance and to increase national awareness about the importance of
  • 9. GCompris Usability Testing 9 numeracy.”(Anonymous, 2008 ,Para. 2) the establishment of a National Mathematics and Numeracy Policy, which is aimed at improving the performance of students’ island wide. While there are many theories that speak to learning and the use of multimedia application to aid the learning, the theories I will be focusing on for the purpose of this research are The Cognitive Theory of Multimedia, Cognitive Load theory, The Multimedia Principle and the activity theory. According to Mayer & Moreno (2002) The Cognitive Theory of Multimedia states that multimedia narration and graphical images produce verbal and visual mental representations, which integrate with prior knowledge to construct new knowledge.(to cite). According to Mayer and Moreno (1998) and Mayer (2003), the Cognitive Theory of Multimedia Learning is based on several assumptions. First, working memory includes auditory and visual channels, which are equivocal to the phonological loop and the visu CognitiveTheory & Multimedia Instruction spatial sketch pad in Baddeley’s (1986) theory of working memory. Second, each subsystem of working memory has a limited capacity, consistent with Cognitive Load Theory (Sweller, 1988, 1994). Third, humans are knowledge-constructing processors who produce meaningful learning when they attend to relevant incoming information, organize the information in coherent representational structures, and then integrate it with other existing knowledge (Mayer, 1996, 1999). Fourth, connections can be made only if corresponding visual and verbal representations are in working memory at the same time, which is similar to Paivio’s (1986; Clark and Paivio, 1991) Dual Coding Theory.
  • 10. GCompris Usability Testing 10 “Cognitive Learning Theory is concerned with the way in which a learner’s cognitive resources are focused and used during learning and problem solving, suggesting that for instruction to be effective, care must be taken to design instruction in a way as to not overload the mind’s capacity for processing information.”(to cite) Multimedia Principle The multimedia principle states that better transfer occurs from animation/pictures and narration/ words than from words alone. When words and pictures are both presented, learners have the chance to construct verbal and visual cognitive representations and integrate them. Limitation to this study includes the fact that there have not been much recorded accounts of similar studies conducted. This limits the resources available to the researcher and the information for which to base the research on. The Activity Theory based usability testing methodology offers evaluators an insight into the natural context of use in an artificial laboratory setting which offers a high degree of control. (to cite) Methodology For the purpose of this research, the group that I have decided to study is grade one (1) students at the August Town Primary School. Eighteen students were randomly chosen, Nine from each of the two grade one students classes that were present. In an effort to effectively choose methods that would reflect the required data needed, two main usability evaluation methods based on Preece (1993), were considered and chosen. These are
  • 11. GCompris Usability Testing 11 observational evaluation and survey evaluation. “Observational evaluation implies collecting data that provide information about what users do when interacting with educational software.” Preece (1993) stated that two broad categories of data may be obtained: how users tackled the tasks given, where the major difficulties lie and what can be done; and performance measures like frequency of correct task completion, task timing, frequency of participant errors. “Surveys are employed to know users' opinions or to understand their preferences about an existing or potential product through the use of interviews or questionnaires.” The techniques that were used include direct observation, video recording, software logging, verbal protocols and Questionnaire administered by interviews. The Gcompris usability evaluation was conducted as monitored test sessions. Eighteen (18) students were tested. While this number is not representative of the general grade one population in Jamaica for a true experimental design, a minimum of 10 to 12 participants per condition must be used (Spyridakis, 1992). On the other hand, new research has shown that four to five participants can reveal up to 80% of usability insufficiencies in products, which often times tend to be most of the major problems (Virzi, 1990). The test sessions were held in a computer lab on the computer science department of the University of the West Indies, Mona campus. Procedure The eighteen students were divided into two groups of nine and the tests carried out in four sessions, each group being exposed to two. Each testing lasted for twenty minutes per session. From henceforth on, the two groups will be referred to as Group A and Group B. For the first session, Group A had three persons monitoring the test and Group B had five persons. This was due to the availability of persons to participate as monitors. The students were
  • 12. GCompris Usability Testing 12 transported from their school to the lab. On arrival, the participants were lead to the computer lab where there were each placed in front of a computer which had the software on screen. The students were then given one minute without instructions to see if they would know what to do. After the initial minute, the participants were first introduced to the computer keyboard aspect of the software where they were allowed to do one (1) activity of falling letters to allow them to familiarize themselves with the keyboard. After the basic introduction with the computer, the students were then exposed to basic addition and subtraction mathematics activities. All 9 students were exposed to the mouse where they were allowed to do an activity that required them using the mouse to click the correct answer. Two activities were chosen for this. 1. where they were allowed to connect the numbers to draw an object. This not only aided their mouse using skill but also tested their skills in counting. 2. Others were tested based on their ability to read and identify objects where they were shown different objects and they were required to click the letter that was missing. This first session was used as an introductory session. The students were monitored and comments and expressions were monitored. The second session however was of a different format, as the students were first exposed to mathematics addition activities for the first 10 minutes then subtraction activities for the other ten minutes. Their individual score was documented, after which a simple questionnaire was administered to the participants individually.
  • 13. GCompris Usability Testing 13 Participant Summary Eighteen participants participated in the Gcompris usability study. Participants were randomly selected based on their availability to participate in the study from the two grade one classes at August Town Primary. The selected participants were mixed in their performance in mathematics. That means some participants were exceptional performers, some needed extra help and the others were rather slow. The participants were each given permission letters to give to their parents/guardians, which were signed and returned to the researcher. Participant Age Group Sex Previous Computer Used other Previous Usage(1-Expert, 5- educational Gcompris Novice) software Usage 1 6 1 F 5 None None 2 6 1 M 5 None None 3 6 1 M 5 None None 4 7 1 F 5 None None 5 7 1 F 5 None None 6 6 1 F 5 None None 7 7 1 M 4 None None 8 7 1 M 5 None None 9 6 1 F 4 None None 10 7 2 F 5 None None 11 7 2 M 5 None None 12 6 2 M 4 None None 13 7 2 M 3 None None 14 6 2 M 5 None None 15 7 2 M 5 None None 16 6 2 F 5 None None
  • 14. GCompris Usability Testing 14 17 6 2 F 5 None None 18 6 2 F 5 None None Gcompris Set-up Gcompris was pre-installed for each participant on the linux operating system at the computer science computer lab. During the pretesting familiarizing section of the test, the participants were introduced to the welcome screen where no instruction was given at first. The participatory observation as an instrument of data collection is more appropriate in this case than other instruments that exist. In that , while dealing with the children it is much better to be able to interact with them while observation their reaction and behavior towards the software. Furthermore, this is best suited as a usability study consists of and generally involves measuring how well test subjects respond in four areas: efficiency, accuracy, recall, and emotional response, which will be useful in what we are trying to achieve. The aim of the evaluation was to validate the usability of the activities specifically the addition and subtraction mathematical activities which, are features of Gcompris. The testers were given activities to complete and their effectiveness, efficiency, reaction and satisfaction of the software were noted.
  • 15. GCompris Usability Testing 15 Data Presentation Of the eighteen participant respondents, nine were female and the remaining nine male. Giving a fifty-fifty percentage. However, group one had five females and four males while group two had four females and five females. The respondents are all between the age of six and seven. In fact of the ten of the participants are six years of age while the remaining eight are seven years of age. Usability Problems Number of Participants affected Time given for 18 activities Needs Supervision 14
  • 16. GCompris Usability Testing 16 Navigation 14 Participants were 18 unsure what to do Figure 1: Major Usability Problems and participants affected. Figure one shows the major usability problems and the number of participants affected by it. The Time given to complete the activities is at the top because it is the greatest concern among the participants. However, with time and with practice, the respondents will be allowed to complete the activities within the given time. Participant Age Group Average Activity Average Activity Score- Subtraction Score- Addition 1 6 1 8/10 7/10 2 6 1 5/10 3/10 3 6 1 7/10 8/10 4 7 1 3/10 3/10 5 7 1 6/10 5/10 6 6 1 5/10 6/10 7 7 1 9/10 8/10 8 7 1 10/10 9/10 9 6 1 7/10 5/10 10 7 2 3/10 1/10 11 7 2 2/10 2/10 12 6 2 4/10 3/10 13 7 2 3/10 2/10
  • 17. GCompris Usability Testing 17 14 6 2 8/10 7/10 15 7 2 6/10 5/10 16 6 2 4/10 3/10 17 6 2 5/10 5/10 18 6 2 6/10 6/10 Figure 2: Table showing average performance by the participants in the mathematics activities. Group one had a stronger set of participants when compared to the average score of group two. Although, there were a few strong participants in group two, majority of the participants required extra attention and was much slower in their performance. The list below represents some findings based on our participant responses to the feedback survey: • Of our 18 participants, 14 indicated that they had no previous computer exposure.
  • 18. GCompris Usability Testing 18 • Two of the four with previous experience said they had access to a computer at home. . • Of the 18 participants, none had previous exposure to Gcompris prior to the testing
  • 19. GCompris Usability Testing 19 10 out of the 18 participants suggested they would only use the software again if they had someone to help them as these students stated that the math activities were too hard. These 10 students happen to be the ones that scored the lowest on the activities. When asked why they would not want to use the software again, the most prominent answer was because they did not know how to do the math. However, some said they would use it again if they were supervised. Other Important Findings that came out of the Research 1. Participant 11 Boy is confused when it comes it comes to math, you have to completely break it down for him or work with him. He is easily distracted and works better on other activities 2. Participant #12 is unable to do basic addition, she is very slow and loses focus easily when she gets the answers incorrect. She s unable to recognize basic mathematics signs on her own.
  • 20. GCompris Usability Testing 20 3. Participant number 10 is very fast, she uses her fingers to assist her in solving problems. She works off her own initiative and is twice as fast as the others 4. Participant #1 likes math Likes to do it on the computer Likes the “thing” that she is using to do the math 5. Participant #7 likes subtraction -father encourages him to do math -likes the software -the math was too hard -wanted someone to help him because he did not know what to do -would not use it again because it was too hard. 6. Participant #5 – likes it because its nice Likes the game with the fish It was too hard because he can not do the math He would use it again if he had somebody to help him 7. Participant # 3 it was fun Fish game is her favourite
  • 21. GCompris Usability Testing 21 The math was not hard Yes would use it again Has a computer at home. 8. Participant #2 thought some of the math were hard, wanted someone to help with the ones that were hard, could do the math When asked if they would like the teacher to use it in class 90% said yes Activities the children liked 1. Letters that fall from the sky - 2 2. One participant response was her stomach hurts her when she does not know what to do. 3. Another one likes the addition Disapproval - The time was too short - They were not familiar with the layout of the computer keyboard It is also important to note that the age group used for this exercise was 6-7 so one might say that the level of work is very easy, but the ability of the students was mixed and so there were those who were performing at a lower level and those who were excelling. Nevertheless, the students of both mental capacities were able to use the software with relative ease and explanation at times.
  • 22. GCompris Usability Testing 22 Analysis From the test conducted and from general observations and posttest evaluation, various findings from which generalizations can be made were found. With reference to the effectiveness of the GCompris it was found out that although the majority of the participants enjoyed playing the other activities they did not like the math activities that were available for more than one reasons, time availability and their ability perform well at the activities. Although the math questions being asked in the activities were appropriate, 80% of the participants felt that they
  • 23. GCompris Usability Testing 23 needed to be guided when participating in the activities. This is due to their ability to calculate and function on their own as these participants were not as academically inclined as the others. While 50% of the participants did not have a problem with the mathematical activities, the time issues was of a major concern to these participants as they used their fingers to help them calculate. However based on the researcher’s observations, the participants will eventually perform better once they are exposed to the software and general computer usage skills on a weekly basis. Twenty percent of the participants managed the mathematical program very well because they were able to apply the skills taught in basic computation to those that were being tested on the software. For example, one student was given a computation of 10-6=___, immediately she held up her ten fingers and subtracted 6 and was able to reel the answer (4) in seconds. Another option was the reading skills which asked them to fill in the missing letters. This was done with ease as well because the students already had a grasp of the words and so identifying the missing letters was pretty easy. Owing to the fact that they were using the computers for the first time the difficulty came when they had to identify the keys on the keyboard in order to get the right answer. It is also important to note that the age group used for this exercise was 6-7 so one might say that the level of work is very easy, but the ability of the students was mixed and so there were those who were performing at a lower level and those who were excelling. Nevertheless, the students of both mental capacities were able to use the software with relative ease and explanation at times.
  • 24. GCompris Usability Testing 24 Overall, the participants were enthusiastic about the software and the activities available on the software. They enjoyed playing the various games. Conclusions and Recommendations From this usability testing research on the extent to which G-Compris, as a tool for teaching basic mathematics skills can improve or aid learning in mathematics for grade one students at August Town Primary school, numerous recommendations can be made. Some of these recommendations have been put forward by monitors and participants themselves and the others will be put forward by the researcher based on the process and findings of the research. Recommendations put forward by the monitors/evaluators/participants include
  • 25. GCompris Usability Testing 25 o The length of the time on the software needs to be slower which will seek to accommodate students that have learning disabilities and are not so quick. o Students on initial exposure to the software have to be closely monitored in order to fully utilise and captivate the benefits that can be had from the software. Recommendations put forward by the researcher include: • A comparative study to be done between a preparatory and a primary school. • More recorded research including that of usability testing should be done in the Caribbean so that Caribbean researchers including students may have readily available literature to them when conducting research of this nature and others. • The developers of children software needs to take into consideration children who are not tech-savvy hence the researcher is recommending that the developers of GCompris allot more time to the completion of the initial activities in order to allow users to get familiar and comfortable with the product. • A computer lab should be made available to the students with academic software such as Gcompris installed so as to exposure them to alternative methods of learning which will be beneficial to the teachers and students alike. • The Ministry of Education should re-visit the syllabus In conclusion, the researcher discovered that while GCompris can indeed improve or aid the performance of students in mathematics, the timing available for completion of the activities needs to be revisited for students that are not that academically inclined and needs more time to work on the activities. The fact that the most of the participants who were all unfamiliar with GCompris could learn to use it fairly with just two exposures suggests that GCompris is generally usable for the intended target population. However the developers of the software
  • 26. GCompris Usability Testing 26 needs to look in lengthening the time given for completion of each activity as this present timing is not conducive or receptive of slow learners.
  • 27. GCompris Usability Testing 27 References Clarke, O (2009, April 18) Curing the math phobia - Math competition shows problem solving to be a good tool. Retrieved January 27, 2009 from http://www.jamaica-gleaner.com/gleaner/20090418/lead/lead4.html Douglas, L (2009, April 5) 'WE HAVE A MATH PROBLEM'. Retrieved January 27, 2009 from http://www.jamaicaobserver.com/magazines/Career/html/20090404T220000-0500_1488 20_OBS__WE_HAVE_A_MATH_PROBLEM__.asp Francis, P (2006, September 15) Poor CXC results - Jamaican students lag behind region in Maths, English. Retrieved January 26, 2009 from http://www.jamaica-gleaner.com/gleaner/20060915/lead/lead1.html Gaffney, G (1999) Information & Design: Usability Testing Series. Retrieved January 26, 2009 from http://www.infodesign.com.au/ftp/UsabilityTesting.pdf Marlaine, L., Harris, A, Gammill, P & Barrow, K (2006) IMPACT OF NEW HORIZONS FOR PRIMARY SCHOOLS ON LITERACY AND NUMERACY IN JAMAICA 1999-2004. Retrieved on February 5, 2008 From http://www.equip123.net/JEID/articles/2/NewHorizons.pdf Preece, J. (1993). A Guide to Usability: human factors in computing. Addison Wesley, the Open University.
  • 28. GCompris Usability Testing 28 Rubin, Jeffrey. (1994). Handbook of Usability Testing: How to Plan, Design and Conduct Effective Tests. New York: Wileyl Spyridakis, J. H. (1992). Conducting Research in Technical Communication: the application of true experimental designs. Technical Communications, Fourth Quarter, 607-624. Virzi, R. A. (1990). Streamling in the Design Process: Running Fewer Subjects. Proceedings of the Human Factors Society, 291-294. Educators count on new maths strategy (2008, June 4) Retrieved on January 27, 2009 from http://www.jamaica-gleaner.com/gleaner/20080604/news/news3.htmlv Mayer, R. E. (1996). Learning strategies for making sense out of expository text: The SOI model for guiding three cognitive processes in knowledge construction. Educational Psychology Review, 8, 357-371. Multimedia in Jamaican Schools retrieved on January 31, 2009 from http://www.iicd.org/articles/IICDnews.import42/ http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/36/25/e2. pdf http://expanding-educational-horizons.com/Numeracy/NumeracyPolicy.pdf
  • 29. GCompris Usability Testing 29 Appendices Caribbean Institute of MEDIA and COMMUNICATION The University of the West Indies 3 Sherlock Drive, Mona Campus Kingston 7 Jamaica, W.I Tel. (876) 927-1481, 977-2111, 977-0898, 977-1398, 977-9990, 702-3353, Fax (876) 977-1597 E-mail carimac@uwimona.edu.jm Caribbean Institute of MEDIA and COMMUNICATION Web www.mona.uwi.edu/carimac Dear Parent/Guardian, Re: Permission for your Child/Ward to participate in a computer study This letter seeks permission for your child/ward to participate in a multimedia computer study to be conducted at the University of the West Indies Mona Campus. Although we would have preferred to conduct the study on the premises of the school, the computer lab is currently unavailable for use, hence the need for the study to be done on UWI’s campus. The project will run for 2 to 3 weeks commencing on Tuesday March 17, 2009. It will require that your child visits the UWI computer lab on two occasions within one of the following sessions (Tuesdays and Thursdays 9:00 am – 10:30 am) during duration of the study. Shakarah McCrae, a final year student at the Caribbean Institute of Media and Communication (CARIMAC), will conduct the study, which is intended to fulfil her research requirement to investigate multimedia software and children in the classroom. Your child will be required to use a computer with Mathematics learning software for a period of 20 to 30 minutes. Sincerely, ----------------------------------------
  • 30. GCompris Usability Testing 30 David Bain Multimedia Co-ordinator CARIMAC, UWI Mona ----------------------------------------- Shakarah McCrae Final Year Student CARIMAC UWI Mona ----------------------------------------- Morlton Wilson Principal, August Town Primary School -------------------------------------------------------------------------------------------------------------------- - I ____________________________________ have given permission for Parent/Guardian __________________________ my child/ward to participate in the multimedia computer study to be held between March 17, 2009 and March 26, 2009 at the UWI campus.
  • 31. GCompris Usability Testing 31 Screenshots showing sample activities played by participants Addition
  • 32. GCompris Usability Testing 32 Connect the dots
  • 33. GCompris Usability Testing 33 Identify the Letters Guess the object
  • 34. GCompris Usability Testing 34 Subtraction