If one tries to meet the ever growing demand to replace the cumbersome exams on paper with a digital version, the obvious choice to use the assessment tools of the virtual learning environment (VLE) may not be as obvious as one might think. The modern VLE’s are increasingly ‘open’, ‘searchable’ and ‘connected’ with the outside world AKA The Cloud. On the other hand ‘restriction’ or ‘control’ is a key concept for exams. First of all for students, control means: a simple workflow, so that the sole focus can be on answering the questions. And for staff, this means control the access to the exam ‘room’, restrict or block access to documentation sources (including the ‘regular’ VLE content ), monitor the progress of students and detect impostors. At KU Leuven, we have attempted to impose this level of control to our Blackboard Learn deployment by using some simple tools. Our solution was dubbed Texas (Toledo EXams Advanced System) as a working title and is a combination of front-end (Javascript, CSS) and backend (Java) technologies based on opensource libraries and the Blackboard API’s. First of all it compromises a dedicated theme, which among others strips down the course menu to the bare minimum, and simplifies the process of taking assessments. Also, we built a ‘dashboard’ tool for staff to control the access of students to the exam, monitor student progress and to detect impostors based on IP addresses, login times, et cetera.

1. Texas Rangers to the Rescue:
turning your VLE into an exam
centre
Sven Gladines & Herwig Stulens
KU Leuven - Belgium

2. Overview
2
• Introduction
• What's this presentation about?
• Why a Blackboard exam setup?
• Requirements instructor & student view
• Under the hood
• Live demo

3. Texas Rangers
3
Sven Gladines
• KU Leuven since 2008
• Software architect
• Java developer
o B2 development
o SIS/Campus integration

4. Herwig Stulens
• KU Leuven since 2008
• Business consultant
(customer support Bb & QMP)
Texas Rangers
4

5. KU Leuven & Toledo
5
• Self-hosted since 2001
• Learn April 2014 release
• 10 app-server (RHEL 6/HP)
• 2 CPU / 48 GB RAM
• 1 DB-server (Oracle/HP)
• 4 CPU
• Storage
• SAN / NetApp cluster
• Netscaler

6. KU Leuven & Toledo
6
• 6.5 (Java) developers
• 3.5 Bb-customer support
• 1 MOOC @ edX
(e-psychology)
• KU Leuven Association
• 1 university
• ±13 colleges
• + adult/evening education
• 38K active courses
• 9K active community’s
• 121K "active" users

7. Toledo
Exams
Advanced
System
Change the interface
and the behaviour of your
Blackboard system for
specific use cases (mobile,
exams,...)
7

8. High Level - How it works?
8
1. Create a new entrance to your Bb-system, f.i. x.blackboard
2. Associate a specific CSS & javascript (js) with this entrance
All x.blackboard pages will have their own stylesheet and
specific behaviour
(For more details, see further "Under the hood")

9. Example 1
9
Netscaler
url:
• The same environment
• Different requirements
• Different layout (CSS)
• Different behaviour (js)
https://blackboard.kuleuven.be

10. Example 2
10
Netscaler
• The same environment
• Different requirements
• Different layout (CSS)
• Different behaviour (js)
url: https://m.blackboard.kuleuven.be

11. Example 3
11
Netscaler
• The same environment
• Different requirements
• Different layout (CSS)
• Different behaviour (js)
url: https://x.blackboard.kuleuven.be

12. Background - Why an exam setup?
12
• In Belgium, an instructor organises exams him/herself
Online or traditional is instructor's choice
• General trend towards (automatically scored) online exams
Why?
• Reuse of formative test material
• Growing number of legal disputes
( shift to less questionable multiple choice-tests)

13. • Student still has access to course material, personal CMS
space, file exchange (group), collaborate, …
• "Accidents" with course setup: test feedback, my grades,
material remains available after exam, backup, ...
• Hard to inform everybody about possible pitfalls
• Standard Bb offers too many features
Online?
Alright, let's use
our Blackboard Possible problems
13

14. Requirements
student side
In the context of online
examinations, which changes
are necessary?
(cfr. standard Bb-functionality)
14

15. 1. Extremely large top bar, brightly coloured with clear
indication of student name, number and picture.
(important for authentication & for proctor surveillance)
Portal page
15

16. 2. Only 1 module is present: My Exams
i.e. list of x-courses with available membership (mostly only 1)
(for closed exams, all student memberships are set to unavailable)
Portal page
16
(x-course = exam course)

17. 3. Disclaimer:
• avoid "unintended" access to an exam
• inform student about logging
• inform how to finalize?
(Accepting the disclaimer , generates a course entry record;
see further in "under the hood")
4. Bilingual instructions
& warnings
Entry to the exam
17

18. 5. Entry point = Read me first instructions
The x-course
6. Simplified interface:
• no tabs
• no logout button
• ...
18

19. 7. Make an official submission
Finalize
19

20. 8. After submission, the x-course membership
becomes unavailable student no longer
has access to the x-course
Finalize
9. Submission receipt
is mailed
20

21. Requirements
instructor side
21
In the context of online
examinations, which changes
are necessary?
(cfr. standard Bb-functionality)

22. Requirements instructor side
22
1. Specific documentation focussed on online examination:
• preparations to do beforehand
• preferred test settings
• decimal/thousand separator
• testing with the preview user
• backup scenario
• ...

23. • Instructors have an extra module to create x-courses
• All x-courses are based on a highly tuned template course
Simplify
23

24. All the documentation is part of
the x-course
What to do before, during and after the exam?
Info about the student point of view…
Simplify
24

25. Requirements instructor side
25
2. Make it easier, hide all unnecessary tools:
no fora, no wiki's...

26. There are only 2 tools available: contacts and groups, no
my grades, no email...
Simplify
Only 4 buttons to add
content
26

27. Requirements instructor side
27
3. Make it safer:
• tweak test settings
o no force completion
o no self-assessment options...
• assignment behavior (turn off Crocodoc)
• availability grade center columns, turn off my grades tool, ...

28. Why exclude Crocodoc service?
• hard to exclude (pc room)
• university policy about distribution of exam material
Assignments
28

29. Requirements instructor side
29
4. Enhance tracking: keep a finger on the pulse with an online
dashboard:
• status
• (multiple) start time(s)
• end time
• (multiple) IP address(es)
• other Bb-sessions?
• exam progress (in development)
• group membership

30. 30

31. Under the hood
31

32. The address bar
32
SIMPLE URL, but distinguished from VLE
• VLE:
• blackboard.kuleuven.be
• Exam:
• x.blackboard.kuleuven.be
• Add DNS entry
• Add new host to HTTPS CERTIFICATE
url: https://x.blackboard.kuleuven.be

33. Login
33
SEPARATE LOGIN
SHIBBOLETH CONFIGURATION
• Add "application"
• application bound to "x." hostname
login

34. LOOK & FEEL
34
? SPECIFIC LOOK & FEEL
Blackboard Learn’s "Brands & Themes"
• CREATE BRAND
• Bind it to hostname "x.blackboard.kuleuven.be"
• CREATE THEME
• Assign theme to brand
• theme.css
• theme.js
login

35. • SPECIFIC COLOR PALETTE
• High(er) contrast
• Soft Colors
• SIMPLE INTERFACE
• Simple widgets
• Reduce "cognitive overhead" : hide stuff that isn’t needed
• VISIBLE INTERFACE (Surveilling)
• Very large top bar
• Large names and account ID
• Picture (via B2, Bb avatar not used)
THEME.CSS
35

36. THEME.JS
36
• FURTHER REDUCE COMPLEXITY
• Hide unnecessary elements : Remove from DOM tree
• SUPPORT REQUIREMENTS FOR ENVIRONMENT
• Show disclaimer upon entry ("I have read and understood")
• Inject "Finalize"-button in x-course menu
• Blocking of certain URL’s
• …
• To use this, we altered Blackboard’s page template
• Inject reference to theme.js in HTML <head> tag

37. PORTAL PAGE
37
PORTAL PAGE PER "ENVIRONMENT"
• LOADBALANCER (NETSCALER) REDIRECT RULES
If x.blackboard.kuleuven.be and not logged in, then:
• redirect to: /webapps/portal/…/tabs/tabAction?tab_tab_group_id=_313_1
If m.blackboard.kuleuven.be and not logged in, then:
• Redirect to /webapps/portal/…/tabs/tabAction?tab_tab_group_id=_230_1

38. PORTAL PAGE EXAMS
38
EXTREMELY SIMPLE PORTAL PAGE
• 1 module for students ("My Exams")
• 2 modules for instructors ("My Exams" & "Create New Exam")
• Portal page depending on "environment"
• Blackboard’s TAB GROUP & TAB
• MODULES
• "My Exams" for everyone
• "Create New Exam" only for specific institution role "xcourse-admin"

39. PROCTORING & TRACKING
39
? ENABLE TRACKING & PROCTORING
• Timing: start time, end time
• IP address
• Access to learning environment
• Student progress
• WEB SERVICE (BUILDING BLOCK)
• Collect data & log events
• DASHBOARD
• Visualize data at a glance

40. AND MORE …
40
? SUPPORT SPECIFIC REQUIREMENTS
• Send confirmation email upon finalisation
• Automatically block access upon finalisation
• Make x-course unavailable when last student has finished
• Simplify access control for proctors
• …
• WEB SERVICE
• Business logic & feed back to dashboard
• DASHBOARD
• Dynamic buttons for block/allow

41. WEB SERVICE
41
REST
DB
Java
• Building block
• JAVA 7
• Third party libraries: Spring, Hibernate, …
• Blackboard API’s
• Toledo dependencies: utility libraries, API’s …
• 2 DB tables
• Tracking data
• Events log
• REST API
• AJAX (get/post/put)
• JSON objects
• Decoupling: web frontend - Java backend
• Built with Maven 2

42. DASHBOARD
42
Javascript
Ajax
HTML
• Building Block
• HTML 5
• Twitter Bootstrap (Web GUI framework)
• Awesome Font (fonts & icons)
• Javascript
• jQuery
• Underscore.js (JS utilities)
• Backbone.js (Ajax framework)
• Marionette.js
• Moment.js ( date/time )
• Noty ( notifications )
• Built with GRUNT

43. 43
Time for demo
1. Surf to:
http://bit.ly/1Oe21AM
(change name of demo user & submit)
2. Next go to:
toledo.kuleuven.be/english/exam
(wait a few seconds and continue)

44. 44
Questions?
please go ahead
or
send us an email at toledo@kuleuven.be

45. 45