Improving the effectiveness of information security learning programs

Improving Delivery Effectiveness of Information Security Learning Continuum
Mansoor Faridi
Fort Hays State University
July 28, 2015
Author Note
Mansoor Faridi, Department of Informatics, Fort Hays State University.
Mansoor Faridi is a graduate student at Fort Hays State University specializing in
Information Assurance Management. He lives in Toronto and can be contacted at
[m_faridi@mail.fhsu.edu].

Improving Delivery Effectiveness of Information Security Learning Continuum ii
Table of Contents
Abstract .......................................................................................................................................1
Introduction ..................................................................................................................................2
Components of Information Security Learning Continuum ........................................................3
Awareness …………………………………………………………………....................3
Education ………………………………………………………………………. ............3
Training ………………………………………………………………………................4
Critical Success Factors ...............................................................................................................5
People ……………………………………………………………...................................6
Process ……………………………………………………………. ................................7
Technology ……………………………………………………………. .........................7
Improving Effectiveness ...............................................................................................................7
Baselining Pre-training Results ........................................................................................8
Continuous Improvement .................................................................................................9
Rebaselining Post-training Results ..................................................................................9
Shortcomings and Best Practices .....................................................................................9
Conclusion ................................................................................................................................10
References ..................................................................................................................................11

Improving Delivery Effectiveness of Information Security Learning Continuum 1
Abstract
Users in all organizations globally are either the strongest or the weakest link, when it comes to
ensuring confidentiality, integrity, and availability of critical data. Various organizations design,
develop, and implement information security learning programs, however, effectiveness of their
implementation levels vary owing to a variety of factors.
This research paper proposes a model to improve delivery effectiveness of information
security learning continuum. The research is aimed at identification, analysis, and evaluation of
the essential ingredients required by this learning model, such as, a detailed methodology, critical
success factors, and organizational best practices. The success of this model lies by being
dynamic in nature; its continuous feedback collection mechanism is aimed at finding
efficiencies, and incorporating those efficiencies (on a continuing basis), to ultimately improve
the delivery of organizational learning activities.
Among the numerous best practices, developing and quantifying metrics is paramount to
the success delivery of the information security learning program, and continuous improvements
(based on the collected feedback) to the continuum is the key to successful program delivery.
Keywords: information security awareness, information security governance, information
security education, continuous improvement

Mansoor Faridi
Fort Hays State University
Introduction
security learning continuum. The research is aimed at identification, analysis, and evaluation of
the essential ingredients required by this learning model, such as, a detailed methodology, critical
success factors, and organizational best practices. The success of this model lies by being
dynamic in nature; its continuous feedback collection mechanism is aimed at finding
efficiencies, and incorporating those efficiencies (on a continuing basis), to ultimately improve
the delivery of organizational learning activities.
Components of Information Security Learning Continuum section describes the three
essential components of information security learning continuum, including awareness,
education, and training.
Critical Success Factors section established people, process, and technology and their
overlap to produce the sweet-spot which helps establish critical success factors for improving the
delivery effectiveness of information security learning continuum.
Improving Effectiveness section delves into the details for improving the effectiveness of
information security learning continuum through baselining, engaging in continuous
improvement activities (based on the results of which), and rebaselining the learning program. It
concludes by presenting a list of shortcomings and best practices to address those shortcomings.
Conclusion section presents a summarized conclusion of this report while highlighting
the importance and relevance of this topic.

Components of Information Security Learning Continuum
Information Security (or InfoSec) is the practice of ensuring confidentiality, integrity, and
availability of data from unauthorized access. In order to improve the effectiveness of an
organization’s information security, the quality of education, awareness, and learning activities
should be designed and developed with due care to improve its delivery effectiveness.
In most organizations, Information Security learning activities comprise of awareness,
education, and training in some shape or form. All three elements entail both formal and
informal activities that are discussed below in more detail. It is important that all three stages are
designed and developed by a qualified professional with an intimate familiarity with the nuances
of adult education. Most common dominant learning styles (visual vs. auditory) should be kept in
view when designing the learning activities. In addition, it has been proven that adults learn more
effectively by performing (and discovering) the task at hand in social settings, hence these
known trends need to be incorporated for fun learning experience (Michigan, 2015).
Awareness
This component is the most important of all (others being Education and Training), as
this is the starting point where users attention is focused on security issues, their
acknowledgement of security issues. At this stage, users are normally the recipients of
information, and do not actively participate (NIST, 1998, p.15). Aids used in awareness
campaigns depend on the scope, breadth, and budget; however, the common items include
newsletters, posters, brochures, flyers, videos, promotional slogans, trinkets, mouse-pads, etc. An
effective awareness campaign will stress the ever-changing threat landscape, identify threat-
vectors, and demand timely adjustments to the awareness contents being delivered
Education

After awareness comes Education. At this stage, users are aware about the security issues
that exist and are looking forward to educate themselves. This stage integrates all of the security
skills and competencies of the various functional specialties into a common body of knowledge.
It also adds a multi-disciplinary study of concepts, issues, and principles (technological and
social).This stage strives to produce users capable of recognizing the threats and being proactive
in their response (NIST, 1998, p.16). An important characteristic of education is that users must
understand why information security is important for the organization (Schlienger & Teufel,
2003).
Training
This is the third and final stage in the learning life cycle. By this time, the users have
been educated on the security issues and now they are ready to get trained on how to behave
securely in the information security context. This level strives to produce relevant and needed
security skills and competency by practitioners of functional specialties other than IT security
(e.g., management, auditing). Training of special security tools (or features) within applications
must be also be offered (NIST, 1998; Schlienger & Teufel, 2003).
Another important aspect of these learning programs is the adoption of a multi-level
approach vis-à-vis test design. For example, users should only be asked to recall, recognize,
and/or understand information security concepts at this initial level (or Primary State). For
example, confidentiality, integrity, availability, and non-repudiation.
The next intermediate level (or Secondary State) of learning should test users’ ability to
apply the learned concepts to real-life situations, to enhance their understanding of the issues at
hand. For example, identity and access management workflows, data retention issues, evolving
threat vectors, need for data quarantine and sanitization, etc.

The advanced level (or Target State) of testing should encourage users to synthesize
learning in order to analyze and interpret real-life information security situations, and draw
meaningful conclusions. This also helps users become proactive participants by supporting
organizational security initiatives, and raise flag in case of any abnormal online activity.
Users having attained the Target State will seek knowledge proactively. This target level
of expertise goes way beyond exploring basic information security concepts, and should be the
ultimate sweet-spot that trainers should aim for when designing test exercises.
Critical Success Factors
The integration of people, process, and technology entities form an important troika; an
overlap of which leads to the creation of critical success factors (See Figure 1 below). All three
elements entail both formal and informal activities necessary for effective implementation of the
learning program. Each entity represents various essential components, discussed below in more
detail.
Figure 1. Troika – People, Process, Technology

People
First and foremost, effective implementation of information security learning program
requires executive sponsorship to set the ‘tone-from-top’, which helps secure the required
resources, and highlights the importance of this important initiative. Executive sponsors can also
influence their counterparts in ensuring that the message is received positively across the
organization.
While Executive sponsorship is a must-have, however, the delegation of sponsorship at a
local level (e.g. local Business Unit Champion) does wonders. It is important that this local
sponsor be at the management level with a good amount of influence.
Secondly, users are always deemed to be the weakest link. However, it is important for
individual users to buy-in to the idea, realize the importance of this mission-critical initiative,
and be able to view themselves as an empowered user that makes a significant difference,
protecting the organization’s critical assets, on a daily basis.
Users should be sent short quizzes over time. The responses, both correct and incorrect,
are a gold-mine of information to identify users' understanding of various information security
issues, and to reinforce concepts which most users failed to fully comprehend.
Unannounced drills, such as, planned fishing attacks in coordination with IT should be
executed (and data collected) to determine level of readiness and by analyzing the number of
users who fell prey to such attacks. This data will help remediate the understanding of
information security concepts, and reinforce those concepts as well.
Finally, subject matter experts (SMEs) delivering the program play an important role in
delivering relevant, appropriate, and engaging contents, to produce a well-informed class of

users. It is paramount to select SMEs with the right qualifications, most importantly with
superior communications skills to deliver an effective learning experience.
Process
This entails formalization of policies, procedures, and standards, while defining metrics,
measurements and feedback mechanism in order to integrate the overall learning program. An
important aspect of this component is the sharing of knowledge and information via an internally
shared repository. Various aspects defined here will be discussed in further detail in later
sections.
Technology
Various technologies can be leveraged, suiting the size of organization. A small
organization may want to measure and report manually, whereas, an enterprise-level
organization may choose to automate the entire process, end-to-end.
Regardless of the size, organizations should have tools to record, measure, and report on
metrics, such as, non-compliances, course completion statistics, and continuous monitoring (e.g.
accessing in-appropriate web-sites) of users' online activities. Technology should also be
leveraged to solicit user-feedback on various issues, and to share knowledge and information via
online spaces (e.g. Wikis, SharePoint, intranet, etc.). With the aid of Active Directory
authentication, technology should also help with Role-based Access (RBAC) Controls,
segregation of duties, least privilege, need to know, limited time access to only let authorized
users in.
Improving Effectiveness
Figure 2 (below) represents information security learning continuum, which
conceptualizes a proposed model to baseline, monitor, improve, and re-baseline the program on a

continuing basis. According to this model, a gap assessment should be performed to compare
current state with desired future state. This target setting promotes competition, while serving as
a roadmap towards the final destination (i.e. Target state). This model also requires
quantification of the time horizon to set milestones and deliverable, and metric definition to
baseline against.
Figure 2. Information security learning continuum
Baselining (Pre-training Results)
Next step is to consolidate and baseline in-scope organizational metrics. To perform this,
current measurements need to be recorded. This starting point serves as an indicator throughout
the learning continuum vis-à-vis organization current state, and the remaining ‘distance’ to the
target state. It is recommended that half-way through the journey, feedback is formally solicited
from all stakeholders, in addition to the measurements obtained for the pre-defined metrics. This
step helps in determining if any changes/modifications are warranted to any part of the process
and/or the overall learning program.

It is recommended that, half-way through the journey, user-feedback is formally solicited
from all stakeholders, in addition to the measurements obtained for the pre-defined metrics
(Greaux, 2013). This step helps in determining if any changes/modifications are warranted to any
part of the process. Some of the suggested metrics are as follows:
Table 1. Metrics and their rationale
Metric Data Collected & Reviewed
Use engagement
Successfully reaching out to all uses and the rate of completion of all education,
training, and awareness activities as they are rolled out during the course of a year.
Quality of responses
It is important to identify wrong responses for all learning activities, and then draw out
trends for subsequent analysis. This enables developers identify user strengths, and
also identify areas that require further emphasis to readily address knowledge gaps.
Security breaches (internal)
Internal security breaches should be recorded for later root cause analysis. This will
serve as an input when designing learning activities.
Periodic testing
Data from testing activities (e.g. internally generated fishing emails) should be
analyzed to gauge users’ knowledge level vis-à-vis InfoSec issues.
Continuous Improvement
After baselining, the program needs the continuously monitored and improved. Input can
be in the form of automated monitoring, user feedback, process change requests, etc. Refer to
Figure 2 for mechanisms in place vis-à-vis feedback, process change requests, etc.
Re-baselining (Post-Training Results)
After formal training delivery, measurements need to be taken again, which should be
compared against the initial readings taken when baselining. The delta between the two will help
determine the level of implementation effectiveness of the overall program, while identifying
specific opportunities for improvements.
Shortcomings and Best Practices
Following table (Table 2) lists some reasons why information security controls fail
(SANS, 2015; Thacker, 2013; Winkler & Manke, 2013) and the best practices that can be
developed and implemented to address these shortcomings.

Table 2. Reasons for shortcomings and best practices
Reasons Shortcomings and Best Practices
Lack of user
awareness
Shortcoming: Simple ‘box-checking’ without understanding the concepts hinders the spirit of
defenses.
Best practice: Different learning activities can help raise user’s awareness level.
Lack of
engagement
Shortcoming: Users are provided with literature, but not tested formally.
Best practice: Users should complete mandatory learning activities to ascertain their
knowledge levels via testing activities.
Operating
without metrics
Shortcoming: In the absence of metrics (quantification), it is impossible to determine if learning
activities are being rolled out, completed, shortcomings being identified, and addressed.
Best Practice: Designing and implement appropriate metrics to quantify activities.
Misplaced
accountabilities
Shortcoming: Business often relinquishes data protection aspects to their IT function, including
governance and oversight.
Best Practice: Data owners (business) need to be continuously involved in all aspects of data
protection, in conjunction with IT. They need to take the ownership of their data, and clearly
understand IT function as mere custodian of their data.
Conclusion
security learning continuum. It presents three essential components of information security
learning continuum, including awareness, education, and training. The troika of people, process,
and technology is established as the required component to improve delivery effectiveness of
information security learning continuum. This is achieved by baselining, continuous
improvement, and rebaselining the learning program. Finally, some shortcomings that hinder the
successful implementation are highlighted and suggested best practices are listed to address those
shortcomings.
With proper awareness, users can be the strongest defense, supporting the overall
delivery effectiveness of information security learning continuum; leading the paradigm shift
from static to dynamic mode of learning.

References
Ashford, W. (February 13, 2015). Data breaches up by 49% in 2014. ComputerWeekly.com.
Retrieved from http://www.computerweekly.com/news/2240240346/Data-breaches
-up-49-in-2014-exposing-more-than-a-billion-records
Awan, I. (2014). Debating the term cyber-terrorism: Issues and problems. Internet Journal of
Criminology. Retrieved from http://www.internetjournalofcriminology.com/Awan_
Debating_The_Term_Cyber-Terrorism_IJC_Jan_2014.pdf
Council of Europe. (2015). Standards: the convention and its protocol. Retrieved from
http://www.coe.int/t/DGHL/cooperation/economiccrime/cybercrime/default_en.asp
Cyberwarfare. (2015). In Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Cyberwarfare
Cyberwarfare In the United States. (2015). In Wikipedia. Retrieved from http://en.wikipedia.org/
wiki/Cyberwarfare_in_the_United_States
Defence IQ. (2010, May 26). CIA, US military step up cyber space security strategies.
Retrieved from http://www.defenceiq.com/defence-technology/articles/cia-us-military-
step-up-cyber-space-security-strat/
Feldman, N. (2015). Brainy quote. Retrieved from http://www.brainyquote.com/
quotes/keywords/cyber.html
Glennon, M. (2013). The dark future of international cybersecurity regulation. Journal of
National Security Law & Policy, 4, 563-570. Retrieved from http://jnslp.com/wp-c
ontent/uploads/2013/04/The-Dark-Future-of-International-Cybersecurity-Regulation.pdf
Greaux, S. (October 15, 2013). Use metrics to measure and improve security awareness.
PHISHME. Retrieved from http://phishme.com/use-metrics-measure-improve-
effectiveness-security-awareness/

Hathaway, O., Crootof, R., Levitz, P., Proctor, H., Nowlan, E., Perdue, W., Spiegel, J. (2011).
The law of cyber-attack. Yale Law & Economics Research Paper No. 453, 100 (4), 1-76.
Retrieved from http://www.law.yale.edu/documents/pdf/cglc/LawOfCyberAttack.pdf
ICJ. (2015). Jurisdiction. Retrieved from http://www.icj-cij.org/jurisdiction/index.php?p1=5
IMPACT. (2015). Mission & vision. Retrieved from http://www.impact- alliance.org/
aboutus/mission-&-vision.html
InfoSec Institute. (2013). 2013 - The impact of cybercrime. Retrieved from
http://resources.infosecinstitute.com/2013-impact-cybercrime/
INTERPOL. (2015). Cybercrime. Retrieved from http://www.interpol.int/ Crime-areas/
Cybercrime/Cybercrime
Kanuck, S. (2010). Sovereign discourse on cyber conflict under international law, Texas Law
Review, 88, 1570-1597. Retrieved from https://www.law.upenn.edu/institutes/cerl/
conferences/cyberwar/papers/reading/Kanuck.pdf
McAfee. (2013). The economic impact of cybercrime and cyber espionage. Retrieved from
http://www.mcafee.com/ca/resources/reports/rp-economic-impact-cybercrime-
summary.pdf
Michigan State University. (2015). Design for adult learning, Teaching and Learning Theory,
Feedback. Retrieved from http://learndat.tech.msu.edu/teach/teaching_styles
OAS. (2015). Cyber-security program. Retrieved fromhttps://www.sites.oas.org/
cyber/en/Pages/default.aspx
Ophardt, J. (2010). Cyber warfare and the crime of aggressions: The need for individual
accountability on tomorrow's battlefield. Duke Law & Technology Review, 9(2), 1-27.
Retrieved from http://scholarship.law.duke.edu/dltr/vol9/iss1/2

Passeri, P. (2015, April 13). March 2015 Cyber Attacks Statistics. Retrieved from
http://hackmageddon.com/category/security/cyber-attacks-statistics/
SANS. (2015). Resources: measuring results. Retrieved from
http://www.securingthehuman.org/resources/metrics
Schjolberg, S. (2007). Terrorism in cyberspace - myth or reality?. Retrieved from
http://www.cybercrimelaw.net/documents/Cyberterrorism.pdf
Shinder, D. (2011, January 26). What makes cybercrime laws so difficult to enforce. Tech
Republic. Retrieved from http://www.techrepublic.com/blog/it-security/what-makes-
cybercrime-laws-so-difficult-to-enforce/
Stockton, P., Goldman, M., (2014). Prosecuting cyberterrorists: Applying traditional
jurisdictional frameworks to a modern threat. Stanford Law & Policy Review, 25, 211-
268. Retrieved from https://journals.law.stanford.edu/sites/default/files/stanford-law-
policy-review/print/2014/06/stockton_goldman_25_stan._l._poly_rev._211.pdf
Thacker, N. (2013). Top 10 reasons information security defences fail. TRUSTMARQUE.
Retrieved from http://www.trustmarque.com/top-10-reasons-information-security-
defences-fail/
Wegener, H. (2014). Regulating cyber behaviour: Some Initial Reflections on Codes of Conduct
and Confidence-Building Measures. Retrieved from https://www.unibw.de/infosecur/
publications/individual_publications/wegener_regulating_cyber_behaviour_paper_2014
Winkler, I., Manke, S. (July 10, 2013). 7 reasons for security awareness of failure.
CSOONLINE. Retrieved from http://www.csoonline.com/article/2133697/metrics-
budgets/7-reasons-for-security-awareness-failure.html

Improving the effectiveness of information security learning programs

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (6)

En vedette

En vedette (17)

Similaire à Improving the effectiveness of information security learning programs

Similaire à Improving the effectiveness of information security learning programs (20)

Improving the effectiveness of information security learning programs