1. 1/31
www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
Jan Claes
Teaching assistant : PhD 2009 – 2015
Supervisors UGent : Geert Poels & Frederik Gailly
Supervisors TU/e : Paul Grefen & Irene Vanderfeesten
Why do people struggle with the complexity
of constructing a process model?
2. 2/31
www.janclaes.info
Business Process Modeling (BPM)
Assumptions
Constructing process models is useful
Constructing the models is cognitively challenging
Business Process Modeling (BPM)
Process model quality
“what is a good process model?”
Process of process modeling (PPM)
“how to make a good process model (effectiveness)
using a good approach (efficiency)?”
8. 8/31
www.janclaes.info
Observations (objective)
Obs. 1. Majority used serialized approach
Obs. 2. Large group flow-oriented modeling
Obs. 3. Small group aspect-oriented modeling
Obs. 4. Large group combine FO and AO
Obs. 5. Small group undirected modeling
Obs. 6. Remainder is uncategorized
Obs. 7. Undirected sessions lasted longer
10. 10/31
www.janclaes.info
Impressions (subjective)
Impr. 1. Modelers need serialization
Impr. 2. Structured serialization
helps avoiding ‘mistakes’
Impr. 3. Structured serialization
does not support every modeler
to the same extent
12. 12/31
www.janclaes.info
Kinds of human memory
input
Sensory
memory
Working
memory
Long term
memory
performance
OBSERVATION SELECTION
ORGANIZATION
RETRIEVAL
TRANSFER
Working
memory
13. 13/31
www.janclaes.info
Types of cognitive load
Time
Instantaneous
load
overall load
intrinsic
load
extraneous
load
germane
load
free
capacity
assumed capacity
instantaneous
load
overload
14. 14/31
www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
15. 15/31
www.janclaes.info
Cognitive Fit Theory (CFT)
Need for structure
Learning style
Field dependency
Sensory
Visual
Inductive
Active
Sequential
Intuitive
Auditory
Deductive
Reflective
Global
I ♥︎ chaos I #$@&! chaos
DON’T SEE THE FOREST THROUGH THE TREES
18. 18/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
structuredness
of serialization
– –
– –
1 2 3
20. 20/31
www.janclaes.info
Methodology
1. Cognitive tests prior to the test session
Collect modeler data
2. Observational modeling session
At TU/e in BPM lecture of September 26
Collect modeler, modeling and model data
3. Post-experiment interviews
with selection of participants (e.g., outliers)
Collect additional, qualitative data
23. 23/31
www.janclaes.info
Experiment set-up
1. Prior to the experiment session
Software to be installed on personal laptop
( = download from website and unzip)
5 cognitive tests (time range 5-30 min)
• Reading span test (RSPAN)
• Operation span test (OSPAN)
• Counting span test (CSPAN)
• Hidden figures test (HFT)
• Need for structure (NFS)
24. 24/31
www.janclaes.info
Experiment set-up
1. Prior to the experiment session
To be completed at home
Before deadline
Software collects data in one-zip file per test
which participants have to submit
25. 25/31
www.janclaes.info
Experiment set-up
1. Prior to the experiment session
Students will receive their individual results
as incentive to not cheat on these tests
Late responses or too much errors reveals cheating
Student is not allowed to start actual experiment if
the test results are not submitted in time
27. 27/31
www.janclaes.info
Experiment set-up
2. At the experiment session
Modeling task and post-task survey
In the same, already installed program
(using a different start-up code)
Software automatically collects data
• Each operation in the tool
( = each activity in the modeling process)
• Constructed process model, number of operations,
modeling time
in one-zip file which they have to submit
28. 28/31
www.janclaes.info
Experiment set-up
2. At the experiment session
Student does not get credit if not all data (6 zip files)
are submitted
Technical problems > replacement assignment
(e.g., help me with coding the collected data)
30. 30/31
www.janclaes.info
Experiment set-up
3. After the experiment session
Data will be analyzed and outliners will be defined
Selection of students will be invited for post-
experiment interview (voluntarily)
Collect qualitative data
• Explain how they have modeled
• Explain their cognitive profile
• Explain which results we expected and why
• Ask about their opinion: why did they (not) achieve the
predicted results?
31. 31/31
www.janclaes.info
Thanks for you attention!
> Do you have feedback on my research plans? <
> set-up is useful for theory testing? <
> set-up is feasible (enough)? <
> other suggestions? <
Does someone has some time
to test the software? Jan Claes
jan.claes@ugent.be
http://www.janclaes.info
Twitter: @janclaesbelgium
33. 33/31
www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
34. 34/31
www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
working memory
capacity
Reading span - Operation span - Counting span
Reading span
For many years, my family and friends have been working on the farm. SPOT
Because the stuffy was room, Bob went air for some fresh outside. TRAIL
We were fifty miles out at sea before we lost sight of the land. BAND
Counting span
Count the dark blue circles
Operation span
IS (8 / 4) – 1 = 1 ? BEAR
IS (6 X 2) – 2 = 10 ? BEANS
IS (10 X 2) – 6 = 12 ? DAD
35. 35/31
www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity prior knowledge
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
input material
representation fit
Fit with modeling task = constant
Fit with modeler through questions
Example questions
I did experience problems using the modeling tool (5-point Likert)
Using the modeling language was not difficult (5-point Likert)
I found it easy to understand the case description (5-point Likert)
36. 36/31
www.janclaes.info
Cognitive Load Theory (CLT)
prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
case complexity
Case complexity = constant
(number of activities and arrows in solution model can be used as proxy for number of concepts and relations)
37. 37/31
www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
prior knowledge
Prior domain, modeling language and modeling method knowledge
through questions
Example questions
I am familiar with the mortgage handling process (5 point-Likert)
How many process models did you ever create?
I am experienced in process modeling (5-point-Likert)
38. 38/31
www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
extraneous
cognitive load
germane
cognitive load
cognitive overload
intrinsic
cognitive load
Through questions
Example questions
I found it hard to complete the task (5-point-Likert)
39. 39/31
www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
– schema construction
40. 40/31
www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
process model quality
Syntactical quality: number of cognitive mistakes
Semantic quality: assessment by experts using checklist (penalty for over- and underfit)
41. 41/31
www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
overall construction effort
Time between start of reading case description and last operation in the tool
Number of operations
42. 42/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
structuredness
of serialization
– –
– –
43. 43/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
structuredness
of serialization
– –
– –
Through questions
Example questions
When I write a text:
• I usually work on content, structure, wording and format right from the start
• I usually first decide on the content, then the structure, next I write sentences
and finally I work on the formatting
44. 44/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
structuredness
of serialization
– –
– –
Through questions
Need for Structure scale
12 questions with 6-point Likert scale
45. 45/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
structuredness
of serialization
– –
– –
Hidden Figures Test
46. 46/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
– –
– –
degree of
serialization
structuredness
of serialization
Assessment by experts
Flow-oriented – Aspect-oriented – Combination
Happy Path First – Undirected
Uncategorized
47. 47/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
structuredness
of serialization
– –
– –
serialization style fit
Theoretical match
Post-experiment interview
Global learner = aspect-oriented, sequential = flow-oriented
High need for structure = no undirected
Field-dependent = flow-oriented
48. 48/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
+ + +
serialization style fit
structuredness
of serialization
– –
– –
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
See earlier: through questions