Researchers in model-driven development (MDD) should be intimately familiar with how MDD is used in industry. If they are not, there is a danger that new methods and tools are developed without proper consideration for the way that MDD developers actually work and think. A thorough understanding of current MDD industrial practice can inform research problems and ensure that research solutions are actually adopted.
This talk will describe results from a year long study, which applied methods from social science to understand how MDD is actually used in industry. Based on a survey of over 400 MDD practitioners, in-depth interviews with 22 industry professionals from 17 different companies, and a small number of on-site observational studies, the talk will discuss the current state-of-practice in industrial use of MDD, and will offer some insights on current research gaps.
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Whittle Modeling Wizards 2012
1. The Truth about Model-Driven Development
in Industry
– and Why Researchers Should Care
Jon Whittle
joint work with John Hutchinson, Mark Rouncefield
School of Computing & Communications
Lancaster University, UK
j.n.whittle@lancaster.ac.uk
16. talk outline
• Project EAMDE
– Social/organizational/technical
factors
• Discoveries
– Some observations
– Some lessons
– Some tips
17. Project EAMDE
• Widely-distributed questionnaire on how MDD is
used
– 35 questions pertaining to MDD application
– 449 responses
• In-depth interviews with MDD practitioners
– 22 interviews; 17 different companies
– 150,000 words of transcribed data
– >360 years of cumulative experience
• On-site ethnographic studies (ongoing)
– 2 done (by Steinar Kristoffersen); more planned
18. What EAMDE is not
• an attempt to quantify the penetration of MDD in
industry
• a study on UML
– deliberately broad view of MD*
• an attempt to evangelise or promote MDD
– interested in failure as much as success
21. Examples of MDD
“the broader the domain you try and cover, the
less the productivity increase… with
DSM, you’re not looking at building a modeling
language for embedded applications … you’re
not looking at building one for mobile
applications… mobile embedded, or even
mobile phones or even Brand X mobile
phones, but a particular product family of Brand
X mobile phones…”
23. Which modeling languages do you use?
(tick all that apply)
UML BPMN Vendor In-house SysML Matlab/
DSL DSL Simulink
24. DSLs favored over general-
purpose modeling
• mostly, companies write their own code
generators for very specific tasks
– In contrast, companies often ditch commercial tools
because they cannot modify them the way they want
or because they “don‟t do everything”
– Multiple references to the fact that off-the-shelf tools
could have killed an MDD effort
• Generation of whole systems is not widespread
26. Code generation
“I guess at the end of the day, this dream of code
generation from models doesn’t exist – I mean
everything ends up being done by hand because
either we don’t trust code generators or they just
don’t generate the code we need… it’s actually
impossible to get in non-functional requirements
into code generators – it’s too difficult”
27. Offsetting gains without
realising it
• “sometimes the code generated makes it
necessary to use a larger CPU which costs more
money than the efficient code of an experienced
programmer”
• 8x more expensive to certify generated code
28. Don‟t obsess about productivity
• 65-100% code
generated
• Figures on
productivity gains
differ
– 20-800% gain
– 27% loss
30. So if not productivity, then what?
• The real benefits of MDD are
quality, architecture, reuse
• “whenever you name any single
advantage…you can always achieve the same
advantage with another approach…”
– “it tends to be that a model-driven approach is more
likely to have a well articulated design and
architecture”
31. Example
• An organisation finds itself developing lots of
little (modeling) languages over time:
– “we were generating 70% of the system off these little
XML languages… we would try to separate out the
pieces that were generateable and the pieces that
weren’t… it motivated us to have better separation of
concerns”
33. Doing things faster and better
is not enough
• If the status quo works (or is perceived to
work), there will be insufficient buy-in to change
– MDD should be sold not based on how it can do
things (slightly) better, but in terms of how it can fix
things that are broken
– “software is no longer a bottleneck”
44. Business Practice Doesn‟t
Support MDD
• “lead architects need to understand that just
because their models are simple, they weren’t
going to be put out of a job…”
• Developers are defined by their expertise for a
technology not a domain; so it is not in their
interests to innovate
45. MDD Works Best in Non-
Software Companies
• domain experts already model
– “they already have an established way to design in
Powerpoint”
– “I think they are more open than a company that has very
long years of experience in software development”
48. • mature domain
• non-software context
• ground-up effort
• real business driver
49. “we put it directly in the line of the
product”
“we are not allowed to fail”
50. “the benefit was not only because we
introduced model-driven design”
“we also started a re-use group”
“if you didn’t introduce model-driven
design, the reuse initiative would have
failed”
51. “at this moment, embedded software is
not bottleneck in any project”
52. “56% of all our code is from re-used
building blocks, but in the beginning it was
only 5 or 10%”
“normally, decisions are made as low in
the organisation as possible
53. The Good
• critical path
• non-software context
• real business driver
• MDD an enabler for reuse
65. “it was a totally new concept of switching system”
66. “he made this huge decision ”
“50 developers all using this CASE tool”
67. “if there is a problem, they just contact the
[CASE tool] engineers.. That was kind of
their strategy”
68. “and suddenly the tool doesn’t do something
expected”
“they try to contact the vendor but they don’t
really know what’s going on”
69. “they couldn’t optimize the generated code”
“so the way they had to do it was asking the
hardware guys to have more hard disc, more
memory, because of the tool”
70. “it was a nightmare to them to add all the
features”
71. The Ugly
• immature domain
• top-down effort
• no real business driver
• lack of control
87. • Significant software engineering experience
– Over 1000 years total experience
– 70% involved in software development for 5 years or
more
88. • Employed in a range of different roles
– Developer 17%
– Modeler 21%
– Team leader 19%
– Project manager 16%
– Domain expert 4%
– Researcher 23%
– Architect 7%
89. • Good spread of size of company
– Roughly a third each had:
• 1-100 employees
• 100-1000
• >1000
• A significant number of employees are not
engaged in software development
90. Experience with MDE
• Initial exploration: 10%
• Prototyping: 10%
• First major project: 18%
• ~ 1/3: extensive experience of MDE on many
projects and/or over many years
91. What are models used for?
“Do not use” percentages for MDE activities
93. Use of multiple languages
• 62% of those using custom DSLs also use UML
• Almost all users of SysML and BPMN also use
UML
• UML is the most popular „single use‟ language
– 38% of all respondents
• UML used in combination with just about every
combination of modeling languages
– 14% of UML users combine with vendor DSL
– 6% with both custom and vendor DSL
96. Effect of MDE on Training Costs
Q: Does using MDE allow you to employ
developers with less software engineering
experience?
46% agree, 34% disagree
Q: Does using MDE require you to carry out
significant extra training in modeling?
74% agree, 9% disagree
97. Benefits of code generation
Q: Is your use of code generation an important
aspect of your MDE productivity gains?
75% agree, 10% disagree
Q: Is integrating generated code into your existing
projects a significant problem?
36% agree, 40% disagree
98. Changes on model or code?
Q: Do you mainly make updates on the model
rather than the code?
70% agree, 15% disagree
Q: Do you spend a lot of time synchronizing the
model and the code?
35% agree, 45% disagree
99. MDE and Agility
Q: Does MDE make you faster at implementing
new requirements?
75% agree, 12% disagree
Q: Does MDE prevent you from responding to
business opportunities?
32% agree, 38% disagree
100. Complexity of UML
Q: Is UML too complex?
44% agree, 32% disagree
Q: Is UML powerful enough?
52% agree, 31% disagree
101. Promoting understanding
Q: Does your use of MDE lead to better
understanding between stakeholders?
66% agree, 16% disagree
Q: Does your use of MDE result in unexpected
confusion and/or misunderstandings between
stakeholders?
25% agree, 49% disagree
102. Tooling
Q: Are MDE tools too expensive?
45% agree
Q: Do organizations attempt to deploy MDE using
inappropriate and/or cheap tools?
55% agree
104. Key findings
• Productivity gains from code generation tend to
outweigh losses from integration with existing
code
• MDE allows for faster turn-around on new
requirements
– But there is a significant risk that it may prevent
organizations from responding to new business
opportunities
105. Key findings
• MDE increases overall training costs
• UML is not yet universally accepted as the
modeling language of choice
– Indeed, DSLs are far more prevalent than anticipated
119. Bedtime reading…
• Model Driven Engineering Practices in
Industry, ICSE 2011
• Empirical Assessment of MDE in
Industry, ICSE 2011
• Mismatches between Industry Practice and
Teaching of Model-Driven Software
Development. MODELS 2011 Educators‟
Symp.
• A Survey of Practitioners’ Use of MDE (ask
me for a copy)
Examples to show the diversity of how models/MDD is currently being used in industryBanking example: effort to define an industry-wide standard that will unambiguously and precisely define a set of domain concepts for financial applications (e.g., model for account transfer). Clearly, a massive, industry-wide undertaking. Limited code generation – since essentially trying to extract business rules currently hidden in Java code.
Need some anecdotes here
Up to here: estimated 25 minutes
Up to here: 32 minutes
Up to here: 42 minutes
Mainly due to lack of resources
Mainly due to lack of resources
Business driver
Ground-up
The motivation behind the start of the MDE/reuse effort was that engineers were developing for their own designs for different vehicles/vehicle types – which led to huge complexity in terms of manpower
Before the effort, specs were being produced and then outsourced for writing code – which inevitably led to lack of consistency between spec and code
Although this was a centralized effort coming from management, it was implemented ground-up not as big bang
Business driver: the software engineers didn’t exist (or were too expensive)
Motivation for building an in-house code generator
2 side effects of obsessing about code generation: (i) cost arguments wont make sense, because holistic effects not taken into account; (ii) abstractions will be too low level
All new software/hardware from scratch; hardware guys were applying new technology for performance improvements
Not ground up
No control since tool too big/not understandable/didn’t do what they wanted
No control since tool too big/not understandable/didn’t do what they wanted
Gains offset elsewhere
That is, in the model – thousands of features that were really hard to add in the modeling language and then figure out how the code would be generated
Disclaimer image
Snowball image
Disclaimer image
Disclaimer image
Disclaimer image
Comment on how some managers prefer agile because it is (perceived to better) support decentralised management
Pragmatism prevails!
Inconclusive but illustrates that UML is not universally accepted
Inconclusive but illustrates that UML is not universally accepted
Inconclusive but illustrates that UML is not universally accepted