EclipseCon Europe 2012 - Big Models

EclipseCon Europe 2012

Challenges in the Automotive Industry:
EMF Model Repositories - Scalability vs Performance
Or
The “Big Model” Problem

Christof Hammel Andreas Rytina

Cross Divisional Group - Software, Methods and Tools
1 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any disposal, exploitation, reproduction, editing,
distribution, as well as in the event of applications for industrial property rights.


Outline
Part I
 Automotive

 Domain

 Software Development

 Software Architecture & Use Cases

 Development Environment

 The “Big Model” Problem

Part II
 Model

 Structure

 Technology

 CDO

 Scalability & Performance

 Conclusion

2 CDG-SMT/EMT | 04/09/2012 | © Robert Bosch GmbH 2012. All rights reserved, also regarding any
disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.


Automotive Domain
Case Study: Domain & Products:
Powertrain Engine Control Units

Our Customer (OEM) Products:

5500 employees worldwide

400 projects with all OEMs Passenger Car Heavy Duty

20 Mio ECUs / year
Off Highway


Our Embedded Product-Software
500.000 LOC

~500Mio LOC (> than 1st Space-Shuttle)

 Product-Line development with high variability
 1.500 variation points (pre compile) / “Meta” project

 Automotive SW Component-Model (e.g. BlueArX, Autosar)

.xml .c .xml .c

 Interface consistency mainly determined by XML Component-Model data
 Interfaces by Name-Refs (no class-path)

 ~ 1.000 components (= PlugIn) / project


Desktop Software Domain
PlugIn Development Project Integration

(ww) Distributed teams
develop independent

Integration

A B C A B C

Target Target Target Target

PlugIn-Teams use same target


Automotive Software Domain
Component Development Project Integration

2.0.0
C 2.0.0
1.0.0 C

C

2.0.0 Component Selection
B
B B A
1.0.0

2.0.0 1.0.0
2.0.0
A
1.0.0 A



Our Software Architecture (Powertrain)



Our Use Cases
Use-Case 1: Validation
 Check interface consistency across complete project: < 1min

~30 Interfaces
... ... ...
...
1.000 components
( PlugIns)
 30.000 Traverse, Search, Check & Logging operations < 1min
 Fast data cache
 Complete project data in memory as Object-Model (Ecore)

Use-Case 2: Diff/Merge
 Updates require comparison to Reference/Predecessor-Proj.

 Multiple projects loaded in IDE



Our Development Environment
IDE
Applications / Use-Cases:
• View, Navigate, Search, Edit
• Build (Processing)
• Validation
• Diff/Merge

BDOM-Model (EMF based)

Loader / Serializer

Workspace

Version-Control, SCM

File-Server Meta-DB

Comp-Baselines



Our BDOM-Model
 Based on EMF
 A few numbers:
 900 entities (meta classes)
 2.800 attributes and
 2.700 references
 ~ 800MB heap consumption / project
 Windows XP (32Bit):
 Max. 4GB address space

 2GB User-Space

 1GB for other tools (Outlook,...)

 1GB heap size remaining for: Java classes, EMF domain model

 Only 1 medium size project can be loaded into IDE

 Big or multiple projects exceeds 32Bit Java systems

 The Big Model Problem !!!



Model Structure – Variant Management
extends
PL<CLASS> P<CLASS>

implements
PLIConditional instance of
instance of
+sysCond: PLExpression
Product Models
a' : P..
Product Line Model
 The BlueArX Meta Model is
seperated into a Product Line and a : PL.. b' : P..
a Product specific part
b : PL.. c : PL.. d' : P..
 Each meta class which is
System
configurable in the product line
Condition a'' : P..
has a +sysCond attribute d : PL.. configuration

b'' : P.. c'' : P..



Model Structure – Name Based References

<source> <target>NameRef <target>
* 1 1 *

 Handle incomplete data structures
 Easy loading, adding and deleting
 Deleting data structures from model without damaging the model consistency

Conditional Name Based References

Cond<target>NameRef
<source> <target>NameRef <target>
+sysCond: PLExpression
1 * 1 * 1

 Makes it possible to relocate complete sub trees in the architecture tree



Premises and Technology Selection
 Large number of applications have been implemented on top of BDOM
 API must be stable (protection of investiment, quality)

 Premise: Solution must not disrupt existing applications

 Technology Selection:
 Teneo:
Relational DB. Not selected because SW-Engineering data is a graph model
 Mongo/EMF
 EMFStore
 CDO :
At the time of evaluation, CDO was most visible in the Eclipse community.



Solution with CDO – the components
Net4j
JVMConnector

Client Server

TxPool RafStore
ReadOnlyView +Accessor

*.arxml
*pavast.xml

Builder
Loader Random
BDOM
Access File
Cache
Processor



Solution – write complete model to store
Net4j
JVMConnector

Client Server

TxPool RafStore
*.arxml
*pavast.xml ReadOnlyView +Accessor
…

Builder 2. Directly fill random access file

Loader Random
Loader Access File
Cache
Processor



Solution – query a model instance
Net4j
JVMConnector
• View
• Validation Client Server
• Editing
• Processing
TxPool

Existing API
RafStore

*.arxml
*pavast.xml
3. get raf pointer from index
4. load element from raf
Builder
Loader Random
Loader Access File
Cache
Processor



Solution – edit and save a model instance
Net4j
• View JVMConnector

• Validation
Client Server
• Editing 10. generate raf pointer
11. insert raf pointer at the index end
• Processing 12. write new & changed elements

Existing API
TxPool RafStore
6. serialize

*.arxml
*pavast.xml 3. get raf pointer from index
Builder 4. load element from raf

Loader Random
Loader Access File
Cache
Processor



Reduce memory consumption
 The memory consumption can be reduced after the the model building step when
the model is written to the database. But due to the cdo overhead the model
building step consumes about 50% more memory.

 Building a big model… (vmargs: -Xmx3g -XX:MaxPermSize=256m)
…w/o CDO …w/ CDO
building writing to store



Don´t lose too much performance...
 This couldn„t be achived for very query intensive use cases, i.e. model validations,
content assist,….

Some measurements with EMF Validation Framework based constraints:
(64bit JVM, vmargs: -Xmx3g -XX:MaxPermSize=256m)

Use-Case w/o CDO w/ CDO
BlueArX validation with Validation time: 2 min Validation time: 12 min
a small model ~ factor 6 performance decrease
Heap-Peak: 450MB Heap-Peak: 742MB
BlueArX validation with Validation time 1:30 min Validation time: 23 min
a medium size model ~ factor 15 performance decrease
Heap-Peak: 1.1GB Heap-Peak: 2.2GB
BlueArX validation with Validation time: 2 min n.a.
a big model
Heap-Peak: 1.9GB OutOfMemoryError with –Xmx3g 
(~ 4GB estimated)



Conclusion
 Other Design Options:
 Manually Optimizing all queries

 Replace EMF by Memory-Mapped-Files

 3- or 2-tier architecture, using powerful servers for model queries

 High effort (at least 1-2 years)

 Developer‟s PC Infrastructure:
 32Bit are gradually going extinct

 64Bit/Win7 will be rolled-out at the time other design options would be

implemented.
 No benefit in investing in a different tool architecture

 Efforts to be invested to improve speed performance:
 E.g. EMF Indexing

 Evaluate / Define standard solution for Automotive (e.g. Inc-Query)

 Coordination in the Eclipse Auto-IWG


EclipseCon Europe 2012 - Big Models

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (17)

Similaire à EclipseCon Europe 2012 - Big Models

Similaire à EclipseCon Europe 2012 - Big Models (20)

Dernier

Dernier (20)

EclipseCon Europe 2012 - Big Models