Enterprise application integration is a headache for many IT organizations especially when different legacy applications are involved. The presentation will illustrate how several «Toyota Way» principles are applied within the Enterprise Architecture department of Toyota Motor Europe to guide the design of integration solutions. These will be practically illustrated using Toyota’s product datahub as an example case. The presentation will elaborate on the product datahub’s REST API as a mechanism for provisioning product data to internal IT applications. The talk will conclude with some reflection and «lessons learned» from the first implementation of the product datahub as we’re expanding its utilization into the cloud.
How «Toyota Way» principles guided the architecture of Toyota’s product datahub and its REST API by Philip Rademakers
1. TOYOTA MOTOR EUROPE
How "Toyota Way" principles guide Toyota's REST APIs
Case study: Product Datahub
Philip Rademakers
Senior Manager, Enterprise Architecture
Information Technology
Toyota Motor Europe
2. TOYOTA MOTOR EUROPE
• European headquarter in Brussels (Toyota Motor Europe - TME)
• Began selling cars in 1963
• Over €8 billion invested since 1990
• €20 billion turnover in 2014
• More than €4.3 billion spent with European-based suppliers per year
• 888,015 vehicles sold in CY 2014
• Close to 1.000.000 hybrid vehicles sold in Europe to date
• 4.8 % market share in CY 2014
• Employees 20,000 (direct; including TPCA, 50/50 joint venture Toyota/PSA Peugeot
Citroën)
Toyota in Europe
2
3. TOYOTA MOTOR EUROPE
Toyota’s European Facilities
• Head office in Brussels (Belgium)
• R&D technical centre in Zaventem (Belgium)
• Toyota motorsport in Cologne (Germany)
• Toyota ED2 – European Design/Development in Nice (France)
• European manufacturing
• 6 vehicle plants (Avensis & Auris, Yaris, Camry, Aygo, Corolla & Verso, Dyna)
• 3 engine / transmission plants
• 30 National marketing and Sales companies (NMSC) (56 countries)
• 2749 Toyota dealers, 274 Lexus dealers
• 14 Parts logistics centres, 9 Vehicle logistics centres
3
4. TOYOTA MOTOR EUROPE
TME Information Technology (IT) division (pan-European)
• 260+ employees
• Responsible for:
• (Application) Development
• Infrastructure
• Networking, (mobile) communication
• User support
• Functional areas within IT:
• Business facing: R&D/Purchasing Systems, Manufacturing Systems, Sales Systems, Corporate Systems
• IT support functions: Systems engineering, Pan-European IT management
• 5 different countries (7 different locations):
• Belgium, Germany, Poland, United Kingdom, Turkey
4
5. TOYOTA MOTOR EUROPE
Enterprise Architecture department within Toyota’s IT
5
Support project’s application architecture
Application integration
Development & tools support
Define IT standards and technology roadmaps
Establish reference architectures
Master data management
R&D
Applications
Manufacturing
Applications
Sales
Applications
Corporate
Applications
Infra &
Data centre
6. TOYOTA MOTOR EUROPE
Architectural principles
6
Toyota Way principles (applied in IT)
Just in time
Solutions support
small PDCA
cycles
Architecture
starts small and
supports only
what is needed
Jidoka
Each process
passes on quality
results
Software has
check & stop
capability
Architecture principles
Eliminate Waste
Avoid duplication
of data, platforms
standards
Solutions are
resilient without
redundancy
Built-in quality
IT solutions have
built-in quality
with clear
ownership (JKK)
Components
have low degree
of dependency
Standardization
Use standards
where possible
Solutions employ
open and
extensible
standards
Kaizen
IT solutions are
flexible to kaizen
Components can
grow / evolve
independently
• Allow us to assess that we’re “doing the right thing” (before “doing things right”)
• Embody company’s culture in IT architecture processes
9. TOYOTA MOTOR EUROPE
Application integration: Shared database
9
• Advantages
• Simple, consistent view on data across applications
• Disadvantages
• Very tight “coupling” between applications
• Creates dependencies on physical data models (which tend to change over time)
• Violates an important Toyota way principle
• Any improvement to an application’s data structure immediately impacts other apps
(continuous improvement in one application can be destructive for others)
Kaizen
Continuous improvement
10. TOYOTA MOTOR EUROPE
Application integration: Enterprise Service Bus (2000’s style)
Product Order
Car
Config
Customer
Enterprise Service Bus
10
11. TOYOTA MOTOR EUROPE
Application integration: Enterprise Service Bus
• Advantages
• Decouple service from physical data/process implementation
• Support various adaptors, protocols, messaging styles
• Disadvantages
• Usually requires (expensive) “middleware” to deploy services supported by middleware team
• Violates an important Toyota quality concept
• Often service bus gets cluttered with (hidden) business logic (lack of ownership)
• Middleware team is not involved in application use cases (lack of ownership)
• Middleware team can’t grasp impact in case of failure (decreases quality)
11
Built-in quality with Ownership
(JKK: Jikotei Kanketsu)
Team responsible for an IT service:
- “knows” the service (ownership)
- can see (visually) problems with the service
- can analyse and fix problems with the service
12. TOYOTA MOTOR EUROPE
Application integration via REST
Product Order
Car
Config
Customer
REST API REST API
REST API REST API
HTTP
- GET
- PUT
- POST
- DELETE
URLURL
URL URL
12
13. TOYOTA MOTOR EUROPE
What is REST?
• Representation State Transfer (REST) is an architectural style
centered on resources (vehicle, customer, order) and
representations of those resources
• Typically implemented over HTTP (but does not have to)
and relies on a stateless, client-server interaction
• REST is a lightweight alternative to mechanisms like RPC (Remote Procedure Calls) and
Web Services (e.g. SOAP)
• REST is language independent, platform independent
• REST allows for cacheable responses (to improve performance)
• The client of a REST API sees a representation of a resource, not the actual resource
itself, state is only transferred between client/server via resource representation
• Resources are accessible through a URL; CRUD operations (create, read, update, delete)
are performed using HTTP verbs (POST, GET, PUT, DELETE)
13
14. TOYOTA MOTOR EUROPE
Simple REST example
14
Resource URL: https://customer.toyota-europe.com/customer/1234
GET /customer/1234
Accept: application/json
GET /customer/1234
Accept: application/xml
/customer/1234
/customer/5432
Resources
Server
{ “Name” : “Mike Orzen”,
“ID”: 1234 }
<customer>
<name>Mike Orzen</name>
<ID>1234</ID>
<model>Avensis</model>
</customer>
Client
Client
Representation
/customer/5432
15. TOYOTA MOTOR EUROPE
Why use REST for application integration?
• Simple and general
• Scalable (loose client/server coupling due to “stateless”-ness)
• Independent (few dependencies on other technologies)
• Latency (caching)
• Security (on top of HTTPS, coupled with – separate – authentication)
• Encapsulation (only show things that one needs to show)
• No heavyweight tool chain required (standard web server)
• Relatively easy to test (simple HTTP requests)
• No “intermediary” message handling components (plain HTTP)
• Does not require extra middleware (no ESB)
15
16. TOYOTA MOTOR EUROPE
Case study: Toyota’s product data
16
Vehicle
Access
ory
Vehicle
pricing
Part
pricing
France
Italy
United Kingdom
1
Many point-to-point integrations
(muda: over-production)
2
Different integration interfaces
(muda: unnecessary motion)
3
Not fully consistent
data model (quality)
SOAP
XFB
4
Some data is duplicated across
systems (muda: inventory;
inconsistencies)
5
“Master” data scattered
with “operational” data
Poor value proposition resulting
from silo-based (legacy)
systems
17. TOYOTA MOTOR EUROPE
So how did we resolve those problems? Overall steps:
17
4S
business
data
Single
consistent
data model
and
semantics
Introduce
“Product”
master
data
domain
with clear
business / IT
ownership
Decouple
application
integration
from
(legacy)
source
systems
with REST
+ + +
Eliminate waste,
improve quality
Standardize Customer first,
Just in time
Kaizen
legacy
condition
4S: Sort, Straighten, Sweep, Standardize
*
18. TOYOTA MOTOR EUROPE
Product data architecture
18
Vehicle
Access
ory
Vehicle
pricing
Part
pricing
France
Italy
United Kingdom
1
Central product
data hub
Product
data hub
3
Single quality data
model, central logic
to guarantee
data consistency
5
Clients access
data only through
API
4
Resource
based
REST API
R
E
S
T
2 4S source data
6 Atom change
feed
19. TOYOTA MOTOR EUROPE
Details on the data model (simplified)
19
Vehicle
family
Variation Suffix Accessory
Color
{ Avensis,
Auris,
Yaris,
... }
{ Sedan - 3 door – petrol,
Wagon – 5 door – diesel,
...}
{ Sunroof,
SatNav,
Roof rack,
... }
{ White metallic,
Black,
Green,
... }
20. TOYOTA MOTOR EUROPE
Detail on the REST API
GET /family
{ “family” : [ { “name” : “Avensis”,
“code” : “123”,
“links” : [ { “rel” : “self”,
“href” : “/family/avensis” } ] },
{ “name” : “Auris”,
“code” : “234”,
“links” : [ { “rel” : “self”,
“href” : “/family/auris” } ],
“link” : { “rel” : “self”, “href” : “/family” },
}
GET /variation
{ “auris” : [ { “name” : “variation1”,
“body” : “3door”,
“engine” : “petrol”,
“links” : [ { “rel” : “self”,
“href” : “/family/auris/var1” },
{ “rel” : “parent”,
“href” : “/family/auris” }
20
1
2
5
3
1. JSON as default representation (XML also possible)
2. Resources follow entities from the data model as collections and individuals
3. Collections contain “most important” detail of children
4. “Links” are provided to obtain detail on child resources
5. Individual entities provide back links to their parent collection (navigation)
4
21. TOYOTA MOTOR EUROPE
Principle 1: Jidoka (software has check & stop capability)
21
Error
Code
Condition Error Message
401 When authentication fails Error message produced by the TAAS Service
403 When authorization fails Error message produced by the TAAS Service
503 Service Unavailable due to
database down or other reasons.
Service Unavailable
500 Any internal service logic error Internal Error
400 URL Parameters not correct or
invalid URL requested
No specific Error Message
404 When the resource could not be
found
No specific error message will need to be passed.
410 When the resource is already
deleted in Datahub
No Specific Error Message
• All API calls
• utilize standard HTTP response codes
(e.g. 200=OK, 201=resource created)
• must implement a standardized set of error codes
• may implement API specific error codes
• Any internal error in an API call gets automatically logged
22. TOYOTA MOTOR EUROPE
Principle 2: Just in time (architecture starts small, supports only what is needed)
• API focuses on what current set of clients need
• Additional “requirements” can be developed (later) through
• additional API calls (evolution)
• additional representations (e.g. XML over JSON)
• descriptive links (indicating what can be done “next”)
• API calls provide “most common” info with links to obtain more “detail” (if client wishes)
22
GET /variation
{ “auris” : [ { “name” : “touringsport”,
“body” : “3door”,
“engine” : “petrol”,
“links” : [ { “rel” : “self”,
“href” : “/family/auris/variation1” },
{ “rel” : “parent”,
“href” : “/family/auris” }
23. TOYOTA MOTOR EUROPE
Principle 3: Eliminate waste (solutions are resilient without redundancy)
• API itself only requires an HTTP server as infra + a network
• No other intermediary middleware between client/server
• Compute “load” can be controlled through server scaling
• Data “availability” can be controlled in back-end
(e.g. “always-on” configuration; redundancy)
• These “choices” are independent of the API
23
R
E
S
T
R
E
S
T
R
E
S
T
R
E
S
T
R
E
S
T
Product
data hub
Product
data hub
24. TOYOTA MOTOR EUROPE
Principle 4: Built-in quality (components have low degree of dependency)
• Functional monitoring
24
• Utilization logging
(Problem visualization)
(Jidoka)
(Problem solving) (Ownership)
25. TOYOTA MOTOR EUROPE
Principle 5: Standardization (solutions employ open standards)
• REST is not a “formal” or open internet standard but
widely used (e.g. Google, Twitter, Amazon, ...)
• HTTP, JSON are standards and allow to leverage
caching, security and a simple data format
• Promote REST as primary standard for
inter-application communication and
data exchange
• Promote REST as primary standard for
interaction between client (GUI) and
server
25
REST
Product Customer Dealer Part
http http http
26. TOYOTA MOTOR EUROPE
Principle 6: Kaizen (components can grow / evolve independently)
26
• Easy to scale (stateless) horizontally (new APIs)
• API can also “grow” without affecting existing clients
for example through versioning or by enhancing multiple
representations
• Backend systems can evolve independently without affecting
existing clients (REST API serves as a “contract”)
• API is evolving into V2 (incremental
version)
• Design to run in the cloud
• Data aggregated to JSON
documents stored in Mongo DB
Aggregated
Data
27. TOYOTA MOTOR EUROPE
Is REST the holy grail for application integration?
For some use cases it is not a good match:
• “Streaming” based data (e.g. video, social data)
• Push style notifications (e.g. as in mobile,
server-initiated communications)
• Bulk offline data transfers (e.g. use an ETL tool)
• Asynchronous data (e.g. event interactions)
• Big data (e.g. better use event streams)
But REST:
• Fits well with our Toyota way architectural
principles for inter-application integration
27
28. TOYOTA MOTOR EUROPE
Future work
• Reengineer source applications to directly interface with master data
domain (eliminate duplication)
• Yokoten (best practise sharing) concept to manufacturing applications/area
• Implement “data as-a-service” in other domains (customer, dealer, parts)
• API “management”(documentation, discovery, usage, lifecycle, reporting,
governance)
• Partner and Open APIs (allow 3rd parties to interface with Toyota’s data)
28
29. TOYOTA MOTOR EUROPE
Q&A
Thank you !
29
Philip Rademakers, Senior Manager, Enterprise Architecture, IT division, Toyota Motor Europe