[Rakuten TechConf2014] [B-6] Rakuten Travel Architecture and Development Process
1. Rakuten Travel Architecture and
Development Process
October/25/2014
Shunsuke Hoshino / Mitsuru Saito
Travel Development Department, Rakuten Inc.
http://travel.rakuten.co.jp/
2. 2
History of Rakuten Travel
Service started on January 1996 as “ホテルの窓口” with just 2 PCs.
Started with just 2 PCs
Network : 128Kbps
Browser : Netscape 2.0
Fax by Manual => Automatic
Process by Windows3.1
3. 3
History of Rakuten Travel
1996 - 1999
ホテルの窓口
1999 - 2004
My trip net
2001 - 2004
Rakuten Travel
http://www.mytrip.net/
2003
M&A by
Rakuten Inc.
2004
Integration into
Rakuten Travel
Business Model by 旅の窓口
Service Name by 楽天トラベル
http://travel.rakuten.co.jp/
Integration of Rakuten Travel
10. 10
Rakuten Travel Services
Web Booking ASP
for Original Home Page
Corporate Travel
Call Center
Open API
http://travel.rakuten.co.jp/corporation/demand.html
http://travel.rakuten.co.jp/callcenter/
http://travel.rakuten.co.jp/webservice/
11. Overseas Website
Travel Mail
Platform
11
Product Map of Travel
Product
Private APIc
DB/KVS/BigData
Hotel
(package)
Search Engine
Booking
Process
Vacant Search
Smartphone
Apps
AD
FAX
Mobile
Review
Keyword
Search
Retargeting/
Personalization
Booking
Sight
seeing
Hotel’s
Extranet
R-with
Bus
Car Rental
In-house
tool
Overseas
air
Overseas
hotel
Overseas
DP
JAL
Raku-pack
ANA
Raku-Pack
R-connect
(Connect hotels)
SearchAPI
Rakuten
ID
Point
Coupon
Payment
API
DB/DataStore
Product
My Page
Corporate
Booking
Accounting
Prize
Mail-magazine
R-mail
Master
Inventory
Price
Log Review
room
Car Rental
Hotel
(overseas)
Booking
Confirmation
・・・
OEM
Open
Platform
Affiliate
WTS GMO ECON
Airline Company/ Whole Seller/ outside engine
Chain Hotels
BI Tool
Car Rental
connect API
Car Rental
Company
DWH
inventory
pricing
BookingAPI
Booking
Air ticket
Management
Review
・・・
Amazon
SAP
12. Search Engine(plsql)
Solr
Qsol
roma
fee
Stock
Info
Rsv
Rakuten
Web Service
DB
Review
Timesten
Overseas Website
API
Database
Product
R-connect
Hotel
Web Service
…
…
External API
ana
jal
GMO
ECON
WTS
MIKI
GTA
ANA
JAL
Overseas
DP
Overseas
hotel
Overseas
PMS
Data Sync
API Data Sync
API
DP
air-ws
rax-api
Data Store
Personal Page
api
ID
Point
Coupon
Payment
Rakuten
Review
Keyword
Search
Ashiato
Hotel
Management
In-house
tool
Rent a car Bus R-with
Reservation
Japan Hotel Search
Date Map Station
Overseas
air
F
R-download
Download
Racco
Mailmagazine R-mail
Prize
AD
Coupon
Access Log
Internal Operation Tool
EJC/iNu
Sales Accountant BU
FAX
API
Travel Mail Platform
F
Air Hotel DP
fee
Stock
Info
Rsv
Review
…
…
China DB
F F
F
VPN
Booking
MQ
F
F
Ainori
...
App
Zenrin
Google
Private Cloud
PaaS
Tabi Note
F
System Overview of Rakuten Travel
Amazon
Hadoop
Data Warehouse
SAP
14. 14
Challenges of Recent Travel
System Complexity
Old Technology(PL/SQL, C)
Dependency on Single Technology
Non transparent product development
Too Many Manual Processes
15. 15
Challenges of Recent Travel
System Complexity
Old Technology(PL/SQL, C)
Dependency on A Single Technology
Non Transparent Product Development
Too Many Manual Process
16. Search Engine(plsql)
Solr
Qsol
roma
fee
Stock
Info
Rsv
Rakuten
Web Service
DB
Review
Timesten
Overseas Website
API
Database
Product
R-connect
Hotel
Web Service
…
…
External API
ana
jal
GMO
ECON
WTS
MIKI
GTA
ANA
JAL
Overseas
DP
Overseas
hotel
Overseas
PMS
Data Sync
API Data Sync
API
DP
air-ws
rax-api
Data Store
Personal Page
api
ID
Point
Coupon
Payment
Rakuten
Review
Keyword
Search
Ashiato
Hotel
Management
In-house
tool
Rent a car Bus R-with
Reservation
Japan Hotel Search
Date Map Station
Overseas
air
F
R-download
Download
Racco
Mailmagazine R-mail
Prize
AD
Coupon
Access Log
Internal Operation Tool
EJC/iNu
Sales Accountant BU
FAX
API
Travel Mail Platform
F
Air Hotel DP
fee
Stock
Info
Rsv
Review
…
…
China DB
F F
F
VPN
Booking
MQ
F
F
Ainori
...
App
Zenrin
Google
Private Cloud
PaaS
Tabi Note
F
System Overview of Rakuten Travel
Amazon
Data Warehouse
SAP
Many services are dependent
Hadoop
17. 17
Platform Strategy
User Smart Phone Server
Layer
Travel
Front
services
PC
API I/F
for alliance Domestic Overseas Transport
Etc.
Search API Booking API
Inventory
API
Hotel API Etc.
Inside
Travel
APIs
Connect to Travel services
Data
Inventory
Search
Update
Inventory
Hotel
Information
Enhance API layer
18. 18
Example of API-nization
Master
Slave
Oracle
User
Multilingual Travel Site
Hotel Extranet
Booking API, Search API Inventory API
Memory Bus
NoSQL DB
19. 19
Challenges of Recent Travel
System Complexity
Old Technology(PL/SQL, C)
Dependency on A Single Technology
Non Transparent Product Development
Too Many Manual Process
20. Search Engine(plsql)
Solr
Qsol
roma
fee
Stock
Info
Rsv
Rakuten
Web Service
DB
Review
Timesten
Overseas Website
API
Database
Product
R-connect
Hotel
Web Service
…
…
External API
ana
jal
GMO
ECON
WTS
MIKI
GTA
ANA
JAL
Overseas
DP
Overseas
hotel
Overseas
PMS
Data Sync
API Data Sync
API
DP
air-ws
rax-api
Data Store
Personal Page
api
ID
Point
Coupon
Payment
Rakuten
Review
Keyword
Search
Ashiato
Hotel
Management
In-house
tool
Rent a car Bus R-with
Reservation
Japan Hotel Search
Date Map Station
Overseas
air
F
R-download
Download
Racco
Mailmagazine R-mail
Prize
AD
Coupon
Access Log
Internal Operation Tool
EJC/iNu
Sales Accountant BU
FAX
API
Travel Mail Platform
F
Air Hotel DP
fee
Stock
Info
Rsv
Review
…
…
China DB
F F
F
VPN
Booking
MQ
F
F
Ainori
...
App
Zenrin
Google
Private Cloud
PaaS
Tabi Note
F
System Overview of Rakuten Travel
Amazon
Data Warehouse
SAP
Many P/L SQL and C
Hadoop
22. 22
Challenges of Recent Travel
System Complexity
Old Technology(PL/SQL, C)
Dependency on A Single Technology
Non Transparent Product Development
Too Many Manual Process
23. 23
Current Search Architecture
English site
To Japan
Booking
Domestic
Search
Domestic
DP Search
Hotel Info
Other Travel Service
For PL/SQL
Sync
DB Link
Inefficiency of cost
Difficulty of scale out
25. 25
NoSQL DB for Multilingual Site
Search API
FRONT Service
Booking
Cancel
Inventory Core
Queue
Documents
Stock/Price
Get Documents
Search / Result
dirver
Stock
Price
Documents
26. 26
Challenges of Recent Travel
System Complexity
Old Technology(PL/SQL, C)
Dependency on A Single Technology
Non Transparent Product Development
Too Many Manual Process
27. 27
Non Transparent Product Development
Non clear responsibilities
Short sight vision
No definition of the products
Random process to develop
Lower quality products and services
33. Product Requirements Document
• Clear definition of requirements
• Sign-off process with managers
• Base document for dev spec and test spec
34. The volume of members
34
Quality Assurance
QA team
2014 12
2015 20
35. What Has Been Improved So Far
• Easier to prioritize projects
• No extra tasks for designers and developers
• Test cases are easier to write
• No back and forth development
We are managing over
60 projects!
36. 36
Challenges of Recent Travel
System Complexity
Old Technology(PL/SQL, C)
Dependency on A Single Technology
Non Transparent Product Development
Too Many Manual Process
37. 37
Automation
Design
Development
Improvement/
Measurement
Release QA
Full-swing
introduction in All
services.
Staging
40. 40
Next Challenge of Rakuten Travel
24/5 Development
T.R.I.P.
DEV-OPS
41. 41
24/5 Global Development Team
Travel development team aims to develop all over the world for 24/5 development.
France
India
Japan We are planning to establish
development team.
1. India development team
(From mid of 2015)
2. France development team
(From 2016)
42. 42
T.R.I.P
We start to use Rakuten DC from 2015
Osaka DC
Tokyo DC
Virtualization Ration
Osaka DC Tokyo DC
Real 67% 21%
VM 33% 79%
・Increase the Virtualization
・New Infrastructure Architecture
Notes de l'éditeur
Jan 1996 Start up Sun IPX
Oct 1998 At a faster pace Sun Enterprise 4500
Jun 2000 5 times capacity IBM RS6000 S80
Jul 2001 HA IBM RS6000 S85
Dec 2003 Oracle 9i R2(RAC) IBM eServer p690+670
Dec 2004 Oracle 10gR1(RAC)+PL/SQL server IBM eServer p690+670+595
Dec 2005 Oracle 10gR2(RAC) IBM eServer p5 570+595
Feb 2007 PL/SQL server IAnization + Oracle 9node RAC IBM eServer p5 570+595
IBM x3950
Dec 2008 Oracle11g R1 IBM eServer power 570
IBM x3950
Oct 2009 Install TimesTen IBM eServer power 570
IBM x3950
Dec 2012 DB server IAnization
Jun 2013 Install Violin memory for storage
Role sharing (division of roles)
Sharpen each talent per role
Help each other with professionalism
Require professionals who orchestrates engineering team-> PM (Program manager, Project manager and Product manager)
An important aspect of our process is the use of clearly defined roles.
In the early days of the Internet, technology was simple enough so that supermen like Linus Torvalds who invented the Linux OS, and Vint Cerf who worked on Internet Protocols could be relied on to create products and services almost single handedly.
Now, in the age of complicated technology like mobile and Web 2.0, the industry needs specialists to focus efficiently on one type of work.
Travel uses these clearly defined roles to help manage tasks:
Developer
No new operation for DU
Reduce meeting time for services
BU
All requirements/discussion will be turned into PRD for 2013
Customer
Simplify UX for travel site
Multilingual platform
Extranet for tablet
Windows, Google connection
Mail content improvement
Developer
No new operation for DU
Reduce meeting time for services
BU
All requirements/discussion will be turned into PRD for 2013
Customer
Simplify UX for travel site
Multilingual platform
Extranet for tablet
Windows, Google connection
Mail content improvement
Let me briefly explain our development process.
We begin by documenting what we want and need to make, and then we make it, test it, and release it.