Context-aware apps - apps that know who you are, where you are, and what you've done - have been all the rage the last few years. Facebook's news feeds, Google Now, and Amazon Recommendations are examples of context-aware applications. Over the last few years, advancements in machine learning, big data, NoSQL, and API technologies has drastically reduced the complexity of building such apps, but requires a brand new approach system architecture. This presentation covers: Lambda architecture and Microservices - two new architectural styles to build contextual apps at scale How companies like Twitter and Netflix have implemented lambda architecture and microservices for recommendations, targeting, and more How Apigee uses both new architectures to implement predictive analytics through Insights (our big data predictive analytics product)

1. API Centric Architectures for
Context-Aware Apps!
Webinar

2. @karlunho
Alan Ho
@gbrail
Greg Brail

3. youtube.com/apigee

4. slideshare.com/apigee

5. Agenda!
1. Context-Aware App Overview!
2. Intro to Lambda architecture and microservices!
3. Industry overview on context-aware architecture!
4. Netflix Recommendation Engine Deep Dive!
5. Insights Deep Dive!
5

6. Context Aware Apps!
6

7. What is Context ?!
7
Historical !
Events!
Recent!
Events! Now!

8. Technology Choices Today!
8
Datawarehouse RDBMS J2EE AppServer

9. Modern Architectures for Adaptive Apps!
9
Lambda Architecture
Batch
Storage
RealTime
Storage
Batch
Processing
(MR)
RealTime
Processing
Microservices Architecture
Micro
Service
Micro
Service
Micro
Service
Micro
Service
Micro
Service
Micro
Service
Query DB
Batch
Layer
Serving
Layer
Speed
Layer
APIs
Direct Mail
Email
Web
Mobile
Outreach

10. Two Major Context-aware Use Cases!
10
Targeting
Recommendations

11. Industry View!
11

12. Needle in a Haystack!

13. Dynamic Decision Brokers!
13

14. Netflix Deep Dive!
14

15. • Offline = Batch
• Nearline = Speed Layer
• Online = Serving Layer +
Microservices

16. Batch Layer (Offline)!
• Singular Value
Decomposition
• Batch like operations
• High algorithm complexity
• Results Flow into Cassandra

17. Speed Layer (Near Line)!
17
• Medium request/response!
• Medium data!
• Medium algorithm complexity!
• Intermediate caching,
incremental learning algos!

18. Serving & Microservices layer (Online)!
• Fast request/response,
performance is key – high
availability/SLA!
• Fallback mechanism to
Nearline/Offline operations!
• Low algorithm complexity!

19. Final Thoughts:!
!
• Very complete system!
• Very custom and high performance!
• Lots of specialized systems for
specialized tasks!
• Not practical for most enterprises!

20. Insights Deep Dive!

21. Recommendations Example!
21

22. Recommendation Architecture!
22
Hadoop
In-Memory
Insights
(GRASP)
Node.js
Node.js
Node.js
Node.js
Node.js
Node.js
Node.js
API BaaS
(APIs)
Direct Mail
Email
Web
Mobile
Outreach
(Batch Scores)
(Counters / Activities)
(Query)
Historical
Events
Realtime
Events
(Context)
Backend
(Mashup)
3
4
5
1
2
6

23. Batch Layer!
• Uses R for building predictive
models!
• Algorithms based on Event
Sequences!
• Lots and lots of tooling!
23
Hadoop
Insights
(GRASP)
API BaaS
(Batch Scores)
1
Apigee UI

24. 24
R Code!

25. 25
Scored Export to Serving Layer!

26. Speed Layer!
!
• Easy to program via node.js!
• Low algorithm complexity – Storm is overkill
for most Apigee customers!
• Simple aggregates via C* counters good
enough!
• Good for capturing locations, activity stream,
etc.!
26
Node.js
API BaaS
Apigee UI

27. BaaS Node.js Counter Example!
var event = new Usergrid.Counter(event_options, function (error, result) { … });
// Then we POST the event to increment the counter
var counter_options = {
name: productName + ’.add_to_cart',
value: 1
}
event.increment (counter_options, function (error, result)
27
curl -X GET https://api.usergrid.com/my-org/my-app/counters?
counter=appleIpad3.add_to_cart
Input code:
Output code:!

28. Microservices Layer!
• API Layer (Swagger)!
• Context Injection!
• Service Orchestration!
• Fallback logic!
28
Node.js
Node.js
Node.js
Node.js
Node.js
Node.js
API BaaS
Direct Mail
Email
Web
Mobile
Outreach
(Context)
Backend

29. Swagger Based API Design!
29

30. Node.js Code!
30

31. APIs are Everywhere!
31
Hadoop
In-Memory
Insights
(GRASP)
Node.js
Node.js Node.js
Node.js Node.js
Node.js Node.js
API BaaS
Direct Mail
Email
Web
Mobile
Outreach
Historical
Events
Realtime
Events
Backend

32. Summary!
• Context-Aware apps are the new experience benchmark
• Modern Apps Architecture = Microservices & Lambda
Architecture
• Netflix is an example of a successful company implementing this
architectures
• Apigee helps companies implement Lambda and Microservices
architectures
32