Handle your Lambdas - From event-based processing to Continuous Integration / Munich AWS User Group - Apr Mar, 2016

1. Handle your Lambdas:
From event-based processing to Continuous
Integration
Sergii Khomenko, Data Scientist
sergii.khomenko@stylight.com, @lc0d3r
Munich AWS User Group - Mar 17 2016

2. Sergii Khomenko
2
Data scientist at one of the biggest fashion communities, Stylight.
Data analysis and visualisation hobbyist, working on problems not
only in working time but in free time for fun and personal data
visualisations.
Speaker at Berlin Buzzwords 2014, ApacheCon Europe 2014, Puppet
Camp London 2015, Berlin Buzzwords 2015 , Tableau Conference on
Tour 2015, Budapest BI Forum 2015, Crunchsconf 2015, FOSDEM
2016, PyData Amsterdam 2016

3. Profitable Leads
Stylight provides its
partners with high-
quality leads enabling
partner shops to
leverage Stylight as a
ROI positive traffic
channel.
Inspiration
Stylight offers
shoppable
inspiration that
makes it easy to
know what to
buy and how to
style it.
Branding & Reach
Stylight offers a unique
opportunity for brands to reach
an audience that is actively
looking for style online.
Shopping
Stylight helps users search
and shop fashion and lifestyle
products smarter across
hundreds of shops.
3
Stylight – Make Style Happen
Core Target Group
Stylight help aspiring women between 18 and 35 to evolve their style through shoppable inspiration.

4. Stylight – acting on a global scale

5. Experienced & Ambitious Team
Innovative cross-
functional organisation
with flat hierarchy builds a
unique team spirit.
• +200 employees
• 40 Engineers/PhDs
• 28 years average age
• 63% female
• 23 nationalities
• 0 suits
5

6. 6

7. A Brief History of Computing
7

8. 8

9. 9
Charles Babbage's machine - 1832

10. 10
1940 – 1956: First Generation – Vacuum Tubes

11. 111956 – 1963: Second Generation – Transistors

12. 121964 – 1971: Third Generation – Integrated Circuits

13. 131972- … : Fourth Generation – Microprocessors

14. 14

15. 15

16. Immutable infrastructure
16
Infrastructure as Code

17. 17
N e e d t o u p g r a d e ? N o p r o b l e m .
B u i l d a n e w , u p g r a d e d s y s t e m a n d
t h r o w t h e o l d o n e a w a y . N e w a p p
r e v i s i o n ? S a m e t h i n g . B u i l d a
s e r v e r ( o r i m a g e ) w i t h a n e w
r e v i s i o n a n d t h r o w a w a y t h e o l d
o n e s .

18. 18

19. 19

20. 20

21. 21
CloudFormation

22. 22
Terraform

23. 23

24. 24
Terraform
Kubernetes and Docker {Swarm, Compose}

25. Serverless architecture
25

26. 26

27. 27

28. 28

29. 29

30. 30

31. 31

32. Installation
32
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
go get github.com/apex/apex/cmd/apex
export AWS_REGION=eu-west-1

33. 33

34. 34

35. 35

36. 36
main.go

37. 37
main.py

38. Python dependencies
38
pip install requests -t functions/dependency

39. 39
apex invoke

40. Outcome
40
• all Lambdas in one place with version control
• integration tests with real events
• proper CI/CD setup

41. 41
CircleCI deployments

42. 42
CircleCI deployments

43. 43
CircleCI deployments

44. 44
Cloud functions

45. Use-case
CloudWatch alerting
45

46. 46

47. 47

48. 48

49. Use-case of
outlier detection
49

50. 50

51. 51
custom
unification
pipeline
Departments
Business
Intelligence
internal processes variety of event types
and structures

52. 52

53. The Future is Now
53

54. 54
AWS Lambda in FRA

55. 55
AWS Lambda in FRA

56. www.stylight.com
sergii.khomenko@stylight.com
@lc0d3r

57. Related links
57
1. The Babbage Engine
2. The Five Generations of Computers
3. IBM System 360
4. https://github.com/cloudtools/troposphere
5. https://www.terraform.io/
6. (Docker Compose + Docker Swarm) or Kubernetes
7. Google Cloud Functions
8. https://github.com/apex/apex
9. https://github.com/apex/apex/tree/master/_examples/python/functions/dependency
10.Streaming Data Processing with Amazon Kinesis and AWS Lambda

58. 58
USE CODE - AWS20