ANALYTICS

• Local social network
• Web, iOS and Android
• 1.3M users in uk.
• 1.5B events

analytics are core for our business

we track millions of events a day

Subject: Here's what your neighbours are talking about...
To: user@domain.com

Subject: Here's what your neighbours are talking about...
To: user@domain.com

Subject: Here's what your neighbours are talking about...
To: user@domain.com

Susan P.

Susan P.

Susan P.

we currently use the batch pipeline
Redshift

we gather insights once a day
Redshift
user_stats
message_stats
SQL

last_visit last_email_open
2016/01/15 12:33:54
2016/07/12 22:13:10
2016/07/10 04:52:01
2016/03/11 21:01:03
2016/07/11 05:34:45
2016/07/12 14:30:21

day impressions
2016/01/15
engagmentsid
1
u_impressions u_engagments
345 122 299 99
2016/01/152 123 99 94 84
2016/01/153 934 845 843 789
2016/01/141 899 777 744 645
2016/01/142 754 543 634 433
2016/01/143 103 99 91 77
2016/01/131 499 301 382 235
2016/01/132 1893 1400 991 1099

With this data we give businesses insights
Your message got 890 impressions
and 350 engagements.

We tune each message's reach dynamically
2000
target
1200
current
>

It works great, but the data has several
hours of lag
event
ingest
Load SQL View S3 Load

We are going to start using the realtime pipeline
realtime
analytics
realtime
content router

realtime pipeline
Redshift
Kinesis
λ

Kinesis
λ λ
Dynamodb
LambdaLambda

The latency from events happening to us acting
on them would be nearly zero
event
ingest
Kinesis Lambda DynamoDB Load

AWS Lambdas: 20,000 feet view
.py
.js
.java
DEPENDENCIES
ZIP
AWS
S3 λ

λ
• Zero-administration compute platform
• Connect Lambdas with AWS services
• Kinesis, DynamoDB, APIGateway, S3,
CloudWatch, Cron…
• Pricing based on usage:
• ~$0.0000002 per/run
• ~$0.000000208 per/100ms
λλ
AWS Lambdas: 20,000 feet view

There is a ‘but’
Deploying/Managing them is a pitaλ

• Tool to create, wire and deploy AWS Lambdas using
CloudFormation
• Python/Javascript/Java/Golang/Scala runtimes…
• Supported integrations
• APIGateway, CloudWatch, Dynamodb, Kinesis, S3

+
snowplow
events
simulator
λ DynamoDB
Kinesis

We are hiring!
Thank you!
Software Engineer (data team)
Software Engineer (core team)
Infrastructure Engineer
Data Scientist