The road to a better API for Twitch developers: learn about our API Gateway technology, how it works, and why we built it this way.  JN Vollmer, Engineering Manager for API Infrastructure, Twitch  Chris Gamble, API Senior Engineer, Twitch  Jos Kraaijeveld, Engineering Manager for API Product, Twitch

1. Under the hood,
The new Twitch API

2. Introductions
JN Vollmer
Engineering Manager, API Infrastructure
Jos Kraaijeveld
Engineering Manager, API Platform
Chris Gamble
Senior Software Engineer, API Platform

3. #TwitchDevDay
Agenda
Where we are now
What we built, and Why
Where we’re going
1
2
3

4. #TwitchDevDay
Where we are now

5. #TwitchDevDay
Where we are now
Facts about the Twitch API
70,000 requests per second on average
200,000+ requests per second peak
25,000+ client applications daily
200ms p99 internal response time on average

6. #TwitchDevDay
Where we are now
You’re already using the Twitch API
If you have ever used Twitch, you’ve used a product
powered by the API
Desktop Website
Mobile Website
Desktop App
Mobile Apps
Console apps
Broadcasters also use the API daily
Overlay alerts
Subscriber only chatrooms
Chat bots

7. #TwitchDevDay
Where we are now
Evolution of API Philosophy
1st party vs 3rd party API
Different requirements
Different constraints, SLA
Different policies
V1 was released in 2012
Better standards for data exchange with
internal products
Enabled a burgeoning ecosystem

8. #TwitchDevDay
Where we are now
Evolution of API Philosophy
API team formed in early 2016
More direct external developer advocacy
In parallel, Twitch transformed its engineering stack from rails
monolith to Go microservices
These services needed a canonical way to expose their data
➔ Birth of our API Platform: Visage

9. #TwitchDevDay
Where we are now
Visage, an extensible platform, is the pedestal that enables:
/Helix (the new Twitch API) in its current form:
Users
Streams
Follows
Love letter to the Developers
Consistency
Full Transparency
Reliability
New modern, faster end-user experiences:
The new beta Twitch Website
Mobile native and web clients
Extensions
Twitch integration in Games…

10. #TwitchDevDay
Visage: Twitch API Gateway

11. #TwitchDevDay
Handles all incoming API requests
Visage: Twitch API Gateway
What it does
Performs common operations
Parsing
Authentication
Ratelimiting
Cache headers, CORS, UTF8, SSL, Locale, etc
Routes to versioned endpoints
Endpoints make service calls
Calls automatically use a shared cache
Formats response

12. #TwitchDevDay
Visage: Twitch API Gateway
Why not use an existing gateway?
Existing gateways are pass-throughs, they
make one data call per request
Visage is an aggregator, it makes multiple
data calls per request

13. #TwitchDevDay
Visage: Twitch API Gateway
The Visage API Gateway: Benefits
Benefits compared to our older Rails stack:
Provides great latency reduction
Scales better, both horizontally and vertically
Simpler to understand
Well defined ownership for endpoints

14. #TwitchDevDay
Visage: Twitch API Gateway
Core components
Built on Go
Uses the latest version of Go
Lots of expertise about Go quirks within Twitch (see
our blog!)
Stateless Architecture utilizing AWS services
Application Load Balancers
Elastic Beanstalk
Elasticache
Xray tracing
Puppet for configuration and host setup
Local DNS resolvers to decrease latency and increase
availability

15. #TwitchDevDay
Visage: Twitch API Gateway
Data Aggregation
Each API request makes at least two service calls, potentially dozens
Almost all service calls are preceded by a cache lookup
Every service and cache call is wrapped in Hystrix to protect against downstream failure
Service calls primarily use Twirp (Twitch RPC) to auto-generate client & server stubs from protobuf

16. #TwitchDevDay
Visage: Twitch API Gateway
Rate Limiting
Only applies to Helix endpoints
Memcached as a backing store
Keys look like “ip-127.0.0.1-1507932905”
Each bucket stores number of requests per 5 second period
To determine usage we add up the twelve buckets in the last minute
Authed requests are limited by Client-ID, Unauthed by IP

17. #TwitchDevDay
Visage: Twitch API Gateway
Integrations
Swagger API definitions
Automatic documentation generation
Machine readable
End to end tests using Runscope
Runs on deploys to catch breaking changes
Auto generated from Swagger definitions
Metrics emitted to StatusPage.io (devstatus.twitch.tv)
Proxies internal graphite metrics via their API

18. #TwitchDevDay
Visage: Twitch API Gateway
The Other Bits
Other features:
SSL enforcement
Authorization
Authentication
CORS
JSON-P
Tons of metrics and logs
NO Business Logic!

19. #TwitchDevDay
Where we’re going

20. #TwitchDevDay
Where we’re going
The future of Visage API Gateway
Webhooks:
First Announced Today!
Removes the need for polling
Adheres to the w3 WebSub spec
Once the Webhooks platform is online,
more Webhooks endpoints will get
added in upcoming months

21. #TwitchDevDay
Where we’re going
The future of Visage API Gateway
GraphQL:
One call to rule them all!
In full experiment mode, very promising
Used internally already
Architecture and principles mesh very well with our
world of microservices + data aggregating Gateway
Latency latency latency
The future of APIs in general (remember SOAP?), but
still a nascent technology
GraphQL summit on its 2nd iteration only

22. #TwitchDevDay
Where we’re going
The future of Visage API Gateway
Multi Region APIs:
Aka multi data centers
Serving API behind a mix of CDNs is not enough
Latency + Higher availability
Complex project with 100s of microservices
- Deploy complexity ! CI/CD
- Caching strategies
- Data store replication and policies
- Partial Failures and degradation modes

23. #TwitchDevDay
Where we’re going
The future of Visage API Gateway
New Helix endpoints, including new Twitch
features:
Games
VODs
Clips
More Webhooks
…
Continuous internal DevEx improvements:
Make it easier, safer and faster to add endpoints
The platform is never complete

24. #TwitchDevDay
Summary
The Twitch API journey
Visage, the API platform:
Enables Helix for 3rd party developers
Supports 1st party clients as well
Supports experiments of new technologies and
services
Is continuously improved (code and infra) for
internal and external DevEx

25. Thank you