NGINX.conf 2016 - Fail in order to succeed ! Designing Microservices for failure with NGINX

Fail In Order
to Succeed
Designing Microservices
for failure with NGINX
Dragos Dascalita Haut
Project Lead, Adobe I/O

#nginx #nginxconf
If you didn’t fail with microservices at least once you
didn’t really try anything new!
2

#nginx #nginxconf
The harder you push the system,
the harder it pushes back
3
Arie De Geus

#nginx #nginxconf9
Some reasons for failures
1. A client that misbehaves
2. A spike in demand
3. DDoS
4. A failure in one component
generating a cascading effect

Trafﬁc Management
with
NGINX

OPENRESTY
• Nginx Lua Module
• Nginx Redis
• Headers more
• Set misc
• LuaJIT
• ….
API Gateway Modules
• Request Validation
• Throttling & Rate Limiting
• HTTP Logger
NGINX
• Upstream
• HTTP Proxy
• PCRE
• SSL
• ….
API GATEWAY :" …TAKE ONE OF THE MOST POPULAR WEB
SERVER AND ADD API GATEWAY CAPABILITIES
TO IT…"

#nginx #nginxconf12
How we started
ngx_http_limit_req_module

#nginx #nginxconf13
1. limit_req_zone $binary_remote_addr zone=gold:10m rate=300r/m;
2. limit_req_zone $binary_remote_addr zone=silver:10m rate=30r/m;
4. server {
5. ...
6. location /login.html {
7. limit_req zone=silver burst=5;
8. ...
9. }
10.}
Limit the rate of requests
Limit to 30 requests per
minute

#nginx #nginxconf14
1. limit_conn_zone $binary_remote_addr zone=conn_zone:10m;
3. server {
4. ...
5. location /store {
6. limit_conn conn_zone 10;
7. ...
8. }
9. }
Limit the number of connections
Limit to maximum 10
connections for each
client IP address

#nginx #nginxconf15
Setup
api gw
api gw
Service A
Service A
Service B
Service B
Service B
Load Balancer Tier Microservice Tier

#nginx #nginxconf16
NGINX
NGINX
Service A
Service A
Service B
Service B
Service B
Load Balancer Tier Microservice Tier
How to limit
ServiceA to
10 r/m
cross multiple
NGINX
nodes ?
Problem

#nginx #nginxconf17
NGINX
NGINX
Service A
Service A
Service B
Service B
Service B
How to limit
ServiceA to
10 r/m
cross multiple
NGINX
nodes ?
Problem
What happens
when a new
node comes
up ? NGINX

#nginx #nginxconf18
NGINX
NGINX
Service A
Service A
Service B
Service B
Service B
How to limit
ServiceA to
10 r/m
cross multiple
NGINX
nodes ?
Problem
What happens
when a new
node comes
up ? NGINX
… or goes away

#nginx #nginxconf19
Pros: Cons:
Easy to conﬁgure
Easy to manage
Works well for a single node
Can’t deﬁne rules at a cluster level
Can’t apply dynamic rules per
location
i.e. allow one app to send
1000 requests and another
10 requests
ngx_http_limit_req_module

#nginx #nginxconf20
Building a distributed
solution in NGINX

#nginx #nginxconf21
Requirements
1. Work in a distributed environment.
2. Async. Don’t add extra latency to the request when checking quotas.
3. High-performance. Sustain hundreds of thousands of requests/
second.
4. Adaptive. NGINX instances may come up or may go away at any time.
5. Fail-safe. In the event the solution doesn’t function then all trafﬁc
should be permitted until it recovers.

#nginx #nginxconf22
Assumptions
1. The intent is rather to allow than to block
• the focus is to ensure a fair usage policy
2. Favor performance instead of precision
• rather allow a small % over the limit instead of adding latency to the request

#nginx #nginxconf23
Challenges
1. Maintain consistent distributed counters across the cluster
2. Asynchronous and non-blocking

#nginx #nginxconf24
Option #1
NGINX NGINX
Maintain consistent counters across the cluster
Nodes inform
each other
about
their counters
NGINX
Challenges:
Chatty : more nodes, more
messages
Maintaining consistent distributed
counters is a complex problem
Increase NGINX’s complexity

#nginx #nginxconf25
Option #2
NGINX
NGINX
Brokered
Message
Queue
Tracking
Microservice
Usage data
Usage data
Usage data
What to BLOCK or SLOW DOWN / DELAY

#nginx #nginxconf26
Option #2
Challenges:
Maintain a Brokered Message
Queue. Is it needed ?
Maintain a new Microservice to
track the counters
Improvements:
Less chatty
Moved distributed counters
from NGINX into a Micro
service

#nginx #nginxconf27
Option #3
NGINX
NGINX
MQ
Tracking
Microservice
Pull usage data
Pull usage data
MQ
Challenges:
Embed a MQ with NGINX
Maintain a new Microservice
to track the counters
Auto discovery of NGINX
Nodes
Improvements:
Non Brokered Message
Queue
Moved distributed counters
from NGINX into a Micro
service

#nginx #nginxconf28
Selecting a Message Queue
CANDIDATE /
LANGUAGE
PROS CONS
Apache Kafka /
Java, Scala
• rated as highly performant, sustaining 2M
messages
• durable, messages being written to disk ﬁrst
• Zookeeper dependent
• Brokered
• Maintenance complexity
•
ActiveMQ /
Java
• popular
• supports STOMP , AMQP, MQTT, XMPP
• Spring integration
• Brokered
RabbitMQ /
Erlang
• supports STOMP , AMQP, MQTT, XMPP
• community support
• Brokered
• slower than ZeroMQ
nanomsg
• performant socket lib
• it promises a cleaner API than ZeroMQ
• in beta when we analyzed it
• no XPUB/XSUB Proxy
ZeroMQ
• around since 2007
• brokerless, designed for high throughput/low
latency scenarios
• embeddable in Nginx with C/C++/Lua
bindings
• pure Java implementation through JeroMQ
• no auto-discoverability - need to use a Proxy
( XPUB/XSUB )

#nginx #nginxconf29
Moving ahead with Option #3 and ZMQ
NGINX
NGINX
MQ
Tracking
Microservice
Pull usage data
Pull usage data
MQ

#nginx #nginxconf30
Integrating ZeroMQ with NGINX
NGINX Master Process
ZeroMQ Adaptor Process
NGINX
Worker
NGINX
Worker
NGINX
Worker
NGINX
Worker
XSUB Socket
default - ipc:///tmp/ngx_queue_listen
XPUB Socket
default - tcp://0.0.0.0:6001
Tracking
Microservice
Pull usage data
What to BLOCK or
DELAY

#nginx #nginxconf31
NGINX Master Process
ZeroMQ Adaptor Process
NGINX
Worker
XSUB Socket
default - ipc:///tmp/ngx_queue_listen

#nginx #nginxconf32
https://github.com/adobe-apiplatform/

#nginx #nginxconf33
NGINX and Tracking Service
Tracking Service
Persists policies
Sends ACTIONS to the Gateway
based on the tracked information
Concerned with the business rules
managing throttling and rate limiting
Allows only private access to its API
NGINX
Enforces policies
Executes ACTIONS such as:
TRACK
BLOCK
DELAY
Unaware of the business rules
Serves public trafﬁc

#nginx #nginxconf34
Request ﬂow
API GATEWAY
/ NGINX
Microservice
ZeroMQ Adaptor
Gateway Tracking
Service
( GTS )
CLIENT
1
2
3
4
5
6
Asynchronous and Non-blocking

#nginx #nginxconf35
Integration with NGINX
Gateway Tracking Service API
API GATEWAY
/ NGINX
ZeroMQ Adaptor
Gateway
Tracking
Service
(GTS)
POST /api/policies/throttling
[{
"id": 10,
"softLimit": 2,
"maxDelayPeriod": 2,
"hardLimit": 5,
"timeUnit": "SECONDS",
"span": 10,
"lastModified": 1438019079000,
"domain": {
"$service_id": "echo-service"
},
"groupBy": ["$api_key"]
}]
POST /tracking/track
[{
"id" : 1079000,
"domain" : “echo-service;*;",
"format" : “$service_id;$api_key;",
"expire_at_utc" : 1472771655757,
"action" : "TRACK"
}]
POST /tracking/block
[{
"id" : 1079000,
"domain" : “echo-service;app1;",
"format" : “$service_id;$api_key;",
"expire_at_utc" : 1472771550,
"action" : "BLOCK"
}]
ZMQ MESSAGE
1472771550 1079000;echo-service;app1
<timstamp> <rule_id>;<domain>
1
2
3
4

#nginx #nginxconf37
Local Setup
API GATEWAY
/ NGINX
Microservice
ECHO
ZeroMQ Adaptor
Gateway Tracking
Service
Reporting
Graphite
Grafana UI
TEST
RUNNER
1
2
3
4
5
6

#nginx #nginxconf38
Adding a throttling policy
[{
"id": 10,
"softLimit": 4,
"hardLimit": 12,
"span": 10,
"domain": {
},
}]
low watermark
specifying when
to start DELAYing
requests
high watermark
specifying when
to start BLOCKing
requests

#nginx #nginxconf39
[{
"id": 10,
"softLimit": 2,
"hardLimit": 5,
"span": 10,
"domain": {
},
}]
at what
time intervals
to enforce
this policy

#nginx #nginxconf40
[{
"id": 10,
"softLimit": 2,
"hardLimit": 5,
"span": 10,
"domain": {
},
}]
enforce the policy
for all requests
having
service_id = “echo-service”enforce the limits
for each
application

#nginx #nginxconf41
Deleting a throttling policy
DELETE /api/policies/throttling/<policy_id>
Listing all policies
GET /api/policies/throttling

#nginx #nginxconf42
Deﬁning an application plan
[{
"id": 10,
"softLimit": 2,
"hardLimit": 5,
"span": 10,
"domain": {
"$service_id": “echo-service”,
“$app_plan": “silver”,
}
}]
add in addition
an identiﬁed for the app.
$api_key variable could be
used as well

#nginx #nginxconf43
Throttle by HTTP Verb
[{
"id": 15,
"hardLimit": 5,
"span": 10,
"domain": {
"$service_id": “echo-service”,
“$request_method”: “POST”,
}
}]
request_method
is a built-in variable
in NGINX
Limits all POST request for “echo-service” to 5 requests/10 seconds

#nginx #nginxconf44
With a 3s delay
"softLimit": 4,
"hardLimit": 12
"hardLimit": 12
Without delay
DELAY vs BLOCK

#nginx #nginxconf45
Extending Tracking Service
to dynamically rewrite requests
Use the same method but instead of blocking or delaying
requests, rewrite them
Useful for beta testing without affecting the real trafﬁc

#nginx #nginxconf46
Adjust limits dynamically
Measure by QoS (i.e. response time )
Measure by the Capacity of the service
if there aren’t so many consumers allow the current ones to use the remaining capacity
Enhancing Tracking Service

#nginx #nginxconf
Thank You
48
https://www.linkedin.com/in/dragos-dascalita-haut
@dragosche

NGINX.conf 2016 - Fail in order to succeed ! Designing Microservices for failure with NGINX

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à NGINX.conf 2016 - Fail in order to succeed ! Designing Microservices for failure with NGINX

Similaire à NGINX.conf 2016 - Fail in order to succeed ! Designing Microservices for failure with NGINX (20)

Dernier

Dernier (20)

NGINX.conf 2016 - Fail in order to succeed ! Designing Microservices for failure with NGINX