Both API Gateways and Service Meshes offer similar features and capabilities but are each implemented on a different axis in your application’s data plane. Draw those capabilities in a Venn diagram, you’ll see a lot of overlap between the API Gateways and Service Meshes. This is NOT a talk about the overlap. This is a talk about the things outside the overlap and why they matter (#HereBeDragons). The talk focuses on three questions around North/South & East/West Traffic. This talk is for a senior developer/architect on where to best inject cross-cutting concerns around security, observability and resiliency enabled by API Gateways and Service Meshes.
3. API Gateways and Service Meshes offer similar
capabilities (though they handle traffic at a different
axis in the request — often distinguished by terms like
North/South and East/West).
While there is extensive overlap, there are cases where
each excel.
9. The API gateway pattern describes an additional hop in the
network that every request will have to go through in order to
consume the underlying APIs. In this context, some people call the
API gateway a centralized deployment.
14. A service mesh provides a generic mechanism for intercepting
microservice communications. It enables us to transparently
introduce aspects such as security, routing, monitoring, and
testing with no changes to the service themselves — in fact,
reduce complexity within the service. Because of the distributed
approach, the service mesh is considered more decentralized
than an API Gateway approach.
15. Control Plane
Data Plane
Ingress Traffic Sidecar Proxy
Service A
Sidecar Proxy Egress Traffic
Service B
Control
Discovery
Certs
Config
17. API Gateway is responsible for the flow of requests between the
client and the services, aggregating multiple services and
creating and sending the final response to the client (often at a
perimeter). Service Mesh is responsible for the flow of requests
between services (often inside a perimeter).
19. API Gateway and Service Mesh have overlapping functionalities, such
as rate-limiting, security, service discovery, tracing, etc. but they work
on different levels and solve different problems.
22. While API Gateway and Service Mesh have overlapping
capabilities. Things get more complicated in the realm of
Kubernetes.
Ingress, Service Mesh Ingress, and API Gateways can also do very
similar things for North/South traffic.
26. API Gateway
Ingress
(
Gateway API
)
Pod Pod
North/South
Traffic
East/West
Traffic
Cluster
Sidecar
Sidecar
Kubernetes with Service Mesh & API Gateway
Envoy (a highly
performant
proxy) is often
used in all.
31. I think as an industry, frankly, we've done a poor job of using
consistent nomenclature here.
I think you're going to hear people say API Gateway, edge proxy,
Ingress controller, and in many ways, they're going to use them
interchangeably. I don't even know that I could honestly tell you what
the difference is because from my perspective, and I would consider
myself an expert here, I don't think there really is much of a difference.
I think it's more useful for me to come at it from the perspective of, in
modern internet architectures, I have an edge component that is
sitting between the internet and my backend systems. And I can call
this the Ingress or the API Gateway or the edge proxy, but it's better
to focus on the functionality.
-
Matt Klein, Creator Envoy
https://www.infoq.com/podcasts/matt-klein-envoy-gateway
33. API as a product that other developers, partners or
teams will consume.
Are your APIs a
product?
34. Examples of Product’s as an API
● Plaid: The company builds a data transfer network that
powers fintech and digital finance products. Plaid's product, a
technology platform, enables applications to connect with users’ bank
accounts. It allows consumers and businesses to interact with their
bank accounts, check balances, and make payments through different
financial technology applications
● Stripe: Stripe, Inc. is an Irish-American financial services and software
as a service
(
SaaS
)
company dual-headquartered in San Francisco,
United States and Dublin, Ireland. The company primarily offers
payment processing software and application programming interfaces
(
APIs) for e-commerce websites and mobile applications.
● Twilio: Twilio is an American company based in San Francisco,
California, which provides programmable communication tools for
making and receiving phone calls, sending and receiving text
messages, and performing other communication functions using its
web service APIs.
Are your APIs a
product?
35. • Create your services
• Create a self-service portal for developer self registration.
• Create a billing/monitoring service
• Create an AuthN and AuthZ for access / control
• Generate API keys for users
• Create a dashboard (reports) for API usage
• Produce documentation
• Load balance, Rate Limit, Proxy, & apply Policies
To make a sellable API, what do you need to do?
Are your APIs a
product?
36. Service
API Gateway API Management
Dev Portal
Developer Consuming
Product API
AuthN/AuthZ
Dashboard
(reports)
Self-service portal
Docs
API Keys
Load Balancing
Rate Limit
Are your APIs a
product?
Service Service
38. External calls tend to be more focused on the
perimeter and often have different sets of non-
functional requirements.
Internal calls tend to be more homogeneous and
often more focused on the added network
between services (often concerned with
reliability and resiliency).
Is your Traffic Internal vs External?
Is your Traffic
Internal or
External?
39. The fundamental difference between
edge routing (north/south) and internal
(east/west) routing is that with the
edge, you don’t control the client.
-
Richard Li, CEO Ambassador Labs
Is your Traffic
Internal or
External?
40. Is your Traffic
Internal or
External?
Client Server
1
Requests protected resource
2
Presents cert (server.cer)
CA
3
Verifies cert (server.cer)
4
Presents cert (client.cer)
5
Verifies cert
(client.cer)
6
Returns protected Resource
41. Is your Traffic
Internal or
External?
https://www.datacenterdynamics.com/en/news/spotify-sees-hour-long-global-
outage-forgot-renew-certificates/
42. Ingress Traffic Sidecar Proxy
Service A
Sidecar Proxy Egress Traffic
Service B
Data Plane
Control
Discovery
Certs
Config Control Plane
43. Ingress Traffic Sidecar Proxy
Service A
Sidecar Proxy Egress Traffic
Service B
Data Plane
Control
Discovery
Certs
Config Control Plane
44. Observability
At the sidecar, you have insight into the components of a
request.
At the gateway, you have insight into total time of the request
coming in and out of your network.
Is your Traffic
Internal or
External?
48. !
Warning: No silver bullets.
Out of the box observability from a service mesh
gives you a leg up, but it’s rarely enough to
effectively operate a large microservice
environment alone. Providers repeatedly point out
developers still need to instrument code for
effective observability strategies.
Is your Traffic
Internal or
External?
49. Similar Capabilities, Different Lens
● Deployment Patterns
(
Canaries or blue/green deployment)
● Resources used
(
OpEx vs CapEx)
● Cache strategies
● Simplify / Consolidate Architecture
● Defense
(
Bot blocking / WAF
)
Is your Traffic
Internal or
External?
50. Are you okay trading complexity
for flow on a development team?
API Gateways are a simple piece of infrastructure. Service
Mesh are not.
Where are you
okay with
Complexity?
51. Let’s Build an App
iOS, Android Web
Mobile BFF Web BFF
Order Detail
Order Detail
Consumer
Service
Order
Service
Delivery
Service
Data Data Data
Where are you
okay with
Complexity?
52. Let’s Build an App
Resiliency
Observability
Security
Service
Discovery
Deployment
Patterns
Business
Logic
Frameworks Data
Developer
Where are you
okay with
Complexity?
53. Let’s Build an App
Resiliency
Observability
Security
Service
Discovery
Deployment
Patterns
Business
Logic
Developer
Frameworks
Data
Value
Chain
Where are you
okay with
Complexity?
Custom/Unique Commodity
54. Let’s Build an App
Resiliency
Observability
Security
Service
Discovery
Deployment
Patterns
Business
Logic
Developer
Frameworks
Data
Value
Chain
Where are you
okay with
Complexity?
Custom/Unique Commodity
55. Let’s Build an App
Business
Logic
Developer
Frameworks
Data
Value
Chain
Observability
Where are you
okay with
Complexity?
Custom/Unique Commodity
56. Operating things a service mesh comes with
complexity. It is worth it for many
organizations to tackle that complexity and
reduce the cognitive load on development
teams if it improves overall flow. Where are you
okay with
Complexity?
59. API Gateway
Service A Service B Service C
Cache
Logging
Rate limiting
SSL Offloading
Routing
Deploying API Gateway
API Gateway Control Plane
North/South
Traffic
Layer 7
Data Loss
Prevention
WAF
60. Deploying a Service Mesh
Istio Gateway
Sidecar
Service A
Sidecar
Service B
Sidecar
Service C
Sidecar
Service D
Sidecar
Service E
Sidecar
Service F
North/South
Traffic
East / West
Traffic
Tracing
Rate limiting
Circuit Breakers
Routing
Deployment
Patterns
Logging
Resiliency
Policy
61. Deploying API Gateway with a Service Mesh
Sidecar
Service A
Sidecar
Service B
Sidecar
Service C
Sidecar
Service D
Sidecar
Service E
Sidecar
Service F
North/South
Traffic
East / West
Traffic
Istio Gateway
API Gateway
Cache
Logging
Rate limiting SSL Offloading
Layer 7
Data Loss
Prevention
WAF
Tracing
Rate limiting
Circuit Breakers
Resiliency
Policy
62. Deploying API Gateway into a Service Mesh
API Gateway
Sidecar
Sidecar
Service A
Sidecar
Service B
Sidecar
Service C
Sidecar
Service D
Sidecar
Service E
Sidecar
Service F
North/South
Traffic
East / West
Traffic
Cache
Logging
Rate limiting
SSL Offloading
Layer 7
Data Loss
Prevention
WAF
Tracing
Rate limiting
Circuit Breakers
Resiliency
Policy
63. API Gateway
Sidecar
Service A
Sidecar
Service C
Sidecar
Service D
Sidecar
Service F
East / West
Traffic
API Gateway
Sidecar
Service A
Sidecar
Service C
Sidecar
Service D
Sidecar
Service F
East / West
Traffic
Internal API Gateway / Service Mesh
Tracing
Rate limiting
Circuit Breakers
Resiliency
Policy
Tracing
Rate limiting
Circuit Breakers
Resiliency
Policy
API as Product
64. API Gateway & Service Mesh
Some Questions to Think About
Deployment Patterns
Agenda
65. API Gateways and Service Meshes offer similar
capabilities (though they handle traffic at a different
axis in the request — often distinguished by terms like
North/South and East/West).
While there is extensive overlap, there are cases where
each excel.
