MSB to Support Carrier Grade ONAP Microservice Architecture with Service Mesh

1. MSB to Support for Carrier Grade ONAP
Microservice Architecture
Huabing Zhao, PTL of MSB Project, ZTE

2. 2
ONAP Architecture Principle: Microservices
ONAP Architecture Principle: ONAP modules should be designed
as microservices: service-based with clear, concise function
addressed by each service with loose coupling.
In Amsterdam Release, MSB provides infrastructure to support ONAP
Microservices architecture.
 Registration/Discovery
 API Gateway
 JAVA SDK

3. 3
MSB Amsterdam Overview
• Registry : Service information storage, MSB uses Consul as the service registry.
• MSB Discovery: Provides REST APIs for service discovery and registration
• API Gateway: Provides service request routing and load balancing . It can be deployed as an
External Gateway which is responsible for exposing ONAP microservices to external system or an
Internal Gateway which can be used for the inter-service communication inside ONAP
• MSB SDK: Java SDK for point to point communication
MSB
MSB
Discovery
External service Internal service
External Gateway Internal Gateway
Registry
ONAP Microservices
Microservice A
Microservice B Microservice C
2 Service discovery
3 Service request
2 Service request
3 Route request to service provider
Desktop
OSS/BSS
Mobile
MSB
Discovery
External
Gateway
Internal
Gateway
OOM
(Kubernetes)
1 Register services Demo
(Heat Template)
1 Register services
MSB SDK

4. 4
MSB Amsterdam Design Principle
Providing Microservice Infrastructure for ONAP in a transparent way, with no or minimum modification on the
application side. This principle is reflected in the design and implementation of the MSB components.
Example 1: Service registration by Kube2MSB Registrator:
OOM(Kubernetes) deploy/start/stop ONAP components.
 Registrator watches the kubernetes pod event .
 Registrator registers service endpoint info to MSB. It also updates the service info to MSB when ONAP components are
stopped/restarted/scaled by OOM
MSB
MSB
Discovery
External service Internal service
External Gateway Internal Gateway
Registry
ONAP Operations Manager
ONAP
A&AI
Kube2msb registrator
1 Pod Even (Start, Stop etc.)
Policy
SDC
SO
VF-CVF-C
2 Register Service
Update Service k8s
deployment
specs
(annotation)
No registration codes needed on the
application side

5. 5
MSB Amsterdam Design Principle
Providing Microservice Infrastructure for ONAP in a transparent way, with no or minimum modification on
the application side. This principle is reflected in the design and implementation of the MSB
components.
Example 2 : Inter-service communication via Internal Gateway
MSB
MSB
Discovery
Internal Gateway
Registry
Service A
MSB
Discovery
Internal Gateway
Cache
Service AService A
Service AService AService B
Service AService AService C
1 Service request 2 Route request to service provider
Internal Gateway handles the communication details
including service discovery, load balancing, retries, etc.
Multiple Internal Gateways can be deployed as a cluster to
share the workload, which is a similar concept to service
mesh, but not one proxy for each service instance.

6. 6
OMSA-ONAP Microservice Architecture
External API Gateway
Service Discovery
Async Message Bus
Kafka
监控采集
GatewayDiscoveryMicroservicesInfrastructure
Registration
ServiceA
ADF JAVA ADF Go ADF
HTTP/JSON
RPC
agent
Governance
Configure
Management
API
Management
Monitoring
Distributed
Tracing
GRPC/protobuf
MetricsActiveMQ
Bare Metal
Python ADF
Registration
Logging
Orchestration
Update/Change
Deployment
Scaling/Healing
Instance 1
agent
Virtual Machine Containers Clouds
sidecar
Policy/Security
sidecar
Instance 2
Instance n
ServiceB
Instance 1
Instance 2
Instance n
OMSA is the vision of ONAP Microservice
Architecture to support carrier-grade
requirements of ONAP microservices,
which includes service
registration/discovery, service
communication, API gateway, service
orchestration, service governance and
service monitoring, etc.
Note: this diagram is a functional view of
OMSA, which is not mapped to specific
projects

7. 7
Enhance Microservice Infrastructure to Support
Carrier Grade ONAP
 Reliability and security of RPC between microservices
 Visibility of microservices(metrics, distributed tracing)
 Manageability of microservices(policy, API management,
Configure management)
 Performance(Cache, Latency aware load balancing)
MSB plans to provide service mesh to enhance

8. Amsterdam
Kubernetes pod
Brief Introduction of Service Mesh

9. App
container
Kubernetes pod
Amsterdam

10. App
container
TLS
Kubernetes pod
Amsterdam

11. App
container
TLS
retries
Kubernetes pod
Amsterdam

12. App
container
TLS
retries
metrics
Kubernetes pod
Amsterdam

13. App
container
TLS
retries
metrics
Kubernetes pod
Amsterdam

14. App
container
TLS
retries
metrics
Kubernetes pod
Amsterdam

15. Istio
Beijing
TLS
retries
metrics
Proxy
Kubernetes pod
App container

16. Istio
TLS
retries
metrics
Proxy
App container
client server
registry
1. register2. lookup
3. call
client server
registry
1. register3. lookup
2. call
Proxy
4. forward
The remote communication logic is move into the proxy and deployed as “sidecar”
Developers only need to take care of the business logic
Separation of concerns

17. A service mesh is a dedicated infrastructure layer
for handling service-to-service communication.
It’s responsible for the reliable delivery of
requests through the complex topology of services
that comprise a modern, cloud native application.
In practice, the service mesh is typically
implemented as an array of lightweight network
proxies that are deployed alongside application
code, without the application needing to be
aware.
-William Morgan
Network of Proxies: Service Mesh

18. 18
MSB Plan for R2 and Beyond
 Stability and Reliability: Reliable communication with retries and
circuit breaker
 Security : Secured communication with TLS
 Performance: Latency aware load balancing with warm cache
 Observability: Metrics measurement and distributed tracing
without instrumenting application
 Manageability: Routing rule and rate limiting enforcement
 Testability: Fault injection to test resilience of ONAP
Provides Service Mesh to support Carrie Grade, Microservice-Based ONAP

19. Service A
MSB SidecarControl Plane
MSB Architecture for R2 and Forward
MSBDiscovery
Prometheus
zipkin
OtherAdapter
Auth
Service A
MSB Sidecar
Discovery
Load Balancing
Circuit Breaker Metrics
Tracing
Service A
MSB Sidecar
Service B
MSB Sidecar
Service A
MSB Sidecar
Service A
MSB Sidecar
Service C
MSB Sidecar
MSBAPIGateway
ServiceCatalog
SwaggerUI
Routes Telemetry TLS Certs
 MSB Discovery is plugged into Pilot as
a service discovery source
 MSB Internal API Gateway is
enhanced as a sidecar proxy to
address the Carrier Grade challenges
 It can be deployed as the Internal API
Gateway that is a cluster shared by
multiple service instances or the sidecar
that is one per service instance
 MSB API Gateway is used to expose ONAP
services to external system
 Add MSB Portal to control plane to
provide service Catalog ,swagger UI of
Restful API, service mesh configuration
(long term goal),etc.
MSB PortalMixer
Kube2MSB
Pilot
External
requests