GopherCon India 2017 talk on "Building High Performance APIs In Go Using gRPC And Protocol Buffers"

1. Building High Performance
APIs in Go with gRPC and
Protocol Buffers
Shiju Varghese
medium.com/@shijuvar
GopherCon India 2017

2. Agenda
• gRPC
• Protocol Buffers
• Building APIs with gRPC

3. Monolithic Architecture
Catalog
Management
Customer Accounts
Orders Management
Payment
Monolithic
Database
Monolithic E-Commerce App
Shipment
eCom Store Web

4. Moving to Microservices
eCom Store Web
Catalog
Management
Customer Accounts
Order Management
Payment
Shipment
Store DB
Catalog DB
Customer DB
Order DB
Payment DB
Shipment DB

5. Communications on
Microservices
• Inter-process communication between
Microservices
• Serialisation and Deserialisation of Messages
• Scaling APIs into billions of APIs calls
• Building streaming APIs
• RESTful with JSON or RPC with Binary
Encoding?

6. Why Not REST
• Uses HTTP/1.x; Separate TCP Connection per
request
• Text on the wire; Not performance efficient
• Harder API Evolution
• Not Domain-Specific, Not Strongly-Typed
• Lack of Streaming capabilities

8. • High performance, open-source RPC framework
• Open source version of Google’s internal framework Stubby
• Uses Protocol Buffers as the IDL
• HTTP/2 for transport
• Bi-Directional streaming
• RPC is Efficient, Domain-Specific and Strongly-Typed
• Works across languages and platforms

9. • Google's language-neutral, platform-neutral,
extensible mechanism for serialising structured
data
• IDL - Describe once and generate interfaces for
multiple languages
• Structure of the Request and Response
• Binary format for network transmission
• Supports multiple languages
Protocol Buffers

10. Communication between
gRPC Server and Client app

11. Types of RPC Methods
• Simple RPC
• Server-side streaming RPC
• Client-side streaming RPC
• Bi-directional streaming RPC

12. gRPC Workflow
ProtoBuf
Definitions
protoc
Compiler
Go
Ruby
Java
gRPC
Server
gRPC
Client
Define1 Compile2
Implement3
Generate Code
protoc --go_out=plugins=grpc

13. syntax = "proto3";
package order;
service OrderService {
// A simple RPC
rpc CreateOrder (Order) returns (OrderResponse) {}
}
message Order {
string id = 1;
string status = 2;
int64 created_on = 3;
message OrderItem {
string code = 1;
string name = 2;
float unit_price = 3;
int32 quantity = 4;
}
repeated OrderItem order_items = 4;
}
message OrderResponse {
string order_id = 1;
string error = 2;
}

14. type server struct {}
func (s *server) CreateOrder(ctx context.Context, in *pb.Order)
(*pb.OrderResponse, error) {
// Implementation goes here
}
func main() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
// Creates a new gRPC server
s := grpc.NewServer()
pb.RegisterOrderServiceServer(s, &server{})
s.Serve(lis)
}
gRPC Server

15. gRPC Client
conn, err := grpc.Dial(address, grpc.WithInsecure())
if err != nil {
log.Fatalf("Unable to connect: %v", err)
}
defer conn.Close()
client := pb.NewOrderServiceClient(conn)
createOrders(client)

16. func createOrders(client pb.OrderServiceClient) {
order := &pb.Order{
Id: uuid.NewV4().String(),
Status: "Created",
CreatedOn: time.Now().Unix(),
OrderItems: []*pb.Order_OrderItem{
&pb.Order_OrderItem{
Code: "knd100",
Name: "Kindle Voyage",
UnitPrice: 220,
Quantity: 1,
},
&pb.Order_OrderItem{
Code: "kc101",
Name: "Kindle Voyage SmartShell Case",
UnitPrice: 10,
Quantity: 2,
},
},
}
resp, err := client.CreateOrder(context.Background(), order)
}

17. Installation
# Install Protocol Buffers compiler
$ brew install protobuf (macOS)
(Or download from https://github.com/google/protobuf/releases/tag/v3.0.0)
# Install Go Plugin
$ go get github.com/golang/protobuf/protoc
$ go get github.com/golang/protobuf/protoc-gen-go
# Install Go RPC Framework
$ go get google.golang.org/grpc

18. medium.com/@shijuvar
linkedin.com/in/shijuvar
email: gophermonk@gmail.com
THANK YOU
Articles:
Building High Performance APIs In Go Using gRPC And Protocol Buffers
Benchmarking Protocol Buffers, JSON and XML in Go