Zalo is a messaging application with over 3 million users that exchanges 30 million messages per day. It uses a real-time system architecture with connection servers that allow for 1 million concurrent connections and 200,000 messages per second. The architecture is broken into independent layers including load balancing, web servers, searching, databases, and connection servers that use technologies like C/C++, Java, and event-based I/O. Lessons learned include avoiding HTTP long polling, optimizing for large not small data, and developing a custom memory allocator to avoid fragmentation issues.

1. Zalo Real-time System:
Architecture &
Experience
By Nguyen Quang Nam
Zing Senior Software Manager
Web-Technical

2. Content
• About Zalo
• Technology in Zalo
• Real-time System Architecture
• Experience
• Q&A

3. About Zalo

4. Features
• Communication: message (voice, text), call
• Social: “Nhật ký”, “Phòng trò chuyện”

5. Communication

6. Social

7. Updated stat
• Users: 3.0M total
• Messages: 30M per day

8. Technology in Zalo

9. Technology
• Programming languages: C/C++ and Java for
services at server-side
• Client platforms: IOS, Android, Symbian (S40, S60),
Windows Phone (soon)

10. Technology (cont)
Connection servers are key points of the Zalo real-time system
Free / Opensource Inhouse
Load balancing HA proxy ZaCacheProxy, ZaRouter
Web servers Ngnix, Jetty ZaHttp
Searching Elastic, Solr Prefix Search Service
Database PostgreSQL, MySQL ZaDB, List32, List64, …
Connection servers
ZaCommServer,
ZEventbus

11. Real-time System
Architecture

12. N-Tier

13. ZaCommServer

14. ZaCommServer (cont)
• Programming language: C/C
• IO event mechanism: Epoll (libevent)
• Binary protocol with authenticated connections &
encrypted messages

15. ZaCommServer (cont)
• Capacity per one box:
• 1M concurrent connections tested
• 200K throughput messages per second

16. Experience

17. Break the thing into multi-layers
and
let they be independent each
other

18. Move biz out of high performance
services

19. Lessons Zalo learned
• Http long polling is not a good choice for a good
app
• Do not spend much time in optimizing small data
• Memory fragment is a really big trouble in high
performance services, develop our own memory
allocator lib instead of using default system glibc

20. Q&A
Contact: Nguyen Quang Nam
namnq@vng.com.vn
http://me.zing.vn/nam.nq