This document provides an overview of the Big Data CDR Analyzer project. The project aims to develop a system called Kanthaka that can analyze large volumes of Call Detail Records (CDR) to select eligible mobile users for promotional offers in real-time. Kanthaka will use Cassandra, a NoSQL database, and be able to process 30 million records per day with results within 30 seconds. The document compares technologies, describes the architecture, discusses risks and remedies, and lists deliverables including a research paper and final report.

1. © 2012 University of Moratuwa
Big Data CDR
Analyzer
“The Next Generation Mobile Promotions”
Project Supervisors- • 080201N – M.K.P.R. Jayawardhana
Mr. Thilina Anjitha – hSenid • 080254D – P.K.A.M. Kumara
Dr.Shahani Markus Weerawarana
• 080331L – W.D.A.I. Paranawithana
• 080357V – T.D.K. Perera

2. © 2012 University of Moratuwa
OVERVIEW
 Background
 Current Situation
 Scope and Assumptions
 Kanthaka – big data CDR Analyzer System
 Technology Comparison
- Map Reduce
- NoSQL Databases
 Architecture
 Risks and Possible Remedies
 References

3. © 2012 University of Moratuwa
Background
Mobile Promotions

4. © 2012 University of Moratuwa
CURRENT SITUATION
• Promotions based only on their network
usage
• Use only active call switch for triggering
promotions
• No way of analyzing and processing high
volume CDR records
• No efficient CDR analyzing method
• No access to historical data
• Complex rules not supported &@$*#

5. © 2012 University of Moratuwa
TO RESCUE
 Selecting eligible users for both commercial
organizations based and network usage
based promotions.
Eg- giving 20% discount for pizza lovers within age group 16-40 who
have called pizza hut more than 5 times a month
 High volume CDR analysis.
 Near real time selection of eligible users for
promotions.

6. © 2012 University of Moratuwa
 CDR Analyzer system which
 can process 30 million records per day
 can produce results within 30 seconds
 provides a GUI to define dynamic rules
 can be used to offer real-time sales
promotions for mobile subscribers

7. © 2012 University of Moratuwa
This location information retrieving from Location Based System(LBS) can
be replaced with any other information retrieving such as subscriber age
from the Customer Relationship Management system to support attractive
promotions.

8. © 2012 University of Moratuwa
SCOPE AND ASSUMPTIONS
SCOPE
 30 M  30 M
 Multiple Rules  Multiple Rules
 Offer Promotion  Select eligibilities for
promotion only
Real system operation Operation expect by Kanthaka

9. © 2012 University of Moratuwa
ASSUMPTIONS
 CDR records can be only in .CSV format.
 Event type can be in different types like
SMS, Voice call, MMS, USSD, Top-up,
GPRS, LBS.
 CDR can be received as batches to the
system asynchronously.
 Only 6 attributes out of many attributes will
be considered during processing.

10. © 2012 University of Moratuwa
TECHNOLOGY COMPARISON

11. © 2012 University of Moratuwa

12. © 2012 University of Moratuwa
YCSB BENCHMARKS
 With more big users, active mailing lists, most
promising technologies (secondary index,
counters) best to try out is Cassandra.

13. © 2012 University of Moratuwa

14. © 2012 University of Moratuwa
TECHNOLOGY SELECTION
TECHNOLOGIES LEFT BEHIND TECHNOLOGIES SELECTED
 Complex Event  NoSQL DB - Cassandra
Processing engines(CEP)
 No persistency
 Rules Engine
 More layers  More
latency
 Hadoop - latency
 NoSQL DB- Hbase,
MongoDB, Hive

15. © 2012 University of Moratuwa
BRIEF ARCHITECTURE OF ‘KANTHAKA’
Promotion definition
Cassandra Cluster
Pre-processing unit

16. © 2012 University of Moratuwa
TEST RESULTS IN SINGLE NODE

17. © 2012 University of Moratuwa
TEST RESULTS IN TWO NODE- CLUSTER

18. © 2012 University of Moratuwa
CLUSTER BETTER IN HIGH LOADS

19. © 2012 University of Moratuwa
RISKS AND POSSIBLE REMEDIES
 NoSQL databases
High performance More memory
 Use an external cluster with descent memory
 Concurrency Issues Handling
Low speed  Locking database
Use shadow copy
 Handling sudden peaks
Should have an auto balancing mechanism ready

20. © 2012 University of Moratuwa
FINAL DELIVERABLES
Big Data CDR Analyzer system
Research Paper
Final Report

21. © 2012 University of Moratuwa
REFERENCES
 B. F. Cooper, A. Silberstein, E. Tam, R.
Ramakrishnan, and R. Sears,
“Benchmarking cloud serving systems with
YCSB,” 2010, pp. 143–154.
Visit us at Kanthaka

22. © 2012 University of Moratuwa
Thank you
Manoj
Dhanika Amila
Pushpalanka