4. In 2013, according to comScore,
we have reached to %69 of
Turkish Internet Users
Nokta in Nutshell
Source; , Media Metrix, Jan-June 2013, Total Number of Internet Users
12. Proven Track Record in Live
Digital Events
• Vodafone high school music
contest was broadcast live on
Izlesene and Twitter at the
same time through Izlesene
portal. (Izlesene the first
player in Turkey to broadcast
live on Twittter)
• Can Bonomo concert was
broadcast live on Izlesene
channel and got in Worldwide
& Turkish TT list on Twitter on
a soccer night.
13. NOKTA MOBILE ECOSYSTEM / EMPOWER – GROWING MOBILE AUDIENCES
1.4 M+ 12 M+
10 M+ 25:00 min
14. LOREM IPSUM
15 M+ Monthly
Visitors
350M+ Monthly
Video Views
60M+ Monthly
Visits
Time Spent
16:00 min
NOKTA WEB ECOSYSTEM
15. CDN & Private CDN
• Why Content Delivery Network? (CDN)
• CDN caches contents to the nearest POP Server
• Pulls popular content from origin, caches temporarily & saves
bandwidth, saves cost
• Reduces network latency & improves user experience
• Highly Available, when the nearest pop is dead, streams from other
pops
• Why Adaptive Stream?
• Stream quality is selected dynamically by player
• When the bandwidth to the client on any network segment is
saturated, the client reduces the quality and vice versa
• Less buffer emptying occurs and improves user experience
• Segmented, Less VOD (video on demand) start & seek time
• Less storage area for mpeg dash packing
16. CDN Services & Features
• Video & Audio Content Streaming (Both VOD and Live)
• Adaptive Stream, Mpeg Dash (Desktop, Android)
• Adaptive Stream, HLS (IOS)
• Progressive Stream, mp4 (Legacy Platforms)
• RTMP Streaming
• Static Content Caching
• Images, css, js and any static content
• Real time image resize
• Dynamic web site accelerator
• Traffic Management
• DC & Geography Awareness
• Carrier sensitivity
• Rule based
• Low cost
• Secure Links
• Transfer Rate Limits
• Monitoring
• Highly Available
• Player
• Flash player, html5 player, ios app player, android player
• Can integrate with big data analysis service
17. Video Transcoding Services (VTS)
• Fast parallel content download from source
• Parallel mpeg dash, hls and mp4 transcoding
• Configurable resolutions and bitrates, H264 and AAC encodings
• Up to 4k videos transcoded
• Image capture from video content within configurable intervals
• Storyboard image creation
• Optional three replica, highly available object storage service for
transcoded videos and images
• Optional audio fingerprint service
• Fully integrated with CDN service
18. Big Data Analyzer (BDA)
• Real Time and Batch Analysis
• Video views with several breakdowns (platform, category, ownership, etc)
• Video completion rate
• Player buffer rate
• Player connection error rate
• Last mile analysis
• Page view, visit, ctr, etc.
• Any definable metric
• Real time comparison with previous day and previous week day
• A/B Testing
• Data driven solutions (Recommendations, Personalization, Search)
19. Some Highlights
• 65 Gb/s peak traffic
• 150k visit/s at whole Nokta products
• 20k video start/min
• 1+ billion video start/month
• 150k ad view/s
• image load time < 100 ms , ad load time < 20 ms
• 25 MB/s log, 150 GB/day log, 10k msgs/s
• 25 k req/s at load balancer
• 5000 Video/day transcoded to 6 different formats
• 300+ VM on private cloud
• 150+ server
• Layer 1, 2, 3 network administration, monitoring and maintenance
• 200+ metrics on BDA
• 400+ hosts, 10000+ items monitored with 2000+ triggers
• Tubitak Research Projects
• Kullanıcı Video İzleme Davranışlarından Profil Tanıma Sistemi
• Eşle İndirge ile Bulut Üzerinde Yığın Video İşleme Sistemi
20. Behind the Scenes, Storage
• Open Source Distributed Object & Block Storage Ceph
• Highly available, infinitely scalable, DC & rack aware, data redundancy with
replication
• Current infra works on 45 servers
• Each server has 12 disk slot and each slot has 3.5 inch 4 TB NL sas disk and
2x2.5 inch 300 GB OS disk
• No RAID configuration for data disks, RAID 1 for OS disks
• Each server has 1x10 Gb/s, 2x 1 Gb/s, 1x 100 Mb/s interface
• Each data has three replica
• S3/Swift API for Object Storage, qemu end kernel driver for block storage
• Works as object storage for videos, images, statics etc
• Works as networked block storage for os & data volumes of virtual servers
working on Private Cloud
21. Behind the Scenes, Private Cloud
• Based on Open Stack
• Current infra works on 20 servers
• 2-slot 16-core Xeon servers 64-128G RAM
• 1x10 Gb/s, 2x 1 Gb/s, 1x 100 Mb/s network interface
• Multi-tenant networking
• KVM based OS virtualization
• Fast VM provisioning
• Cobbler and Puppet for bare-metal provisioning and Configuration
Management
22. Behind the Scenes, CDN
• 12-14 Gb/s attainable throughput per server for high-cardinality static
content. More for viral content or live streams
• Based on nginx with lua (openresty suite), managed in-house
• Fast SAS disks for caching
• 2x 10Gb/s, 2x1 Gb/s, 2x100 Mb/s network interface
• Highly Available
• Traffic Management
• DC & Geography Awareness
• Carrier sensitivity
23. Behind the Scenes, Transcoding
• Resource scheduling with Apache Zookeeper
• Services have been developed in Python
• Whole extensive video & audio formats supported
• Content is downloaded and transcoded in parallel.
• Deployed on 15 nodes
• Configurable resolutions & bitrates & formats, H264 and AAC encodings
• Up to 4k videos are transcoded
• Images are extracted from video at configurable interval
24. Behind the Scenes, Big Data Analysis
• 2x 20 server cluster
• One cluster for sensitive ads data
• One cluster for analytics purpose
• HDFS, Hadoop, Hbase, zookeeper, flume, storm, spark, impala, kafka are
used
• Services are coded using Scala and Java
25. Behind the Scenes, Monitoring
• BDA product is used for business metrics
• Grafana is used for Visualizing Time Series data with back end Hbase
• Zabbix is used for Network and Hosts
• OMSA is used for physical host monitoring
26. Q & A
26
Hakan Kocakulak, CTO of Noktacom Medya İnternet Teknolojileri
hakankocakulak@nokta.com
http://tr.linkedin.com/in/hakankocakulak
