This document provides an overview of a workshop on streaming video, analyzing it in real-time, and sharing it using Amazon Web Services. The workshop objectives are to work through the video ingestion pipeline, focus on capturing, playback, and machine learning-driven analytics of video, and demonstrate an end-to-end solution. Activities include capturing live video, playing video back using HTTP Live Streaming, analyzing video using Amazon SageMaker, and using services like Amazon Kinesis Video Streams, Amazon SageMaker, and Amazon Rekognition Video.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Stream Video, Analyze It, and Share It
in Real Time
Adi Krishnan
Head of Kinesis Video
Amazon Web Services
A N T 3 5 7
Girish Sood
Sr. Product Manager
Amazon Web Services
Vijay Joshi
Software Engineer
Amazon Web Services
Mitchell Loeppky
Software Engineer
Amazon Web Services

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Session Abstract
Video is ‘big data’. Image sensors—in our smartphones, smart home devices, traffic
cameras—are getting internet-connected. Massive streams of video data are generated,
but currently not mined for real-time insights to drive businesses forward. In this
workshop, learn to capture, process, and analyze video streams. Build and configure
your camera device’s media pipeline to start streaming video into the AWS Cloud using
Amazon Kinesis Video Streams. Next, build and deploy your own machine learning (ML)
model in Amazon SageMaker to generate inferences about objects or activities in your
video stream. Finally, build a browser-based web player to view the video in Live and On-
Demand modes, including the analyzed video stream. In this workshop, you use Amazon
Kinesis Video Streams, Amazon SageMaker, Amazon Rekognition Video, and Amazon
Elastic Container Service (Amazon ECS).

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Workshop objectives
• Work through the video ingestion data pipeline
• Focus on capturing, playback, and ML-driven analytics of video
• Capture live video from your device and stream to AWS
• Playback video using HTTP Live Streaming (HLS) capability
• Analyze video using Amazon SageMaker
• Demonstrate how to use related services to solve typical problems
and leave with a working end to end demonstration

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Target audience
• Architects, software developers / engineers
• Interested in building video oriented workflows from devices and
integrating with ML-services
• Level 300 – Working knowledge of core AWS services
• Sample code will be provided
• A desire to learn

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What prior knowledge do I need?
• Working knowledge of some AWS foundational technologies:-
• AWS Identify and Access Management (IAM)
• AWS AWS CloudFormation
• Amazon CloudWatch
• Amazon ECS
• Your laptop that runs Windows, Mac, or Ubuntu Linux
• Note ML Services will be used and the model shared, not an in-depth
ML session however

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Workshop set-up steps
• Ensure you have a valid AWS account with appropriate access
• Ready access to your AK/ Secret AK (to help with SDK installs)
• Your laptop with Windows, MacOS, or Ubuntu Linux
• Download the ‘step-by-step’ instruction guides for the 3 modules from
the following links:
• Module 1: Capture live video (Producer): https://s3-us-west-2.amazonaws.com/kvsit-
beta-us-west-2/reinvent/producer.pdf
• Module 2: View live video (HLS Playback): https://s3-us-west-2.amazonaws.com/kvsit-
beta-us-west-2/reinvent/hls.pdf
• Module 3: Real-time video analytics with Amazon SageMaker (KIT): https://s3-us-west-
2.amazonaws.com/kvsit-beta-us-west-2/reinvent/kit.pdf

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Video is critical to many applications
Smart home Security
monitoring
Smart city
Industrial automation Computer vision

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building a video system is complex
• Cumbersome to rewrite code for various devices and development
environments
• Difficult to “elastically” scale to millions of devices
• Requires reliable support for cadence, latency, and jitter on streams
• Requires secure streaming and storage by default
• Need easy-to-use APIs to retrieve, replay, and process video at any scale

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Stream video from millions of devices
Easily build vision-enabled apps
Playback in live or on-demand mode
Durable, time-indexed storage
Secure and Fully managed
Amazon Kinesis video streams
Stream video in real-time for playback, storage, and analytics
Kinesis Video
Streams

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Kinesis video streams concept

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Module objectives
• Learn about ways to produce media into KVS with the producer SDK
• Discover basics of Gstreamer-based media pipeline construction
• Capture and stream video feed from your laptop / USB cam
• View video on the Amazon Kinesis Video Streams management console
• (Optional) Perform basic configuration changes such as FPS and/ resolution
changes

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kinesis Video Streams ingestion process
• Stream directly from (supported)
device
• Stream using a proxy/ gateway
that connects with device on the
same
• Both approaches use the KVS
producer SDK assets
Proxied Video
Stream
(Optional)
Customer
Producer
Producer SDK writes directly to
the video stream
RTSP Stream
Read by proxy and
send to stream

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Confidential
Kinesis Video Streams producer SDK
Easily connect and stream from camera sources
Producer SDK offers … … that enables developers to
• Flexible SDK for integration with on-device
hardware media pipelines
• Out-of-the-box integration with AWS auth
mechanism: SigV4 / AWS IoT certs
• Handles streaming Put API to stream
continuously in a reliable manner
• Add metadata to video fragments applied by
the device directly
• Build custom integrations with diverse
camera and device types
• Secure authN and authZ to connect devices
w/ image sensors to AWS
• Stream video data based on the preferred
transmission scenario
• Annotated video fragments enable cloud-
based analytics and advanced indexing

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kinesis Video Streams producer SDK
Easily connect and stream from camera sources
Platform independent layer (C)
OO wrapper layer (C++ and Java)
Android Win Linux MacOS Raspian
For app devs to install on target OS to build
producer applications
For camera-level integrators who want a high-
performance integration model to build their
camera platforms
Docker image
Docker images on target OSes for easy
getting started experience
Gstreamer
plugin
Gstreamer
plugin
Gstreamer
plugin
Gstreamer
plugin

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Module activities
• MacOS: Run pre-built binary of the producer SDK and start streaming from
built-in webcam
• Windows: Run pre-built binary of the producer SDK and start streaming from
built-in webcam. If webcam is inaccessible; request USB cam.
• Ubuntu: Run Gstreamer plug-in as a Docker container (for Windows/ Mac). If
internal webcam is inaccessible; request USB cam.
• Playback on KVS management console to view streaming content
• Change FPS/ Resolution options and see changes in playback –
Note: hardware media pipelines are finicky – so your mileage may vary.

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Gstreamer: Standard environment to construct media pipelines on the devices
• Popular media framework used in cameras / video sources for media pipelines
by combining modular plugins
• KVS GStreamer plugin simplifies integration existing GStreamer media pipeline
on the hardware to stream to KVS
• After integration, you can stream video from on-board webcam or an RTSP
(Real Time Streaming Protocol)/ IP camera to KVS
• Plugin manages transfer of video to KVS by encapsulating the functionality
provided by the Kinesis Video Streams producer SDK

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Module outcomes
• For your target platform: build/ install Kinesis Video Streams producer SDK
asset to capture and stream live video
• Access to video stream in your Kinesis Video Streams management console
• (Optional) Modify producer side settings and see changes

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Playback video streams in live and on-demand mode

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kinesis Video playback using HLS
• HTTP Live Streaming (HLS) is an industry-
standard HTTP-based media streaming
communications protocol
• KVS supports HLS to view the video,
either for live playback or to view
archived video
• Ensure media fragment type is AVC
(Advanced Video Coding) for H264 format
• Ensure non-zero data retention for stream
Connected
camera
Mobile client
Playback Devices

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How HTTP live streaming works with KVS

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Module activities
• Create an HTML webpage that performs the following actions:
• Loads the VideoJS video player
• Creates Kinesis Video Streams SDK clients
• Retrieves the Kinesis Video Streams Archived Content Endpoint
• Retrieves an HLS Streaming Session URL
• Plays a live video stream in the video player
• Plays an on-demand video clip in the video player
• Learn best practices on fine-tuning end-to-end latency

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Video Playback latency best practices
• Fragment duration: Decrease fragment duration (1 second recommended) to reduce the
amount of video that needs to be combined into a fragment before it can be persisted and
loaded by the player
• Bitrate: Decrease bitrate as it takes less time to read, write, and transmit smaller
fragments
• Fragment count in media playlists: Decrease the max number of fragments (3-5
recommended) per playlist because some players start playback from the oldest fragment
in a media playlist.
• Player buffer size: Decrease the player minimum buffer size (0 sec recommended)
• Player "catch up": Players don't automatically catch playback up to the front of the video
buffer. Custom player can avoid this by dropping frames, or increasing the playback speed
(to 1.1x) to catch up to front.

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Module outcomes
• Learn the basics of HTTP Live Streaming (HLS) in the context of Kinesis Video
Streams
• Perform live/on-demand playback actions using sample video player
• Learn playback latency tuning options

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sample, parse, and decode frames in video stream to invoke Amazon
SageMaker in real-time

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Objectives
• Create an Amazon SageMaker endpoint using Amazon’s object classification
algorithm and sample training data through a AWS AWS CloudFormation
Template
• Create associated Amazon Kinesis Video Streams infrastructure to parse,
sample, and decode frames reliably to invoke Amazon SageMaker endpoint
• Investigate AWS Lambda logs to verify detections performed
• Dive into AWS CloudWatch dashboard for high level overview.

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common
problems
Built-in, high
performance
algorithms
One-click
training
Hyperparameter
optimization
BUILD TRAIN DEPLOY

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
KVS Inference Template for Amazon SageMaker (KIT)
Sample, decode, invoke Amazon SageMaker at scale, in real-time

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Module Activities
• Spin up relevant resources: Amazon ECS, Fargate, DynamoDB
• Host a pre-created object detection model in Amazon SageMaker
• Run AWS CloudFormation template to create a Amazon SageMaker
endpoint using object classification algorithm
PDX
• Run AWS CloudFormation template to create inference stack
PDX

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Module Outcomes
• Create an Amazon SageMaker endpoint hosting a pre-built object detection
model
• Deployed infrastructure including Amazon ECS/ Fargate, AWS DynamoDB,
Amazon Kinesis Data Stream, and more.
• Investigated the pre-created Amazon CloudWatch dashboard to understand
the various metrics
• Dove into AWS Lambda logs to verify results processed

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon CloudWatch Dashboard

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Recap
• Understood what a hardware media pipeline looks like, incl. an intro
to Gstreamer
• Dove into key steps needed with focus on capture, playback, and ML-
driven analytics of video
• Streamed from a camera
• Played video back on a webpage
• Analyzed media using Amazon SageMaker
• Built an end-to-end demo from device to ML-backend
• Don’t forget to teardown the AWS CloudFormation stacks

37. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kinesis Video Team
adikri@amazon.com

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.