1. Video Over DDS
DDS Information Day
Brussels, Belgium-June-26,2007
Copyright © MilSOFT,Turkey
UNCLASSIFIED 1

2. Outline
• Introduction
• Image Exploitation System for UAVs
• Why Use DDS for Video Streaming?
• MilSOFT DDS
• DDS Topics
• QoS choices
• Future Work
• Conclusion
Copyright © MilSOFT,Turkey
UNCLASSIFIED 2

3. Introduction
• DDS middleware
• Location transparency with pub/sub mechanism
• Redundancy and fault tolerance mechanisms
• Multiple readers/writers
• Transient and persistent services
• Asynchronous, time-independent data distribution
• Rich set of QoS, suitable for real-time systems
• Wide range of Application Areas
• C4I
• Industrial automation
• Distributed control and simulation
• Telecom equipment control
• Sensor networks
• Use Case: Image Exploitation System for UAVs
Copyright © MilSOFT,Turkey
UNCLASSIFIED 3

4. DDS on UAV Image Exploitation System
DB Server
Raw Data
Recording
High Capacity
Storage Unit
Pre-processed &
Georecorded Data
Sensor Data
(EO/IR, SAR, GMTI)
Raw Data
Telemetry DDS
DDS
based
based
Data Receiver
Data
Data Pre-processor
Streaming
Streaming
Artifacts
Screening Exploitation 1 Exploitation 2 All Data in DB
Copyright © MilSOFT,Turkey
UNCLASSIFIED 4

5. DDS on UAV Image Exploitation System
• Data
• Events
• User information
• Resource Management
• Radar data streaming
• SAR streaming
• GMTI streaming
• Telemetry streaming
• Video streaming
• Live data streaming
• Recorded data streaming
• Data Publisher: DataReceiver
• Data Subscribers: Consoles and Pre-Processor
• Content of data MPEG-2 TS
Copyright © MilSOFT,Turkey
UNCLASSIFIED 5

6. MPEG-2 TS (Transport Stream)
• Elementary Stream
• Video, audio, telemetry data, etc...
• From each elementary stream Packetized Elementary Stream
(PES) is formed
• And each PES is broken into fixed-size Transport Packets
• TS packets are 188 bytes of which 4 bytes are for the header
and 184 bytes are for the payload
• Transport Stream can multiplex multiple elementary streams
• The TS packet header contains a Packet Identifier (PID) to identify
different elementary streams
• 8 Mbit data rate for UAV Image Exploitation System
Copyright © MilSOFT,Turkey
UNCLASSIFIED 6

7. Why use DDS for video streaming?
• Publish-Subscribe simplifies the development process
• The clients do not need to know where the server is, and the server
does not need to know the clients
• DDS specification implies mutlicast communication and the
interoperability wire protocol RTPS supports and recommends
multicasting
• Dramatically decreased bandwith usage with DDS implementations
which support multicast
• Quality Of Service attributes like reliability, durability, transport
priority directly can be used for video streaming channels
• Generally DDS implementations have very low latencies in
data transmission compared to other middlewares, which
makes it a good choice
Copyright © MilSOFT,Turkey
UNCLASSIFIED 7

8. Why MilSOFT DDS
• MilSOFT DDS is compliant with current DDS specification,
version 1.2
• Provides Minimum Profile QoS attributes, which are
neccessary for video streaming
• Completely decentralized design
• No dynamic memory allocation after initialization
• Zero-copy access to data
• Implements multicast communications
Copyright © MilSOFT,Turkey
UNCLASSIFIED 8

9. MilSOFT DDS Middleware
More information on http://dds.milsoft.com.tr
Copyright © MilSOFT,Turkey
UNCLASSIFIED 9

10. Video Over DDS Architecture
Copyright © MilSOFT,Turkey
UNCLASSIFIED 10

11. DDS Types and Topics Used For Video Streaming
• So how do we encapsulate MPEG2-TS packets into DDS
types?
• There are two DDS types and two corresponding Topics.
• VideoHeader  VideoHeaderTopic
• VideoPacket  VideoPacketType
• VideoHeader
• Contains the header of the video, which is needed to initialize the
decoder. (FPS, height, width, etc... )
• VideoPacket
• Contains MPEG2-TS packets
Copyright © MilSOFT,Turkey
UNCLASSIFIED 11

12. VideoHeader Topic
struct VideoHeader
{
long headerType; // key
sequence<char, HEADER_MAX_DATA> headerData;
};
• Published once for each Transport Stream
• Contains the necessary information
• initialize the video decoder
• information about the other multiplexed elementary streams
• Key to distinguish the video header from other header data
• Late joiners also get this topic
Copyright © MilSOFT,Turkey
UNCLASSIFIED 12

13. QOS for VideoHeader Topic
• VideoHeaderTopic is necessary to initialize decoding of the
video
• Header information is critical for clients
• Use Reliability QoS (RELIABLE)
• Both publishers and subscribers are Reliable
• Late joining viewers need this information
• Use Durability QoS (TRANSIENT_LOCAL)
• There are different instances of the header according to it’s
type. We need only the last update of each instance!
• History QoS is default, kind = KEEP_LAST, depth = 1
Copyright © MilSOFT,Turkey
UNCLASSIFIED 13

14. VideoPacket Topic
struct VideoPacketType
{
long frameNumber;
sequence<char, PACKET_MAX_DATA_SIZE> packetData;
};
• MPEG2-TS packets are too small in size (188 bytes).
• VideoPacket type sends multiple TS packets of a video frame
at each VideoPacket sample
• Decreases the overhead and increases the bandwidth utilization
• A frame can span multiple VideoPacket samples (fragmented)
• Each sample contains TS packets of only one frame
Copyright © MilSOFT,Turkey
UNCLASSIFIED 14

15. QOS for VideoPacket Topic
• For VideoPacketTopic we may need both Reliable and Best
Effort delivery, according to client needs
• Use Offered >= Requested semantics
• Server publishes with a Reliable writer ( Offer Reliable communication )
• Clients subscribe with either Reliable or Best Effort readers
• Data is not needed by late joiners
• Durability is Volatile
• VideoPacket Type has no key, but we can use History QoS
for buffering
• History QoS, kind = KEEP_LAST, depth value can be used as a buffer
(set it to more than 1)
Copyright © MilSOFT,Turkey
UNCLASSIFIED 15

16. Other Concerns And Solutions
• We don’t want out of sequence frames
• DDS already satisfies this, no out of order packets!
• We don’t want frames which are too late
• Use Lifespan QoS
• Decoding takes time
• Either read data from a DataReaderListener and immediately put into
the queue of another thread
• Use WaitSets
Copyright © MilSOFT,Turkey
UNCLASSIFIED 16

17. Future Work
• Implement advanced services like rewind, replay, fast-forward
etc...
• Network reservations
• Use RSVP (IntServ) to reserve bandwith
• Analyze different methods to give DDS video streams a
priority
• Use Transport_Priority QoS (not completely standard)
• More standardization is needed in this area
• Standardize Flow Control mechanisms in DDS for better
utilization of network bandwith
Copyright © MilSOFT,Turkey
UNCLASSIFIED 17

18. Conclusion
• Video over DDS is possible!
• DDS provides a good set of QoS attributes for video
streaming
• Performance requirements
• Image Exploitation system for UAVs uses 8 Mbit MPEG-2 TS
• 20 Mbit video streaming is possible with DDS
Copyright © MilSOFT,Turkey
UNCLASSIFIED 18

19. Thank You!
Copyright © MilSOFT,Turkey
UNCLASSIFIED 19