1. Module 2 & 3
MULTIMEDIA HARDWARE & SOFTWARE
Classes of multimedia Applications
1. Streaming Stored Audio and Video
2. Streaming Live Audio and Video
3. Real-Time Interactive Audio and Video
4. Others
Streaming Stored Audio and Video
• The multimedia content has been prerecorded and stored on
a server User may pause, rewind, forward, etc…
• The time between the initial request and display start can be
1 to 10 seconds
• Constraint: after display start, the playout must be continuous
Streaming Live Audio and Video
• Similar to traditional broadcast TV/radio, but delivery on the
Internet Non-interactive just view/listen
• Can not pause or rewind Often combined with multicast The
time between the initial request and display start can be up
to 10 seconds
• Constraint: like stored streaming, after display start, the
playout must be continuous
Real-Time Interactive Audio and Video
• Phone conversation/Video conferencing
• Constraint: delay between initial request and display start
must be small
o Video: <150 ms acceptable
o Audio: <150 ms not perceived, <400 ms acceptable
• Constraint: after display start, the playout must be continuous
Others
•
•
Multimedia sharing applications
o Download-and-then-play applications
o E.g. Napster, Gnutella, Freenet
Distance learning applications
o Coordinate video, audio and data
o Typically distributed on CDs
Audio compression
Several techniques:

2. •
•
GSM (13 kbps), G.729(8 kbps), G723.3(6.4 and 5.3kbps)
MPEG 1 layer 3 (also known as MP3)
o Typical compress rates 96kbps, 128kbps, 160kbps
o Very little sound degradation
o If file is broken up, each piece is still playable
o Complex (psychoacoustic masking, redundancy
reduction, and bit reservoir buffering)
o 3-minute song (128kbps) : 2.8MB
Image compression: JPEG
• Divide digitized image in 8x8 pixel blocks
• Pixel blocks are transformed into frequency blocks using
DCT (Discrete Cosine Transform). This is similar to FFT
(Fast Fourier Transform)
• The quantization phase limits the precision of the
frequency coefficient.
• The encoding phase packs this information in a dense
fashion
Video compression
Popular techniques:
• MPEG 1 for CD-ROM quality video (1.5Mbps)
• MPEG 2 for high quality DVD video (3-6 Mbps)
• MPEG 4 for object-oriented video compression
We can classify multimedia software as system software and authoring
tools used to develop multimedia applications. The former sets up the
communications between the system resources and the application
programs. The most common example of this class is Multimedia Control
Interface (MCI) by Microsoft. Block diagram of this software is shown in the
figure below.

3. We shall explain the various components of this structure in more detail.
Telephony
The most important developments in the recent years is the computer
telephony integration. This way the reliable telephone technology has
collaborated with the innovative computer industry and this generated
switches controlled by software and computers connected by the
telephone network.
MPC level3 standards has defined a Telephony Applications Programming
Interface for this integration. Windows supports this interface. TAPI is part of
Windows Open Architecture and used to implement applications with
computer-telephony interaction such as modem access.
Video for Windows
This software has been developed by Microsoft to prepare video data to
be used in multimedia applications. It is used to compress and
decompress and play the video. Microsoft Video I, Intel Indeo, Cinepak or
Microsoft run Length Encoding (RLE) are examples of the codecs used for
compressing the video data. The videos generated by VfW are stores in
AVI (Audio Video Interleaved) format. In this approach the audio data is
stored together with the image for each frame.
Videos can be captured, edited and played using the set of software
included in the VfW. Editing involves cutting, inserting and merging video
data to generate a new sequence of frames. VfW uses VidCap to
capture video, and VidEdit to edit it. It is a free software which comes with
all the source codes.
Quicktime
This is a system software for Apple Macintosh. It enables the integration of
video, audio, animation and graphics in a standard way in multimedia
applications. It supports JPEG, Apple Compact Video, Apple Animation
Compressor, Real Time Video, MPEG and motion JPEG.
A Windows version of Quicktime -Quicktime for Windows- has been
developed and the transfer of files between these two platforms has been
made possible. In the Windows environment files with MOV extensions are
generated using Quicktime.
Indeo and Digital Video Interactive

4. Indeo is Intel Video (Indeo) technology and defines a file format for
converting NTSC and PAL analog signals to digital video which can be
stored and played back in a PC. The compression defined in Indeo is Real
Time Video which uses a vector quantization scheme similar but less
complicated than MPEG.
DVI on the other hand defines a CD format based on Intel 80x86
processors. It is similar to CD_I and was developed for interactivity in
games and generating special video effects.
New Microsoft Multimedia Technologies
DirectX by Microsoft is an example of new multimedia architecture
designed to improve the application performance of Windows 95. It is a
low level API which includes:
•
•
•
DirectDraw - composition engine for 2d and 3D video and
animation
Direct3D - rendering engine for real time 3D graphics
DirectSound - this provides device independent access to sound
cards

5. •
•
DirectPlay - provides Internet connectivity for multiplayer games
DirectInput - provides connectivity to input devices such as joysticks
and virtual reality goggles.
The block diagram of the above software has been shown in figure 2.7 to
give an insight into the interaction among different components.
The GDI defined in figure 2.7 is 2D graphics engine for graphical elements
such as lines, curves, and other shapes.
In addition to these Active Movie has been developed. This is a software
developed instead of MCI but it is also compatible with MCI.
The next set of software used in multimedia applications is the actual tools
used to develop them. It is possible to divide these authoring tools into 4
categories.
•
•
•
•
Programming languages
Presentation software
Simple Interactive software
Complex interactive software
Programming Languages
These are the 3rd and 4th generation programming languages such as
C/C++, Visual Basic and Delphi. If these are chosen to prepare the
multimedia applications then one can achieve maximum control over the
contents and developing interactive applications become easy. They
require considerable programming experience, however, and they are
not easy to learn. The other tools are developed using these languages.
Presentation Tools
Powerpoint is an example of such presentation tools. We can also include
graphics based word processing software where video and audio can be
inserted. It is ideal for applications which use text and simple graphics.
Limited applications with audio and video can be developed.
These tools are fairly easy to learn and are suitable for developing cheap
multimedia applications in a short time.
Simple Interactive software

6. These are suitable for developing multimedia applications where there is
interactivity. Buttons or Hypertext can be used to navigate within the
application. Most of this software is page based which means the
contents and control elements are placed within pages and they are
used to go to different parts of the application. Again limited use of digital
video and audio is possible.
These programs are more difficult to learn compared to the previous class
but they do not require much programming experience either. Adobe
Acrobat, Astound are examples of this category.
Complex Interactive Software
These are the most powerful multimedia development tools. It is possible
to incorporate programming languages into the applications and
improve the control over the navigation and interaction.
There are three different types.
Page Based
In these tools the contents and controls are placed in pages. These pages
can be accessed through linked items or controls. Since the educational
software is based on page based tools these are very useful in developing
educational applications.
ToolBook is an example of page based multimedia development
software.
Time Based
These authoring software include a time line where the pages or data
components are inserted and display of different components can be
controlled either by assigning a duration to each page or manually. This
way very successful animations and effects can be produced.
The most common examples of these tools are Director and Scala.
Icon Based
These products use icons which makes it possible to access to the
contents or controls. They are useful for producing complex structural
applications. Authorware is an example of such a software.

7. HARDWARE REQUIREMENTS OF THE MULTIMEDIA COMPUTERS
Multimedia computers should support the capture, editing and viewing of
video and audio data and provide facilities for the transfer of these data
types between computers either through modems or through local and
wide area networks. Various parts that should be present in the
multimedia computers (PCs in particular) in order to support these
functionalities and their specifications have been defined by the
Multimedia PC (MPC) group and included in the MPC Level 3 standards.
This standards body has first defined MPC standards and then MPC2 and
finally parallel to the developments in the computer technology the MPC3
standards.
The MPC 3 standards defined the characteristics of the various parts of the
computers shown in Figure below.
USB : Universal Serial Bus
MIDI : Musical Instruments Digital Interface
NIC : Network Interface Card
We shall be dealing with these parts in more detail in the proceeding
sections. Below we have summarized what we think a multimedia PC
should have as minimum requirements bearing in mind the developments
in the computer industry. You have to remember that even these
definitions may change by the time you read them because of the rapid
technological developments.
•
•
•
•
•
•
Processor: Pentium II 350 MHz
Memory: 64 Mbytes
Disk : 4 Gbytes
CD ROM : 24 speed
Sound Card: 16 or 32 bits
Graphics card: one with 4 Mbytes of memory

8. It is possible to expand this list and enhance the specifications, remember
that these are the subjective minimum requirements.
Processor:
Multimedia applications require powerful processors because of their
demanding nature. Applications such as compression or decompression
of MPEG (Motion Picture Expert Group) video for example requires that 25
to 30 frames of video should be processed and displayed per second
depending on the use of PAL or NTSC standards respectively. The
processing involved in the compression and decompression process is
fairly complicated and time consuming. In order to meet these
requirements either the extended instructions have been used in
processors or special multimedia processors have been devised.
Intel adopted the former approach and included the MMX technology in
its processors. In these series there are extra 57 new instructions, which
have been developed with the multimedia applications in mind. One of
the instructions for example can do a number of multiplications in parallel
and sum their results. Digital signal processing requires such operations
and MMX can perform operations on more than one data unit at the
same time. This ability is referred to as Single Instruction Multiple Data
(SIMD) operation. To be able to achieve this Intel uses a 64 bit register
which can be used as units of 8 bits which means that it can hold 8 bits, 16
bits or 32 bits of data and perform operations on the parts independently.
Using special microprocessors for multimedia applications is another
approach adopted by some manufacturers. Mpact/6000 which is a
multimedia accelerator chip developed by Chromatic's is an example of
such an approach.
Memory:
Considering the size of the multimedia applications memory capacity is
important to reduce the number of disk accesses. Normally programs are
transferred to the main memory from the disk and executed there. Those
which do not fit into the memory - and most of the multimedia
applications fit into this category - are transferred as pages. The bigger
the memory the more pages shall be transferred hence the number of
disk accesses will be reduced and the programs will be executed faster.
CD_ROM

9. CD_ROM is the abbreviation of the words Compact Disk Read Only
Memory. The fastest and cheapest way to distribute multimedia
applications is using CD-ROMs. We shall be summarizing the different type
of CD ROM formats that support multimedia applications below.
CD-DA : Supports digital audio
CD-I : This a format developed for applications requiring interactivity.
Supports graphics, audio and video. It is suitable for interactive games.
CD-XA: XA stands for extended architecture. This format improves the CD
ROM's audio and video capability.
MSCDEX: A capability to allow the use of CD-ROMs in the DOS
environment.
PhotoCD: It is used to save photos in the compressed form. The
compression allows for 100 photos to be recorded in one CD.
Video CD: It is used to store video compressed in MPEG_I format. Videos
up to 74 minutes in duration can be stored in one CD using this format.
DVD: Digital Video Disk technology improves the storage capacity to 4.7
to 17 Gbytes. It is mainly used to store video compressed in MPEG II
standards in a CD ROM. It is being supported especially by the film
industry because it supports interactivity and provides protection against
illegal copying of the disks. Although it has been defined as part of the
MPC Level 3 specifications it will be used in other areas as well due to the
capacity it provides.
Sound Card:
The audio system specified in MPC Level 3 includes
•
•
•
•
•
•
•
8 or 16 bits per sample sampling capability
8, 11, 16, 22, or 44 KHz sampling rate
A linear Pulse Code Modulation (PCM) Codec to convert the
sampled analog values to digital data.
A playback and recording of 16-bit stereo that uses no more than
13 % of CPU time for 44.1 K samples/sec.
CD ROM drive with CD-DA outputs and volume control.
Speakers
One MIDI port.

10. The block diagram of a sound card compliant with the MPC Level 3
specifications is shown in the figure.
Video Graphics Interface Card
A graphics display card is used to display an digital data on an analog
monitor. Each pixel is represented with 8 to 32 bits and these are
converted to RGB signals through a palette. These values are then
converted to analog signals for display on the monitor.
MPC Level 3 standards require that a graphics adapter should be
capable of displaying a 352x240 video at 30 frames per second without
dropping any frames. An additional card is usually required to playback
videos compressed using MPEG-I. MPEG corresponds to Motion Picture
Experts Group. Using this card MPEG videos can be displayed on the
monitor.
We need an additional board in order to be able to capture analog
video and store it in our computer. These cards accept video from VCRs
or video cameras and their performance depends on the speed of the
codecs used in them. It has to be remembered that one minute of a
video recording will occupy 10 to 20 Mbytes of disk space depending on
the compression used.
Universal Serial Bus
With the introduction of multimedia applications the number of devices
connected to the computers have increased considerably and this has
created problems in management and maintenance of the systems. To
avoid the increased number of connectors being used the USB has been
developed. Using the USB
•
The current peripheral devices

11. •
•
•
Telephony equipment
Joysticks and other tools for playing games
Video equipment such as cameras
can be connected to the computers from a single port. The total speed of
this connection can be 1.5 Mbits/s or 12 Mbits/s and the configuration is
shown in figure.
The peripheral devices could be connected through hubs or one of the
equipment such as the keyboard could serve as a hub. This way a total of
127 devices can be connected to the bus. The maximum cable length in
any one connection cannot be longer than 5 m.
As stated before the USB has a limited bandwidth. This prevents many
devices to be connected simultaneously and even if they can be
connected they cannot be serviced simultaneously. To solve this problem
High Performance Serial Bus defined by IEEE 1394 standards has been
developed. This bus can support bit rates of 100, 200, and 400 Mbits/sec.
The Apple version of HPSB is called firewire.
Network Interface card - NIC
NICs are used to connect the computers using local area networks. These
cards can support Ethernet, Token Ring or Token Bus structures. The bit
rates depend on the type of network being used and can be 4, 10, 16 or
100 Mbits/sec. There also exist network cards which support cable TV
connections.