Resource-Oriented Architecture offers advantages over other web-service architectures. It is based on a simple, scalable and highly standardised application-level protocol. Multimedia content is commonly managed using the MPEG-7. The MPEG-7 is a standard for representing audiovisual information that satisfies specific requirements based on syntax, semantic and decoding. Content descriptions under MPEG-7 can be organised and characterized without ambiguity. The MPEG-7 eXperimental Model (XM) includes the best performing tools for MPEG-7 normative and non-normative elements. In this paper, multimedia content is managed using the MPEG-7 eXperimental Model functionalities and provided using web-services technology. RESTful principles are the guidelines for achieving multimedia content storage and retrieval. Quantitative evaluation of the proposed web services has shown that this approach has better performance, in term of retrieval speed and storage space

1. Resource-Oriented Architecture for
Managing Multimedia Content
Margareth Florián & María Trujillo
Universidad del Valle
Latin-American Conference on Networked and Electronic Media
LACNEM2009

2. Outline
Motivation
The MPEG-7 Standard
Web Services Architectures
Architecture
Resources for Managing Multimedia Content
Final Remarks
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 2 / 26

3. Motivation
Storage and retrieval in very large repositories
Cell phones Cameras
Scanners Other devices
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 3 / 26

4. Problem Statement
Content Based Retrieval
Manual annotation Automatic annotation
Subjective Objective
Context Knowledge Consistence
Key words
Description MPEG-7
Title
It does not provide a model for The use of the
mapping MPEG-7 descriptors MPEG-7 XM requires
into a database system technical knowledge
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 4 / 26

5. The MPEG-7 Standard (I)
MPEG-7
The MPEG-7 is a standard for describing multimedia content
Searching Filtering Managing Processing
Descriptors: describe individual features of multimedia content
MPEG-7 media
descriptors
= XML documents + media description Schemes
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 5 / 26

6. The MPEG-7 Standard (II)
MPEG-7 descriptors
Color Descriptors Motion Descriptors
Dominant Color Camera Motion
Scalable Color Motion Trajectory
Color Layout Parametric Motion
Color Structure Motion Activity
Texture Descriptors Localization
Homogenous Texture Region Locator
Texture Browsing Spatio Temporal Locator
Edge Histogram
Others
Shape Descriptors Face Recognition
Region Shape
Contour Shape
Shape 3D
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 6 / 26

7. MPEG-7 Technical Knowledge
MPEG-7 extracting and matching
Color descriptor: Dominant Color Color descriptor: Color Layout
Extraction Extraction
Di = Σ h(n) ||x(n)-ci||2, x(n) Є Ci Image
partitioning
Representative
color selection
n
Binary Zigzag scanning DCT
marks and weighting transformation
Matching Matching
√Σw (DY – DY ’) +√Σw (DCb – DCb ’)
N1 N2 N1N2
2 2
D2(F 1, F2) = Σp12i + Σp22j – ΣΣ (2a1i,2j)p1ip2j D= i
yi i i
i
bi i i
i=1 j=1 i=1j=1
+√Σw (DCr – DCr ’)
i
ri i i
2
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 7 / 26

8. MPEG-7 Technical Knowledge
Using the MPEG-7 eXperimental Model
The use of the MPEG-7 XM requires technical knowledge
/usr/local/XM/bin/XMMain_linux6.1.exe –a ColorStructureClient -b bitstreams/1.xml -p parfiles/ColorStructure.par -l
storage/1.lst -r error.log
User
Retrieve
? Retrieve
Client clientDominantColor result
Merge files
Images
Application
Name
? clientColorLayout
Options
….
MPEG-7 Config file
XM ? Parameters ? Bitstream file
program
Server serverDominantColor Storage file
Storage
Images
Application
Name
? serverColorLayout
….
/usr/local/XM/bin/XMMain_linux6.1.exe –a ColorStructureServer -b bitstreams/1.xml -p parfiles/ColorStructure.par -l
storage/1.lst -r error.log
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 8 / 26

9. Requirements
Client Requirements
User options
Output: search result
Storage Retrieval information
Multimedia Query by Example View-Based
resource search
User interface (WEB Portal)
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 9 / 26

10. Web Services
Web services for using MPEG-7 standard by non technical users
requires work
delivers result
does
[Hüttenrauch 2006]
Consumer/Client work
Service Provider
Service-Oriented Architecture Resource-Oriented Architecture
(SOA) (ROA)
[Hüttenrauch 2006] Hüttenrauch, S.: Definitions, Historical Development, Advantages and Drawbacks of Service - Oriented
Computing. Technische Berichte Nr. 16 des Hasso-Plattner-Instituts für Software system technik an der Universität Potsdam (2006).
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 10 / 26

11. Web Services Architectures (I)
Service-Oriented Architecture (SOA)
Service
Broker Commonly based on Remote Procedures Call
(RPC)
UDDI
Provides internal algorithms programming-language-
like interface
WSDL WSDL
Provides one Uniform Resource Interface (URI) for
SOAP every process capable of handling
f(x)
Service Service The method and scoping information goes inside
Requester Provider the envelope
[Grund 2006]
[Grund 2006] M. Grund Service Communication and Discovery - Fundamentals of Service-Oriented Engineering. 2006
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 11 / 26

12. Web Services Architectures (II)
Resource-Oriented Architecture (ROA)
Uses the standards HTTP methods: GET, POST,
Optional
PUT and DELETE
Provides internal data through a document-
WADL
processing interface
URI HTTP
The method information goes into the HTTP method.
Service The scoping information goes into the URI
Client
Provider
There is a URI for every piece of data the client
might want to operate
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 12 / 26

13. Web Service Architecture (III)
Selected Web Service Architecture
Service Oriented Architecture Resource Oriented Architecture
REST-RPC REST
Service Broker
Optional
UDDI
[Grund 2006] WADL
WSDL WSDL
f(x) SOAP URI HTTP
Client Service Service
Client
Provider Provider
Action REST-RPC REST
show image id=2 GET /images/show/2 GET /images/2
create new image POST /images/create POST /images/
destroy image id=2 GET /images/destroy/2 DELETE /images/2
update image id=2 POST /images/update/2 PUT /images/2
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 13 / 26

14. Web Service Architecture
for Managing Multimedia Content
Selected Web Service Architecture
Selection Criteria service architectures SOA ROA
Information in headers Low High
Envelope formats
Final user knowledge Medium Low
WSDL, is WADL, is complementary
Description XML-based vocabulary
necessary information
Languages
Final user knowledge Medium Low
Control provide services YES NO
Discover web services
Final user knowledge Medium Low
WS-Security standard YES YES
Security
Extend standard YES NO
GET, PUT, DELETE and
by defining extra
Reliable Messaging WS-messaging standard HEAD are idempotent
stickers
operations
WS-Atomic transaction YES YES
Transactions
WS-bussinessActivity YES YES
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 14 / 26

15. MPEG-7 SOS
(Service Oriented System)
Architecture
Mapping to
database
Annotation
HTTP
Storage Web Services
User interface (WEB Portal)
Multimedia
Web Services/Resources
resource Automatic –
Database Access Interface
MPEG-7 Manual
Annotation
HTTP
MPEG-7 XM
Web Server
Web Server
Textual
descriptions
DBMS
User Retrieval information
options
Query by Example
MPEG-7
Information descriptors
HTTP
retrieval
View-Based
Web Services
search
Database
querying
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 15 / 26

16. A Database Model for MPEG-7 visual
descriptors (I)
sos_DedgeHistogram
Used Database schema Image_id
1 eh_group1
eh_group2
sos_DcolorStructure 1 1 eh_group3
Image_id eh_group4
DcolorStructure_values
sos_image eh_group5
image_id eh_group6
1 image_name 1
image_desc
Image_author
image_date
image_url
image_size sos_DcolorLayout
1 Image_id
1 cl_YDCCoeff
1
sos_DdominantColor cl_CbDCCoeff
cl_CrDCCoeff
Image_id cl_YACCoeff5
dc_spatialcoherency cl_CbACCoeff2
dc_coloursnumber cl_CrACCoeff2
dc_percentages
dc_indexes
Florián, M., Trujillo, M.: Relational Database Schema for MPEG-7 Visual Descriptors. Encuentro de Investigación en Ingeniería de Sistemas
y Computación. Bogota, Colombia (2008).
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 16 / 26

17. Resources and Database Connection
Content-based Retrieval
MPEG-7 XM
DominantColorDistance.so extracted C function
Store
procedure SQL
SELECT DominantGetDistance(table_columns, example_image_values)
as distance, id
FROM sosdominantcolors
ORDER BY distance
sos_DdominantColor
id
dc_spatialcoherency
dc_coloursnumber
dc_percentages
dc_indexes
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 17 / 26

18. Resources for Managing Multimedia Content
Resources for MPEG-7 visual descriptors
ROA has three kinds of resources:
1. Predefined one-off
2. A resource for every object is providing through the service
3. Resources representing the result of algorithms applied to the data set
Resource URI UI method Representation
Application http://mpeg7sos.com/ GET html
Images http://mpeg7sos.com/sosimages/{imageid} GET, POST jpg, gif, png
Descriptors http://mpeg7sos.com/{descriptorname} GET,POST text/xml
Images http://mpeg7sos.com/{descriptorname}
GET text/xml
directory /{exampleimage}
Support web-services: new image and search images forms
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 18 / 26

19. Resources for Managing Multimedia Content
Resources user navigation
Storing User visible steps
Hidden for user
Input Form
Browse
link
Manual
information form
Save
sosimages/{imageid}
Web-service
POST HTTP method
redirect
sosimages web-service
mapping descriptors save mapping
web-services DB
MPEG-7 XM Proposed DB schema
Server Extraction Tool
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 19 / 26

20. Resources for Managing Multimedia Content
Resources user navigation
User visible steps
Retrieving Query by Example Hidden for user
Color Layout link Example Image Search form Image 1: 90% Link
Dominant Color Browse Image 2: 88%
Color Structure Image 3: 80%
Edge Histogram Search …
Sosimages/{imageid}
GET HTTP method Web-service
Format response
{sosdescriptorname}/{imagename}
web-service
Example Image Execute sql command MPEG-7 XM
execute
extracted C
DB store-procedure
function
MPEG-7 XM
Proposed DB schema
Server Extraction Tool
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 20 / 26

21. Resources for Managing Multimedia Content
Resources user navigation
User visible steps
Retrieving View-based Hidden for user
View based search
Name Search Image 1 link
Description form Image 2
Image 3
Key words
…
Search
Sosimages/{imageid}
Web-service
GET HTTP method Format response
sosimages?{params}
web-service
Execute sql command
DB
Proposed DB schema
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 21 / 26

22. Web-services Usage Example
Simple Connection view-based search example
requires work
http://server_name/images?name=Garden&author=&desc=&keywords
does
Consumer/Client work
Service Provider
Resource URL: http://server_name/images SELECT *
Params GET: FROM ‘images’
name=‘Garden’ WHERE image_name LIKE ‘%Garden%’ and
author=‘’ image_author LIKE ‘%%’ and
desc=‘’ image_desc LIKE ‘%%’
keywords=‘’
delivers result
html / xml / json / …
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 22 / 26

23. Web-services Usage Example
Simple Connection content-based search example
requires work
http://server_name/sos_DdominantColor?image=codified_image
does
Consumer/Client work
Service Provider
Resource URL: http://server_name/sos_DdominantColor SELECT DominantGetDistance(table_columns,
Params GET: example_image_values)
image = http send image as distance, id
FROM sosdominantcolors
ORDER BY distance
delivers result
html / xml / json / …
49557986 50080672 51807768
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 23 / 26

24. Web-services Usage Example
Simple Connection save image example
requires work
does
Consumer/Client work
Service Provider
Resource URL: http://server_name/images Execute XM for extracting descriptors
Params POST: Save information to database
delivers result
html / xml / json / …
Result:OK status 200
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 24 / 26

25. Final Remarks
MPEG-7 Technical Knowledge
Six criteria for selecting web service oriented architecture were proposed
(1) Envelope formats, (4) Security,
(2) Description Languages, (5) Reliable Messaging, and
(3) Discover web services (6) Transactions
Resource oriented architecture (ROA) was selected for accessing MPEG-7 XM
functionalities
In general terms, the HTTP uniform interface, the HTTP header information,
and the natural web navigation provide advantages over SOA for creating
public web services
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 25 / 26

26. Resource-
Resource-Oriented Architecture for
Managing Multimedia Content
Multimedia content
The MPEG-7 Standard
Web Services Architectures
Architecture
Resources for Managing Multimedia Content
Final Remarks
LACNEM09 Resource-Oriented Architecture for Managing Multimedia Content Slide 26 / 26