JPSearch is a set of specifications that aims to provide interoperability for image search across different systems and repositories. It defines interfaces and protocols for data exchange in a modular and flexible architecture. The goal is to ensure portability of metadata and allow consumers to search across multiple sources without being locked into a single system. JPSearch includes specifications for ontology registration, query formats, embedding metadata in image files, and data interchange between repositories. It is developed following ISO procedures and is currently maintaining and extending existing specifications.
On National Teacher Day, meet the 2024-25 Kenan Fellows
An overview of JPSearch
1. 1
An overview of
JPSearch
Touradj Ebrahimi
Touradj.Ebrahimi@epfl.ch
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
2. 2
Motivation
• Phenomenal growth in applications relying on
personal or professional, and shared digital
image collections
– Getty Images, Corbis, …
– Flickr, Facebook, Picasa, … otos
ill ion ph
ove r4b
more than 2.5 billion photos
uploaded every month
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
3. 3
Challenge
• Annotation, Management and Search of
images in personal and professional
image databases:
– Scientific and technological challenges
– Business challenges
– Legal and ethical challenges
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
4. 4
Goal
• Whatever technical solutions adopted:
– Portability of data and metadata should be
guaranteed
– Consumers should not become de facto locked into
one system
• Allow different image management systems to
interoperate.
• Give consumers and businesses confidence
about longevity of their annotations
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
5. 5
JPSearch is …
• A set of specifications allowing interoperability between devices and
systems
– Defines the interfaces and protocols for data exchange
– Restricts as little as possible how each component performs its task
• An abstract framework and a modular and flexible search
architecture
– Allows an alignment of system design to a standard framework
– Address limitations of existing search systems which tightly couple many
elements of the search process
• Offers interoperability is in flexible ways
– Between self-contained vertical image search systems that interact to provide
federated search.
– Between the different layers of image search so that these components could be
supplied by different best-of-breed vendors
– At the metadata level such that different systems may add, update or query
metadata for images and image collections.
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
6. 6
JPSearch is …
• A means to use and reuse of metadata
– Heavy investment when annotating a collection of images
– With JPSearch, the portability of the metadata is guaranteed
– Allows to subsequently migrate to other applications or systems
– Enables owners of an image collection to merge community/
social metadata back into their own management system
• A means to use and reuse of ontologies
– Provides a common language for contexts
– Allows a user to carry his/her context to different search engines
– Allows the context to be owned by the user and not by the
system, hence protecting privacy
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
7. 7
JPSearch is …
• A common query language
– Removes ambiguity in the formulation of a query
– Makes searching over shared repositories consistent
– Defines query management process such as relevance
feedback
• A solution for the carriage of image collections and
associated metadata
– Enables image search and retrieval functionality across multiple
repositories
– Leverages the generally high cost of creating metadata
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
8. 8
JPSearch approach to standardization
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
9. 9
JPSearch architecture is divided in 4 layers
• User layer:
– Aims at the personalization of the search service
• Query layer:
– Dealing with the formulation of the query and presentation of the
results
• Management layer:
– Handling the distribution of a search task over multiple image
repositories
• Content layer:
– Includes image repositories and associated metadata, as well
as the corresponding schema and ontology
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
10. 10
JPSearch Architecture
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
11. 11
JPSearch functional modules
• Query process (query layer):
– Aims at the efficient execution of search tasks
– Forms a machine understandable query from the user’s search
task
– Validates the search results and ranks them according to the
user’s criteria
– Generates a new query based on relevance feedback
mechanisms
• Repository management process (management layer):
– Allows users to simultaneously search multiple distributed image
repositories with the same query
– Aggregates the results returned from the image repositories
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
12. 12
JPSearch functional modules
• Image repository process (content layer):
– Supplies basic search functionalities
– Includes receiving a set of queries, executing the matching of
these queries with the stored metadata, and forming a result
• Metadata creation and update process (content layer):
– Enables to build metadata using a proper schema and ontology
definition
– Provides with the functionality to update the metadata, e.g. by
adding, replacing, removing all or part of the metadata
– Verifies that the metadata is suitably brought up to date
whenever the image content is modified
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
13. 13
JPSearch
specifications
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
14. 14
JPSearch specifications
• Part-1: Framework and system components:
– Technical report (type-3) providing a global view of
JPSearch.
– Review of the traditional approaches to image search
– Use cases to identify user needs and requirements.
– JPSearch architecture composed of 4 layers.
– Organization of the JPSearch specification.
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
15. 15
JPSearch specifications
• Part-2: Registration, Identification, and
Management of Schema and Ontology:
– Standardizes a platform-independent format for the
import, export and exchange of ontologies
– Defines a registry of ontologies which can be
imported into a JPSearch compliant system
– Standardizes basic functions to query and manipulate
one or more ontologies in a repository
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
16. 16
JPSearch specifications
• Part-3: JPSearch query format:
– Standardized functionalities between users and
image repositories
– Allows users to express their search criteria
– Allows users to describe the aggregated return result
sets for user presentation or machine consumption
– Defines query management processes such as
relevance feedback
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
17. 17
JPSearch specifications
• Part-4: File format for Metadata embedded in
image data (JPEG and JPEG 2000):
– Specifies image data exchange format with
associated metadata to accelerate the re-use of
metadata
– Supports two functionalities: the mobility of metadata
and the persistent association of metadata with
image
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
18. 18
JPSearch specifications
• Part-5: Data interchange format between image
repositories:
– Standardizes a format for the exchange of image
collections and respective metadata between
repositories
– Enables the synchronization of repositories in order
to facilitate interoperable exchange across different
devices and platforms
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
19. 19
JPSearch Workplan
Part Title WD PDTR DTR TR
1 System framework and components 05/07 06/07 07/04 07/07
Part Title WD CD FCD FDIS IS
Registration, Identification, and Management of
2 08/07 09/04 09/10 10/10 11/02
Schema and Ontology
3 JPSearch Query Format -- 08/07 09/04 09/10 10/03
4 File format for Metadata embedded in image data 08/03 08/07 09/07 10/03 10/07
(JPEG and JPEG 2000)
5 Data interchange format between image 08/03 09/07 10/03 10/10 11/02
repositories
(following ISO procedures - WD: Working Draft; PDTR: Proposed Draft Technical Report; DTR: Draft
Technical Report; TR: Technical Report; CD: Committee Draft; FCD: Final Committee Draft; FDIS:
Final Draft International Standard; IS: International Standard)
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
20. 20
Current activities
• Maintenance, update and extension of specifications
• Development of a JPSearch reference software (Part 6)
• JPSearch compliant applications demonstrators
• JPEG innovations workshop in Tokyo, Japan, February 23rd, 2011
• Next three meetings
54th
2011
Feb 21-25 Japan (Tokyo)
55th
2011
Jul 11-15 Germany (Berlin)
56th
2011
Nov 21-25 Switzerland (Geneva)
More details available at: http://www.jpeg.org
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010
21. 21
Thank you for your attention !!
Any questions ?
Multimedia Signal Processing Group Networked Media Systems
6th FP7 Concertation meeting
Swiss Federal Institute of Technology, Lausanne Media Search (MS) Cluster
Brussels, November 30, 2010