2. Overview
What is this about?
Future media - evolutions and trends
Some key trends
Resulting research challenges / plans
Achievements so far
a lot of projects: please see IBBT website...
spin-off companies & patents
other
2
3. Future media – evolutions and trends
Approaching the ‘Zettabyte Era’
by 2012, annual global IP traffic will exceed half a
zettabyte, of which 90% will be video traffic
mobile data traffic will double each year
Super Hi-Vision or UDTV (Ultra High Definition Video)
7680 4320 pixels
Japan plans to deploy
UDTV broadcasting in 2015
requires ca. 600 Mbit/s
with current technologies
3
4. Future media – evolutions and trends
Emerging high-definition user-generated content
multiple versions of online content (e.g., YouTube)
soon to be consumed on high-resolution displays
need for on-the-fly adaptation
Managing huge content collections
automatic detection of concepts
interactive semantic search
multi-modal analysis and
retrieval algorithms
multimedia exploration systems
4
5. Future media – evolutions and trends
Recognition of many object classes,
scalable in the number of classes
(both for training and at run-time)
Limit the need for supervision
5
6. Future media – evolutions and trends
Exploit context information
(scene, other objects, geometry, …)
Couple textual descriptions to image content (nouns-
objects, adjectives-attributes, verbs-actions)
and exploit text to provide contextual information
6
7. Future media – evolutions and trends
Distributed smart camera systems
multi-camera analysis, embedded and distributed
processing
applications connect to general purpose video analysis
services
applications concentrate on high-level aspects; camera
system handles low-level and access control
privacy issues
8. Future media – evolutions and trends
In the brain regions always connect bi-directionally,
in computer vision recognition all too often is
the endpoint of a one-way bottom-up pipeline.
Recognition has a pivotal role to play in terms of motion
analysis, 3D reconstruction, inpainting, ...
We need recognition to talk to all kinds of
complementary visual processes.
We even contend that a lot of progress
in these other areas could actually come from
recognition, rather than from further refinements along
the current lines of thinking
8
9. Future media – evolutions and trends
Applications based on recognition
automatic resizing
3D city reconstruction
traffic safety
content-based image retrieval
9
10. Future media – evolutions and trends
Applications based on recognition
video summarization
virtual editors
surveillance
…
10
11. Future media – evolutions and trends
HCI - interactive 3D environments
natural and intuitive interaction
techniques for emerging environments
‘serious games’
(education, therapy, ...)
Video surveillance
cameras are being installed virtually everywhere
video analysis as security sensor
intuitive control room applications
combining all event information
11
12. Future media – evolutions and trends
Mobile and context-sensitive interactive systems
12
13. Future media – evolutions and trends
Multi-modal interaction in 3D and virtual environments
13
14. Future media – evolutions and trends
Interactive workspaces
14
15. Future media – evolutions and trends
User-centered software engineering
15
16. Future media – evolutions and trends
Computer graphics and computer animation
modeling, rendering and animation
video-based CG and animation
stylized drawn animation
volume rendering and information visualisation
Physically based illumination simulation for virtual
Real-time photo-realistic hair
prototyping
and skin for games and movie
special effects
Image based modeling and rendering
for real world objects/subjects/scenes
16
17. In conclusion: some KEY TRENDS
Zillions of data
need for efficient indexing mechanisms
Different platforms
content reformatting
User-generated content, interactivity
semantic-level interaction with video content
multimodal data and multimodal interaction
Uncontrolled environments
increased robustness
better generalization
17
18. In conclusion: some KEY TRENDS
Towards cognitive image understanding
increase number of categories
reduce level of supervision
integrate cognitive information
in other image processing tasks
global reasoning: relations, context, ...
modular systems
"In 10 years we will have the first systems that can
interpret a never seen photo or video and that can be put
into an unknown environment and react intelligently"
18
19. Resulting research challenges
Advanced video coding
coding techniques for very high resolutions
e.g., characteristics of 16x16 pixels
360p 720p 2360p
increasing coding efficiency without increasing
computational complexity (too much)
19
20. Resulting research challenges
New coding paradigm:
Distributed Video Coding
mobile video applications
multi-view video entertainment
visual sensor networks
wireless video cameras
wireless low-power surveillance
20
23. Resulting research challenges
Scalable graphics
move towards real-world applications
design, architecture virtual reality
cartography
education, entertainment
nedicine, modeling
23
24. Resulting research challenges
Long-term goal
adaptation and delivery framework for personalized
and immersive interactive multimedia experiences
super HD / 3D
Immersive Multimedia Experience
(“rollercoaster experience”)
sensory effects
Annotation
Universal Multimedia
Experience
Retrieval
Adaptation
Universal Multimedia
Access
Delivery 24
25. Resulting research challenges
Content adaptation and transcoding
HDTV
originally acquired
high definition material
SDTV Mobile (iPod)
legacy applications semantically repurposed
common pan/scan
editing
(AVID)
27. Resulting research challenges
Multimedia data analysis and feature extraction
texture detection and classification
content-based multimedia
information retrieval
multi-camera video surveillance
object detection and tracking
immersive control rooms
27
28. Future media – evolutions and trends
Camera selection:
Making sense of the
information overload
29. Future media – evolutions and trends
Stitching: making sense of it
captured data
30. Resulting research challenges
Metadata technology and exploitation
semantic web technology
W3C activities on media
annotation
Open Linked Data
linking external
domain knowledge
enable reasoning
30
31. Resulting research challenges
Metadata technology for media production
metadata is much more than ‘data about media objects’
pursue model-driven product development
better communication: between staff and machines
during acquisition (associating audiovisual material)
for editing and repurposing (automate production tasks)
31
32. Resulting research challenges
Computer Graphics Video/film
+ Scene Editing and Interaction + Ease of capture and visualisation
Add / remove / clone objects Film with camera
Change shape / appearance / lighting Project on screen
Animation + Film “language” well understood
Interaction / Navigation
- Synthetic 3D models - Scene editing difficult, no navigation/interaction as in
realistic modelling is a painstaking task CG.
Video-Based Computer Graphics bridges the gap
(multi-view video as shape & appearance primitive in CG)
34. Resulting research challenges
Game technology
efficient
production tools
(e.g., digital sculpting)
real-time interaction with environment (physics)
34
35. Resulting research challenges
Application domains: virtual communities, gaming,
networked virtual environments, …
Research topics:
scalability issues
scalability assessment
37. Resulting research challenges
Media-aware
security and authentication
watermarking
fingerprinting
content authentication
Applications
control of quality, legitimacy
and authenticity of content
disclosure of digital archives
content distribution control
art authentication and dating
37
38. Achievements - a lot of projects...
HI-MASQUERADE
MUTABLE
IFIP
POKUMON
TELEON
GEISHA
PECMAN
COCOMEDIA
VIDEO Q-SAC
ISYSS
VICATS...
38
39. Achievements - spin-off companies
Androme
Voice and video over IP solutions
iDTV
computer graphics&animation
networked virtual environments
aQuartic
media streaming environment
metadata handling and exploitation
semantic web technologies (RDF, OWL)
content adaptation software platform (‘NinSuna’)
39
40. Achievements - spin-off companies
Eyetronics
systems & services for 3D scanning
high quality
flexible & efficient scanning
main applications: gaming & film production
GeoAutomation
3D acquisition of structural information in cities
based on mobile mapping (vehicle & cameras)
main application: digital surveying
40
41. Achievements - spin-off companies
QuESD
sports and video analysis systems
access control systems, including
embedded devices and video streaming solutions
eSaturnus
digital image acquisition & rendering
main applications: supporting surgery & endoscopy
41
42. Achievements - patents
Patents
active patent policy
while NOT excluding open-source approach!
some numbers:
5 granted
3 pending
42
43. Achievements - other
Strong participation in standardization
still image coding: JPEG
video coding: MPEG / VCEG / VQEG
semantic web: W3C
A lot of contract research
bilateral
various projects (EU, IWT, ...)
Strong collaboration with a lot of
recognized research partners
43
44. Contact info
IBBT Offices
Zuiderpoort Office Park
Gaston Crommenlaan 8 (bus 102)
B-9050 Gent-Ledeberg
Belgium
t: +32 9 331 48 00
f: +32 9 331 48 05
e: info@ibbt.be
Or even better: just talk to one of us during iMinds
44
