This document proposes using HbbTV to implement LinkedTV, which links web content to broadcast TV programs. It discusses challenges of a live broadcast scenario and outlines a potential solution. Key points include: (1) Using HbbTV to trigger LinkedTV apps and request companion screen access. (2) Receiving annotations for the TV program via the broadcast stream or polling. (3) Managing connections and displaying concepts/links across main and companion screens synchronized to the TV program timeline. The document analyzes required HbbTV 2.0 functionality and open issues.

1. LinkedTV position paper on HbbTV for Linked Broadcast Television
authors:
Lyndon Nixon (MODUL) lyndon.nixon@modul.ac.at
Jan Thomsen (CONDAT) jan.thomsen@condat.de
thanks to Christoph Zieger (IRT) for input and guidance
last edit: 10­7­2014
Introduction
In the context of fundamental changes in the television experience through the introduction of
Internet connectivity [1], viewers are increasingly engaging in parallel Internet usage to their TV
consumption. While SocialTV is one aspect of this, integrating Internet content such as
Facebook or Twitter alongside the program on the TV set, there is also a trend towards looking
up additional information related to the program being watched using a separate, “second
screen” device. Surveys in 2012 found that 75 to 85% of TV viewers are also using a second
screen while viewing TV, while 37 to 52% are using that screen to follow what was going on in
the TV program [2] (A more recent survey found 40% [3] – with Google the first place to search
for program­related
information). Because viewers have limited means to conduct many of the
information searches they desire, e.g. finding out which painting is hanging on the wall behind a
character is nearly impossible to formulate as a textual query to Google, such searches do not
occur. On the other hand, providing such links as an additional service via connected devices
can be a means to create new value for original television content.
Current approaches focus on costly, fully manual curation of related information for selected
episodes of TV programmes and that information is hardcoded and related mostly to the whole
episode rather than specific topics or objects. This is also largely the preserve of OTT (over the
top) providers (e.g. Shazam or IntoNow, as apps running on a STB or a second screen device)
with whom content owners such as broadcasters must collaborate, without direct access to the
viewers who use that third party's app or feedback on the use of the second screen. LinkedTV1
is a project focused on using open technologies to enable any content owner to provide relevant
links via the Web associated to objects and topics within their content synchronized with its
playback on TV.
HbbTV2 is a European standard for hybrid broadcast­broadband
television which offers
broadcasters the opportunity to offer added value applications alongside their own content, under
their own control. Current extensions of HbbTV (2.0) are considering support for HbbTV
applications to share content with second screen applications and synchronise content to the TV
program3, and hence HbbTV offers an appealing technology for building a LinkedTV application
that broadcasters can insert into their own digital broadcast streams and enable the TV viewer to
call up and browse the related information on a second screen.
1 http://www.linkedtv.eu, LinkedTV overview slides in the right hand column

2. 2 http://www.HbbTV.org
3 http://www.hbb­next.
eu/index.php/archive/37­standardisation­related/
204­news­46
Proposal for our HbbTV prototype implementation
The current LinkedTV implementation takes a fully Web­centric
approach, so we will analyse the
extent to which this can be directly ported into the HbbTV environment and additionally the
possibilities that HbbTV ­now
or in the future ­can
offer through direct API access to the
television and to the companion screens. A hybrid prototype using HbbTV 1.5 on a current
SmartTV will be prepared and shown at IFA 2014 (Berlin, September 2014). Parallel to this, we
are identifying functionality gaps in the current HbbTV specification and are seeking to clarify if
we can introduce those functionalities towards a “fully operative” LinkedTV over broadcast
television once HbbTV 2.0 is defined or whether there are still open issues which we need to
propose solutions for beyond the HbbTV 2.0 release.
The current Web­based
LinkedTV implementation works like this:
(1) The “TV stream” is accessed on a first (main) screen via a particular URL ­in
this case
the “TV stream” is a recorded video streamed from a Web server via HTTP. The URL is not that
of the video file itself, but opens a server­side
application which loads and plays back the video
within the device full screen.
(2) The LinkedTV Player is accessed on any other (companion) screen via opening the
same URL ­the
‘multi­screen
toolkit’ software on the server is able to manage the different
screens that are connected and coordinate the content between them. The Player’s default UI is
focused on showing the viewer the concepts detected by LinkedTV in the TV program as well as
the distinct chapters (or segments) of the TV program (e.g. different news stories within a news
show). For clarity, the concepts are split across rows along specific types, e.g. “Who”, “What”
and “Where”, and when a chapter is active, only the concepts in that chapter are shown.
(3) The server monitors the timepoint of the TV stream and sends updates to the companion
screens. The Web application on the companion screen is largely independent of the main
screen apart from video controls which allow the viewer to pause or skip on the main screen.
The LinkedTV implementation using HbbTV 1.5 is foreseen to work like this:
(1) A “TV stream” is accessed as a channel in the TV which is tuned to a broadcast stream
which is, in fact, a recorded RBB news program being streamed from LinkedTV servers.
(2) An entry in the Application Information Table (AIT) of this (faked) broadcast stream
informs the HbbTV device of the availability of Linked Television functionality ­concepts
and
enrichments for the coming TV program.

3. (3) Through the Red Button functionality, the application bar is shown and a LinkedTV
application appears there for the current TV program. If the viewer chooses this app, a QR code
is displayed on screen.
(4) The viewer must use a QR reader app on their companion device and via reading the
displayed QR code and their giving of permission, an URL is opened in the browser of the
companion device.
(5) The ‘multi­screen
toolkit’ on the server is managing the companion devices connecting
via the provided URL and monitoring the broadcast stream that the URL is associated to. The
identity and timepoint of the TV program running on the broadcast stream is known to the
server­side
application (hardcoded).
(6) The toolkit manages the sending of updates to the companion screens based on the
timepoint of the program in the broadcast stream. Since the main screen is also connected to
the same application, and in order to minimise disruption to viewer attention to the main screen

4. (the TV) when new concepts in the TV program are in focus (with access to related information
available from within the LinkedTV player), a short, small notification is displayed on the side of
the TV program itself so that the viewer can know (if desired) they can now get information about
that concept, as opposed to needing to check repeatedly the second screen.
To achieve this, a dedicated Web application has been implemented which manages both the
HbbTV “broadcast stream” and the LinkedTV Player data, hence being able to maintain the
synchronisation between both.

5. Challenges for a full HbbTV implementation for live broadcast
In a prototypical live broadcast TV scenario, the video stream is no longer a single, referenceable
URL and the timeline is not a single, finite temporal period. Let us consider how we could
foresee a fully integrated LinkedTV service in HbbTV­enabled
interactive broadcast television
based on HbbTV 2.x:
(1) A “TV stream” is accessed as a channel in the TV tuned to a live broadcast stream such
as RBB.
(2) The DVB Application Information Table (AIT) in this broadcast stream informs the HbbTV
device of the availability of Linked Television functionality ­concepts
and enrichments for the
current TV program.
(3) A LinkedTV HbbTV app appears in the app menu for the TV program when the Red
Button is pressed. If the viewer chooses this app, the HbbTV device searches for companion
devices on the home network and sends a permissions request to use a companion
device screen. The user of the companion device must agree to connect to the LinkedTV app.
The app then sends a command to open the LinkedTV Player on the companion screen(s) via a
dedicated URL.
(3b) Alternative single­screen
set­up:
the app uses part of the main screen for display
so it needs to request main screen space alongside a frame for the broadcast TV content.
(4) As the TV program runs, the LinkedTV Player registers for and now receives
annotation and enrichment data for the current program which it can cache and then
display in its User Interface ­in
LinkedTV, this is indications of concepts in the program and
associated links to related Web content for each concept. It could be polled from the Player to
the LinkedTV Platform via IP (e.g. triggered by notifications from the main app which are received
as StreamEvents in the broadcast stream) but, in a broadcast environment, there is the option to
“pull” the data that is sent continually over the wire, cached for the time of activity and then
deleted. e.g. this could be a specialised data track delivered as a Timed Text track in the
broadcast stream. The app sends updates to the display areas (screen independent) for new
concepts or when concepts are inactive; it includes the interactivity to allow viewers to browse
and open related links for a concept, supporting different interaction modalities as appropriate
(on the TV it could be remote control, voice, gesture. On companion devices it is most likely
touch.)
(4b) Hybrid: the app can use a part of the main screen to show the currently active
concepts, and on viewer interaction with a concept, show the related content via a Web link
to a companion screen. This requires that the companion screen is already connected as per
step (3).
(5) As the program ends, or the LinkedTV app is switched off by the viewer, connections to

6. companion screens need to be closed or display areas on the main screen removed. The
system is not to lose resources waiting on annotations which no longer come, so the app needs
to unregister itself from the additional data stream or be unregistered automatically when the
program ends (we presume there is an End of Program marker in broadcast that the app can be
aware of).
The above is based on the simulation of a live broadcast stream, whereas a similar functionality
should also be supported in the case of catch­up
TV or Video on Demand (VoD) where the
programming is a recorded A/V stream delivered via broadband IP. Since DVB AIT is not
available when choosing an Internet video there needs to be another HbbTV trigger to
indicate this video is supported by LinkedTV, otherwise LinkedTV support of such material
will only be possible if the material is selected from within the LinkedTV app (still assuming the
Internet video identification to the HbbTV device is the same as the identification used by
LinkedTV). Then the system runs like above, steps (3)­(
5), except that there is no ‘pull’ of
LinkedTV data, rather the app needs to have an identifier for the video stream and access
to its timeline, so that it can then poll the LinkedTV Platform for any annotations and
enrichments of that content.
These are the HbbTV 2.x functionalities we foresee as necessary in LinkedTV:
Functionality HbbTV 1.5 HbbTV 2.x
trigger made available
DVB AIT for
in a broadband or
broadcast, nothing
broadcast stream on
for broadband
tuning in to content to
indicate availability of
LinkedTV enrichment
DVB AIT
How to send triggers in Catchup TV or VoD?
How to identify uniquely an IP A/V stream to a
HbbTV device?
searches for
companion devices on
the home network
X ­workaround
with QR Code
display
Yes? (Second Screen API)
sends a permissions
request to use a
companion device
screen
X ­companion
device opens URL
in the QR Code
Yes? (Second Screen API)
send requests to
open or close an URL,
or arbitrary
(application specific)
data to an application
on a companion
device
X ­the
opened
Web app on the
companion device
syncs directly with
the server
Yes? (Second Screen API)

7. receives annotation
data for the current
program in the
broadcast stream or in
a parallel broadband
connection
X ­poll
the data via
HTTP GET on
LinkedTV Platform,
triggered by
messages in
Stream Events
Broadcast the LinkedTV additional data in the
DVB stream, listened to only by active
LinkedTV apps
e.g. ­extend
Timed Text to have a track with
the LinkedTV annotation and enrichment data
?
References
1. S. Puopolo et al. “The Future of Television: Sweeping Change at Breakneck Speed” Point
of View document, Cisco Internet Business Solutions Group (IBSG), 2011.
2. The Guardian, “Social TV and second­screen
viewing: the stats in 2012”. Published Oct
29, 2012.
3. Jorge Abreu, Pedro Almeida, Bruno Teles, and Márcio Reis. “Viewer behaviors and
practices in the (new) television environment”. In Proceedings of the 11th European conference
on Interactive TV and video (EuroITV '13), Como, Italy, 2013.
Acknowledgements
Initiated by our position paper “Linking Web Content Seamlessly with Broadcast Television:
Issues and Lessons Learned” (PDF), accepted for presentation at the W3C Web and TV
workshop in Munich on March 13, 2014.