From 2D Map to Mobile 3D Mirror World，A Live Virtual Advertising Use Case

1. From 2D Map to Mobile 3D Mirror
World
A Live Virtual Advertising Use Case
Yu You, David Murphy
Mixed Reality Solutions
NRC, Tampere, Finland

2. Mobile 3D Mirror World
• Mirror world (MR) is “informationally-enhanced virtual
models or ‘reflections’ of the physical world” [2].
• Provides increased precision of an added dimension with more
lively and immersive UX
• A nice complement to Augmented Reality (AR), which is
commonly used in real-time and on-line situation
• Mobile MR can take advantage of mobile AR where mobile
sensors and other positioning and tracking technologies are
used
Nokia Research Center

3. Motivation
• study the mobile advertising scenarios from a narrow
subject (street-level mirror world) and try to answer an
immediate question: how and where to publish the
advertisement content?
• This paper focuses on:
• Accurate but simple content automatic placement
• Mobile content creation and visualization
Nokia Research Center

4. Experiment setup
• Data set
• 360 degree panoramic street-level imagery
• 3D building models with global geo. coordinates
• Ground-terrain 3D mesh data
• Technology
• OpenGL-based client renders the building models and ground
around the chosen panoramic image onto which the panoramic
image is projectedA mobile client
• Video 1: Nokia CityScene
Nokia Research Center

5. The system in a nutshell
• Web-centric end-to-end setup
with RESTful Web APIs for ad.
content management and
mobile 3D mirror world client
• WGS84 geo. boundary or
proximity queries
• KML/KMZ is the main data
representation format
• 3D objects are modeled separated
in COLLADA files
Nokia Research Center

6. Scalability consideration
• Unlike ad. content data, which is rather dynamic, any other static data
are stored in different servers, i.e. provided by Content Delivery Network
(CDN)
• Typically CDNs does not offer geo. query functionality
• Each 3D data has a geo. coordinate and then its geo-hash is calculated (e.g.
we use quad-key method) at a fixed resolution (zooming) level (i.e. 17, appr.
200m2)
• Therefore an index is created for all data in that geo. boundary and stored in
the file system using a directory naming convention, which is correspondent
to its geo-hash value. So does the real data.
• Client needs to fetch the metadata (index) files at one given geo. coordinate
and then all 3D data.
Nokia Research Center

7. Mobile 2D map-based content planning
1
• Two types of ad. content representation
1. Static object like billboards standing on one location
2. Mobile 3D object along the vehicle driving road
• 2D coordinates are needed only; the altitude is calculated from the
ground terrain mesh
• Problems 2
• Hard to position content accurately to the buildings, not
to mention the façade
• That reflects again the common problem in other
systems where geo-tagged data (e.g. point of interest)
are misplaced in the middle of streets; or even worse -
on the wrong side of the street
Nokia Research Center

8. 3D content manual placement
• High-value campaigns requires pixel-level accuracy
• Developed a on-device authoring tool for non-
expert users with following few steps
• Choose the appropriate building façade
• Drawing the size of the bounding box as the content
placement or drawing canvas
• Saving the drawing or inserted content with
metadata (e.g. the normal to the façade and
bounding box data in the 3D)
• The content placement layer can be used to render
any media data properly in 3D scene
• However, manual placement is time-consuming
and not scalable
Nokia Research Center

9. 3D content heuristic/auto placement
• Auto alignment from only 2D coordinates
• Select the closest building model (the center of mass of the surrounding
models)
• Determine the intersected façade by casting a light-ray to the model from a
predefined view point
• External media like banner images are converted to SVG format to scaling of
the scene (e.g. zooming in/out)
Nokia Research Center

10. Video 2
Moving box for parking info
Nokia Research Center

11. Findings and open issues
• Alternative ad. placement (e.g. façade vs. ground)
• Placing banners altered the appearance of the building facade, and thus deteriorated
the representational value of the mirror world
• Lighting, visual improvement
• Illumination enhancement e.g. artificial lighting and content blending with proper
shedder, but lighting blobs on facades and tree shadows are difficult to be segmented
• Proper ad. content selection for moving artifacts
• Fixed geo. location as the context wont always work for mobile artifacts
• Study more metrics concerning mobile/3D-specific user actions for analytical
processing
• Navigation, messaging etc.
• Zooming and panning
• Various pricing strategies vs. moving speed and frequencies
• Mobile Web app. vs. native application
• One solution for all OS
Nokia Research Center

12. Summary
• Bringing ad. content to 3D Mirror World allows
innovative types of advertising, coming along with
new challenges
• We’ve proposed two basic types of ad. approaches
but expect more realistic representations to appear
Nokia Research Center