The emergence of smartphones equipped with Internet access, high resolution cameras, and posi- tioning sensors opens up great opportunities for visualising geospatial information within augmented reality applications. While smartphones are able to provide geolocalisation, the inherent uncertainty in the estimated position, especially indoors, does not allow for completely accurate and robust alignment of the data with the camera images. In this paper we present a system that exploits computer vision techniques in conjunction with GPS and inertial sensors to create a seamless indoor/outdoor positioning vision-based platform. The vision-based approach estimates the pose of the camera relative to the fac ̧ade of a building and recognises the fac ̧ade from a georeferenced image database. This permits the insertion of 3D widgets into the user’s view with a known orientation relative to the fac ̧ade. For example, in Figure 1 (a) we show how this feature can be used to overlay directional information on the input image. Furthermore we provide an easy and intuitive interface for non-expert users to add their own georeferenced content to the system, encouraging volunteering GI. Indeed, to achieve this users only need to drag and drop predefined 3D widgets into a reference view of the fac ̧ade, see Figure 1 (b). The infrastructure is flexible in that we can add different layers of content on top of the fac ̧ades and hence, this opens many possibilities for different applications. Furthermore the system provides a representation suitable for both manual and automatic content authoring.

1. A Vision-Based Mobile Platform
for Seamless Indoor/Outdoor
Positioning
Guillaume GALES
Eric MCCLEAN
John MCDONALD
DEPARTMENT OF COMPUTER SCIENCE
NATIONAL UNIVERSITY OF IRELAND MAYNOOTH

2. Introduction
Internet access
Accelerometer
GPS / Compass
Camera
2

3. Introduction
Vision-based platform
for positioning
Internet access
Position
Camera pose
3

4. Introduction
Vision-based platform
for positioning
Georeferenced façade
database
Façade recognition
system
Client applications
4

5. Introduction
Vision-based platform
for positioning
Client applications
5

6. Example of AR application
Mobile client side
Rendering
Server side
Façade Widget
extraction Retrieval
Database Authoring client
Façade
Matching
6

7. Example of application
7

8. Outline
Vison-based platform
• Façade extraction
• Georeferenced façade database
• Façade recognition system
• Camera pose estimation
Applications
• Augmented reality
• Navigation
• Positioning
Conclusion and perspectives
8

9. Vison-based platform

10. Façade Extraction
Key of the vision-based platform
• Georeferenced façades are the frame of references of 3D content
Input: image of a façade
Output: homography between the façade and its image
Advantages:
• Robust matching (invariant to rotation and perspective changes)
• Façade normalization (used to build a representation of the environment)
• Camera pose estimation (used by the visualization system)
10

11. Façade Extraction
11

12. Georeferenced façade database
For each street, take pictures of façades
Automatic façade extraction, matching and stitching
• Geometrical constraint makes matching robust
• Invariance to rotation and perspective changes
12

13. Georeferenced façade database
13

14. Georeferenced façade database
14

15. Façade recognition system
Candidate selection
• GPS coordinates
• Bag-of-word description for selecting
candidates
Similarity constraint
Figure 2: Database infrastructure for computing planar facade mo-
¸
saics. The individual facades are stitched together into planes to
¸
15 build a frame of reference for authoring.

16. Camera pose estimation
intrinsics are known
extrinsics are given by the façade extraction algorithm
(homography between plane and its image decomposed
into rotation and translation)
2 3
0 0 sh sh
⇥ ⇤ 60 h h 07
x=K R t 640
7
0 0 05
1 1 1 1 0
2 3
0 0 sh sh
= H 40 h h 05 H sh
1 1 1 1
h
1
H Façade
Camera
16

17. Applications

18. Augmented reality
Extension of façade to 3D: frame of reference
18

19. Augmented reality
Desktop application
• Easy and intuitive interface to use for non expert users
• Predefined list of 3D models
19

20. National University of National University of National University of
Ireland Maynooth
Ireland Augmented reality
Ireland Maynooth
Ireland
Ireland Maynooth
Ireland
T
horing is an important stage in the workflow of creat-
gmented reality applications. In this paper we describe
sed database infrastructure for authoring and storing 3D
use in urban environments. It provides frames of ref-
he environment as well as a mechanism to match new
h the facades and thus retrieving associated 3D content.
¸
ucture is flexible in that we can add different 3D “lay-
tent on top of the facades and hence opens many pos-
¸
Façade Extraction
augmented reality applications in urban environments.
e the system provides a representation suitable for both
Pose Estimation
automatic content authoring.
ds: Augmented Reality, Infrastructure, Authoring,
Façade Matching
ed Database, Content Storing and Retrieving.
DUCTION
Widget Retrieval useful in-
mented reality applications provide rich and
o their users about their surrounding environment. To
e augmented reality applications, an efficient infrastruc-
red. Such infrastructures involve:
ng a map of the environment ;
g content ;
ving content.
aper we propose an infrastructure that makes authoring
ve and flexible. Our goal is to create a platform for mo- 20

21. Results
21

22. Navigation
Vision-based platform
for positioning
+
Path finding algorithm
Outdoor/Indoor
navigation application
22

23. Positioning
Accurate positioning (ongoing work)
23

24. Conclusion and
perspectives

25. Conclusion and perspectives
Vision-based platform for positioning
• Georeferenced façade database
• Façade recognition system
Mobile applications
• Augmented reality
- Authoring solution
- User generated content
- Collaborative GI (HTML5)
• Navigation (Ongoing)
• Optimisations
25

26. Acknowledgment
Research presented in this paper was funded by a Strategic
Research Cluster grant (07/SRC/I1169) by Science
Foundation Ireland under the National Development Plan.
The authors gratefully acknowledge this support.
Thank you for your attention
26