iPhone AR Landmark Tagging: From Hello World to GPS Located Overlays

iPhone Augmented
Reality
Jonathan Blocksom Jonathan Saggau
blocksom@gmail.com jonathan@jonathansaggau.com
@jblocksom @jonmarimba

About Us

• Jonathan Blocksom • Jonathan Saggau
(No PhD) (Also No PhD)

VR / AR History

• 1965
Sutherland

VR / AR History

• 1995 -
UNC,
Nintendo
• who is that? ------>

Existing AR Apps:
Theolodite
• Sort of like fancy binoculars

Existing AR Apps:
Layar
• Overlay markers on the world

Augmented Surreality:
Numen

AR Drone

• iPhone controlled remote ﬂyer
• Sends video stream to phone over Wiﬁ
• http://ardrone.parrot.com/parrot-ar-drone/

More Existing AR Apps

• http://mashable.com/2009/12/05/
augmented-reality-iphone/
• http://www.youtube.com/watch?
v=ps49T0iJwVg (acrossair)
• Augmentia
• Simple Geo
• smaato
• Layar
• Wikitude

From zero to hero: starting with the
“View based application” template
and ending with a tagged landmark

• Hello (Augmented) World
• Picture Level (rotation about 1 axis)
• Artiﬁcial Horizon (rotation about 2 axes)
• Artiﬁcial Horizon + Compass (add cardinal
direction / translation)
• GPS Locator: tagging a landmark (add
location / translation)

Baby steps to a better reality :
Hello (Augmented) World

• Goal: Overlay some text onto the live
camera image
• UIImagePicker + overlay
• Step 1: collect underwear
r3

UIImagePicker overlay
-(void)viewDidLoad
{
[super viewDidLoad];
picker = [[UIImagePickerController alloc] init];
picker.sourceType = UIImagePickerControllerSourceTypeCamera;
picker.showsCameraControls = NO;
picker.navigationBarHidden = YES;

picker.wantsFullScreenLayout = YES;

//1:1.124.. = aspect ratio of the screen
picker.cameraViewTransform = CGAffineTransformScale(picker.cameraViewTransform, 1.0f, 1.12412f);
}

-(void)viewDidAppear:(BOOL)animated
{
[super viewDidAppear:animated];
[self presentModalViewController:picker animated:NO];
UILabel *aLabel = [[UILabel alloc] initWithFrame:CGRectMake(10., 50., 300., 30.)];

[aLabel setText:@"Hello augmented world!"];
[aLabel setBackgroundColor:[UIColor clearColor]];
[aLabel setTextColor:[UIColor whiteColor]];
[aLabel setFont:[UIFont boldSystemFontOfSize:12]];
picker.cameraOverlayView = aLabel;
}

Picture (er, shopping cart) Level

• Goal: Overlay a level bar on the center of
the live camera view
• Requires: real time info regarding the
rotation of the camera about the Z-axis
• Adds UIAccelerometer
r7

-(void)viewDidAppear:(BOOL)animated
{
[super viewDidAppear:animated];
[self presentModalViewController:picker animated:NO];

label = [[UILabel alloc] initWithFrame:CGRectMake(10., 50., 300., 30.)];
[label setText:@"Hello augmented world!"];
[label setBackgroundColor:[UIColor clearColor]];
[label setTextColor:[UIColor whiteColor]];
[label setFont:[UIFont boldSystemFontOfSize:12]];

CGRect viewFrame = self.view.frame;
viewFrame.origin.x = viewFrame.origin.y = 0.0;
UIView *overlayView = [[UIView alloc] initWithFrame:viewFrame];
[overlayView addSubview:label];
[overlayView bringSubviewToFront:label];

// Load the image to show in the overlay:
UIImage *overlayGraphic = [UIImage imageNamed:@"artificialHorizon.png"];
overlayGraphicView = [[UIImageView alloc] initWithImage:overlayGraphic];
overlayGraphicView.frame = CGRectMake(30, 100, 260, 200);
[overlayView addSubview:overlayGraphicView];

picker.cameraOverlayView = overlayView;
[overlayView release];

[[UIAccelerometer sharedAccelerometer] setDelegate:self];
}

#pragma mark UIAccelerometerDelegate
- (void)accelerometer:(UIAccelerometer *)accelerometer
didAccelerate:(UIAcceleration *)acceleration;
{
double angle = -atan2(acceleration.y, acceleration.x) - M_PI/2.0;
double angleInDeg = angle * 180.0f/M_PI;
[label setText: [NSString stringWithFormat:@"Angle: %f", angleInDeg]];
overlayGraphicView.transform = CGAffineTransformMakeRotation(angle);
}

Video will go here

Artificial Horizon
• Goal: Keep the bar on the horizon (not just
level)
• Compensate for rotation along x-axis
• Need to know field of view of camera to
find horizon position
• ~53 degrees vertical
• ~37.5 degrees horizontal
r8

iPhone View Angles
53 Degrees Vertical*
37.5 Degrees Horizontal

*Close to this, but not exactly on the iPhone 4

{

overlayGraphicView.transform = CGAffineTransformIdentity;
double vertAngle = -atan2(acceleration.y, acceleration.z) - M_PI/2.0;
double vertAngleInDeg = vertAngle * 180.0f/M_PI;

[label setText: [NSString stringWithFormat:@"V-Angle: %f", vertAngleInDeg]];

CGRect overlayFrame = [overlayGraphicView frame];
overlayFrame.origin.y = 240.0 - 537.8 * sin(vertAngle);
[overlayGraphicView setFrame:overlayFrame];

double angle = -atan2(acceleration.y, acceleration.x) - M_PI/2.0;
//double angleInDeg = angle * 180.0f/M_PI;

}

Directional Horizon Marker

• Goal: Artiﬁcial horizon pegged to a speciﬁc
cardinal direction (North)
• Adds compass direction using
CLLocationManager

locationManager = [[CLLocationManager alloc] init];
[locationManager setDelegate:self];
[locationManager startUpdatingHeading];

r10

- (void)updateUI
{
[label setText: [NSString stringWithFormat:@"H-Angle:%5.1f V-Angle:%5.1f Heading:%5.1f",
angleInDeg, vertAngleInDeg, magCompassHeadingInDeg]];

CGPoint overlayCenter = [overlayGraphicView center];
overlayCenter.y = 240.0 - 537.8 * sin(vertAngle);
overlayCenter.x = 160.0 - 497.8 * sin((magCompassHeadingInDeg) * (M_PI / 180.0));
[overlayGraphicView setCenter:overlayCenter];

}

{
vertAngle = -atan2(acceleration.y, acceleration.z) - M_PI/2.0;
vertAngleInDeg = vertAngle * 180.0f/M_PI;

angle = -atan2(acceleration.y, acceleration.x) - M_PI/2.0;
angleInDeg = angle * 180.0f/M_PI;
[self updateUI];
}

#pragma mark CLLocationManagerDelegate
- (void)locationManager:(CLLocationManager *)manager didUpdateHeading:(CLHeading *)
newHeading;
{
magCompassHeadingInDeg = [newHeading magneticHeading];
[self updateUI];
}

GPS Hotel Locator

• Goal: Speakers must ﬁnd Hotel in any (um)
state, so we’ll tag it for them.
• Add GPS location with
CLLocationManager
• Compensate for “the hotel is over there
and I’m looking over here”

In other words:

Given the position of the hotel, the position of
the iPhone, and the direction the iPhone is pointing,
we want to know the cardinal direction of the
hotel relative to the iPhone’s pointing direction
and the elevation of the hotel relative the iPhone’s
pointing direction in order to overlay a label
onto the hotel in real time.

- (void)updateUI
{
CLLocation *marriot = [[LocationTools class] locationForMarriott];
float directionToMarriott = [[LocationTools class] angleFromCoordinate:[self.currentLocation coordinate]
toCoordinate:[marriot coordinate]];
CLLocationDistance dist = [self.currentLocation getDistanceFrom:marriot];
double deltaAlt = [marriot altitude] - [self.currentLocation altitude];
//verticalAccuracy of -1 == invalid
if ([self.currentLocation verticalAccuracy] < 0)
deltaAlt = 0;
double vertAngleToMarriott = atan2(deltaAlt, dist);
[marriotLabel setText:[NSString stringWithFormat:@"DirHotel:%5.1f Dist: %5f VertAngle: %f",
directionToMarriott * (180.0 / M_PI), dist,
vertAngleToMarriott * (180.0 / M_PI)]];

[anglesLabel setText: [NSString stringWithFormat:@"Horizon H-Angle:%5.1f V-Angle:%5.1f Heading:%5.1f",
angleInDeg, vertAngleInDeg, magCompassHeadingInDeg]];

double relativeDirectionToMarriott = magCompassHeadingInDeg * (M_PI / 180.) - directionToMarriott;
if (relativeDirectionToMarriott < (-M_PI / 2.0))
relativeDirectionToMarriott = (-M_PI / 2.0);

if (relativeDirectionToMarriott > (M_PI / 2.0))
relativeDirectionToMarriott = (M_PI / 2.0);

double relativeVertAngleToMarriott = vertAngleToMarriott - vertAngle;

CGPoint overlayCenter = [overlayGraphicView center];
overlayCenter.y = 240.0 - 537.8 * sin(relativeVertAngleToMarriott);
overlayCenter.x = 160.0 - 497.8 * sin(relativeDirectionToMarriott);
[overlayGraphicView setCenter:overlayCenter];
}

Gotchas
• GPS and compass haven’t enough accuracy
to place advertisements or labels on near
objects.
• On a “bad day” your iPhone’s version of
“here” might be a 1/2 kilometer circle
around a cell tower... or worse.
• Location updates will drain battery...
Quickly.
• But the dream is alive.

“Best practices... ” or “stuff we noticed
making a demo AR app”

• Don’t try to tag a near object, inaccuracies
compound as you near the target.
• Check altitude / location accuracy: Phone
can’t get altitude w/o good GPS ﬁx. See
documentation for accuracy (check for -1)
• The broad side of the barn is easier to hit.
• Bottom line: “here” isn’t here, yet.

Download the code

• http://code.google.com/p/ar360/

iPhone OS 3 Stack
:;;261*'&<=*$>")3$"77*$%
?@&4*"%&)9#3A%

:;0<<*7*$"2*3*$%4*7*163*%
!"#$%&'(")*%+,%-./% ?0<<*7*$"2*3*$A%
0#12*)3*4%5*67&38%099%

B"<6C")D6)61*$%
?E',F%>"296GGA%

iOS 4 Ideal
0:;"#)46<")%
=>&4*"%&)9#3?%

E"$*%B"<")%
!"#$%&'(")*%+,%-./% =0AA*7*$"2*3*$D%C8$"F?%
0#12*)3*4%5*67&38%099%

@"A6<")B6)61*$%
=C',D%E"296FF?%

AV Foundation

• Allows processing of the frame before
display
• Very complicated to setup
• Check out the WWDC samples and video

What’s next?
• Smoothing / Filtering Sensor Data (wobble
wobble)
• Full-on 3D AR world -- Dancing hippos in
swimming pools with Dolphins...
• Image tracking to mitigate the swings
•?
• Proﬁt!

How we might transition to
a Full virtual 3D World
• Need useful local coordinate system to
map onto OpenGL; we’re pretty close with
the demo code, actually.
• Match OpenGL transform matrices with
current position relative to some scene we
might want to add to the real world,
camera parameters; GLULookAt...
• Need to not mind that compositing could
be very slow...

ARKit :
Existing AR Toolkit for iPhone

• http://github.com/zac/iphonearkit/
• Marker based
• Simple CoreLocation - based data model
• Hasn’t been updated for a while

What counts as AR?

• Video with... ?
• Overlay?
• Must be Interactive?
• GPS?
• Modify the virtual environment?

AR News Sources
• Bruce Sterling’s Beyond the Beyond blog
http://www.wired.com/
beyond_the_beyond/
• Tish Shute:
http://www.ugotrade.com/
• Twitter streams:
@anselm, @bruces, @tishshute, @brady

• O’Reilly Radar:
http://radar.oreilly.com

iPhone AR Landmark Tagging: From Hello World to GPS Located Overlays

Recommandé

Recommandé

Contenu connexe

Similaire à iPhone AR Landmark Tagging: From Hello World to GPS Located Overlays

Similaire à iPhone AR Landmark Tagging: From Hello World to GPS Located Overlays (20)

iPhone AR Landmark Tagging: From Hello World to GPS Located Overlays