By now you have heard about Flash Augmented Reality and how it is taking the Flash Development community by storm! Whether you are looking for how to get started, how to improve your own experiments or have a client who desperately needs AR on their site, this session is for you. In this keynote I cover getting up and running as well as the ideal workflow for testing/deploying your creation. I also cover the basics then quickly move into how to build a FLAR Emulator for easy testing/debugging as well as general usability/performance issues. Finally we will look at my own experiments with AR, how they were built and highlight some of the best uses of Flash AR today. The goal of this presentation is to teach you how to build a solid reusable foundation for all of your Flash AR projects which will allow you to quickly prototype your ideas. All code covered in this session is open source and free to use. Documentation on how it works will also be handed out as well.

1. FLAR Workflow

2. Who is Jesse Freeman?

3. Technical Architect at
Roundarch
10 Years of Flash experience
Worked for
Heavy.com,
Major League Baseball,
The New York Jets,
AKQA, HBO, FOX,
Arista Records and BBDO
Specialize in Flash workflow,
large scale xml driven Flash apps
and building Flash Frameworks. Twitter - @TheFlashBum

4. Let’s Talk about
Augmented Reality

5. Augmented Reality (AR)
is a field of computer research which deals with the
combination of real-world and computer-generated data
(virtual reality), where computer graphics objects are
blended into real footage in real time.

6. What is FLAR?
• Flash Augmented Reality allows us
superimpose graphics over video.
• In Flash, this is usually done with a webcam
and a marker card.
• When you hold the card up to the webcam,
the FLARToolKit is able to detect the
orientation of the marker and superimpose, in
this case, a 3d model on top of it.

7. Jargon

8. FLARToolKit
is the name of the Library we will use in order to
implement AR in our project. This library was created by
Saqoosha and is based on NyARToolkit 2.0.0. It is available
as open source under the GNU General Public License.

9. camera_para.dat
is a binary file that is required by the FLARToolkit in
order to run. It represents setting and configuration
values for web cams.

10. Marker or Pattern
is the name of the graphic FLAR will analyze and
calculate its orientation from. This graphic is a special
box with a black border and a graphic inside. You need
this in order to get AR working.

11. My Background
with FLAR

12. • Was inspired by the
AR GE Flash Site.
• Began writing a library
for quickly creating
FLAR projects and wanted a way to
Debug/Test them.
• Was asked by O'Reilly's InsideRIA.com to
write a introduction for using
FLARToolKit.

13. NASA + FLAR

14. • While at my last job at @radical.media,
they selected by the Southwest Research
Institute to launch a website for a future
NASA mission to Jupiter called Juno.
• My job was to evaluate the technology,
understand its limitations/advantages to
create a strategy for implementing it into
the JUNO site.
• I created 2 AR demos for NASA to review
as a proof of concept of the technology’s
potential.

15. NASA Demo I

16. AR Solar System

17. Why talk about
workflow and
Augmented Reality?

18. Building AR apps have 3
workflow roadblocks

19. I
Compiling, launching,
and accessing the
camera is time
consuming

20. II
Creating and printing
out new markers for
testing kills trees and is
“expensive”

21. III
How can we prototype
new ideas without
writing a lot of new
code?

22. We need to make a tool

23. FLARVision Emulator
Flash Augmented Reality + PaperVision 3D

24. Architectural
Approach

25. Incapsulate
Repetitive Tasks
• Setting up FLARToolKit
• Setting up Papervision
• Creating a video source
• Triggering debug mode
• Reusable code base

26. Setting Up The
FLARToolKit

27. Steps Involved
• Load the camera file
• Load the marker file
• Set a canvas width, height
• Define marker width (scale)
• Create a source (bitmap used to detect the
marker)
• Create a Transform Matrix
• Set the threshold & confidence when detect
the marker

28. Thats a lot of ...
.. to set up!

29. ARDetector Class
/**
* Creates the AR Detector class and have it load in the camera.data
* and pattern.pat files.
*
*/
protected function createFlarDetector():void
{
arDetector = new ARDetector();
arDetector.addEventListener(Event.COMPLETE, onActivate);
arDetector.setup('data/camera_para.dat', 'data/flarlogo.pat');
}
/**
*
*/
protected function onActivate(event:Event):void
{
init();
}

30. Encapsulation
the process of compartmentalizing the elements of an
abstraction that constitute its structure and behavior;
encapsulation serves to separate the contractual
interface of an abstraction and its implementation.

31. ARDetector Interface
public interface IARDetector {
function get flarParam():FLARParam;
function set src(target:BitmapData):void;
function setup(cameraURL:String, markerURL:String):void;
function calculateTransformMatrix(
resultMat:FLARTransMatResult):void;
function detectMarker(
threshold:int = 90,
confidence:Number = .5):Boolean;
}

32. Setting up Papervision
• Everyone has their own way of doing it.
• I like to keep my options open.
• Create a simple method in the Main class
anyone can override with their own
settings.
• Default setup should be exactly what you
need to get up and running quickly.

33. Setup Method
public function setupPapervision():void
{
scene = new Scene3D( );
camera = new FLARCamera3D( arDetector.flarParam );
// Create the Viewport
viewport = new Viewport3D( stage.stageWidth, stage.stageHeight, true );
addChild( viewport );
// The base node is where all PV3D object should be attached to.
baseNode = new FLARBaseNode( );
scene.addChild( baseNode );
create3dObjects( );
renderer = new BasicRenderEngine( );
addChild( new StatsView( renderer ) );
}

34. create3dObjects Method
/**
* This default function is where 3d Objects should be added to PV3D's
* scenes.
*/
protected function create3dObjects():void
{
var plane:Plane = new Plane( new WireframeMaterial( 0xff0000 ), 80, 80 );
plane.rotationX = 180;
baseNode.addChild( plane );
}

35. Have it your way!
• If you want to customize your
pv3d setup simply override the
createPapervision method.
• If you want to add your own
custom 3d object simply
override the creat3dObject
method.

36. Video Source(s)
• FLARToolKit relies on analyzing a bitmap to
detect if the marker is present.
• In a normal setup we would draw the
WebCam “video” feed into a bitmap and
pass the bitmapdata over to the
ARDetector.
• Why use just a Video feed?

37. MarkerEmulator
• First I setup a second papervision instance
with a single plane.
• Next I used a JPG of the marker for the
plane's texture.
• Then I have the plane follow the mouse's
movement.
• Finally I pass the PV3d BitmapData into our
ARDetector.

38. Marker Emulator as
a Source

39. Debug Mode
• So now we have 2 feeds one from the web
cam and the other from our second
papervision instance
• We can switch between the two at any
time by pressing the spacebar.
• There is only one bitmapdata instance the
ARDetector watches so we simply switch
between the two sources when we sample
the bitmapdata

40. ARDetecor Source
/**
* Creates a Bitmap for us to scan for valid markers.
*
*/
protected function createCaptureSource():void
{
capturedSrc = new Bitmap(
new BitmapData( arDetector.width, arDetector.height, false, 0 ),
PixelSnapping.AUTO, true );
arDetector.src = capturedSrc.bitmapData;
addChild( capturedSrc );
}

41. Switching between feeds
/**
* Updates the capturedSrc with new bitmap data.
*/
protected function updateCaptureBitmap():void
{
if(debug || ! video)
{
cardEmulator.render( );
capturedSrc.bitmapData.draw( cardEmulator.viewport );
}
else
{
capturedSrc.bitmapData.draw( video );
}
}

42. Lets see a demo

43. Now what?

44. Use Case
• So I have this idea for a AR project
• I want to show a cardboard iPod when I
detect the marker
• I don’t want to spend a lot of time
configuring the project.
• How do I write the least amount of code
to test out my 3d model?

45. CPod FLAREmulator
Demo

46. CPod Demo Code
package
{
import com.flashartofwar.pv3d.componets.CPodContainer;
public class FLAREmulatorDemo extends FLAREmulator
{
public function FLAREmulatorDemo()
{
super( );
}
override protected function create3dObjects():void
{
var cpod:CPodContainer = new CPodContainer();
cpod.attachTo(baseNode) ;
}
}
}

47. And that is just one of the
many workflow tools I build...

48. Automating Builds

49. Why Use ANT?
• Ant is cross platform
• Ant works across multiple IDE’s
• Ant can be versioned
• Ant helps automat complex/repetitive
tasks.

50. Back To AR In Flash

51. Things To Keep In Mind
• Detecting the marker with FLAR is CPU
intensive.
• True 3d in Flash is not native so we rely on
software-based 3d engines like PV3d &
Away 3d.
• Webcam video playback at high resolutions
is intensive. There is no hardware
acceleration for live video.

52. The Future of FLAR

53. • There is a branch of FLARToolKit that uses
Alchemy to speed up the detection
process.
• We need better 3d support from Flash
Player.
• More developers and open source projects.
• Continued interested from companies and
exposure to AR outside of the Flash
Platform.

54. Virtual Physical Computing
- VPC -

55. • In physical computing you use an external
IO device to control what happens on the
screen.
• In Flash we are limited to a mouse and
keyboard.
• FLAR opens up a new way to manipulate
objects on the screen, especially 3d objects.

56. USPS Virtual Box
Simulator by AKQA

57. FLAR Innovators

58. Mikko Haapoja
http://www.mikkoh.com/blog/

59. Eric Socolofsky - FLARManager
http://transmote.com

60. James Alliban
http://jamesalliban.wordpress.com/

61. Seb Lee-Delisle
http://sebleedelisle.com/

62. FLAR Resources

63. http://www.libspark.org/wiki/saqoosha/FLARToolKit/en

64. http://flartoolkitdocs.org/

65. http://flaremulator.flashartofwar.com

66. The Fine Print

67. This Is Just A Workflow Tool
• This is not production ready code
• This was not meant to ever be used by
someone who is not a developer
• I cut a lot of corners to make this fit my
own needs, you should too
• The extensibility of this FLAREmulator may
have been grossly exaggerated
• Use at your own risk!

68. Thank You For Watching
My Site - http://flashbum.com
My Blog - http://flashartofwar.com
Source Code - http://github.com/theflashbum/FLAREmulator