AIGA JWU NEWS & UPDATES

For the past month or so I’ve been reading a lot up on Augmented Reality in the flash environment. According to Wikipedia, Augmented Reality is “is a term for a live direct or indirect view of a physical real-world environment whose elements are merged with (or augmented by) virtual computer-generated imagery – creating a mixed reality.”

I observed many examples including useful applications like the United States Postal Service Virtual Box and GE’s Smart Grid. I wanted to develop something similar but on a simpler scale. After a couple of hours of work, and a few energy drinks later, I created a virtual world for my DME1020 class using Augmented Reality. I’d like to share this experience with the JWU AIGA community.

1 ) There are a few files we need before we begin the project. The Flash Augmented Reality Toolkit (FLAR), some attributes from the Flex framework, the Papervision3D library, and a small adobe air application to create our marker, as well as a .dat file that will control the parameters for our webcam. All of these files can be downloaded via this link.

2 ) We need a 3D object to load whenever we see our marker. Since it is so close to Thanksgiving, I’ve decided to use a turkey. Using Google SkethUp’s 3D warehouse I found a .zip of a turkey to use, which includes the necessary files and textures needed. First, click on “Download Model” and “download Collada .zip”. Extract the zip to a folder on your desktop.

3 ) File and folder management is critical in this type of project. So in order to make sure we can access all of the files later on, we should invest some time in getting it neat and straight now. Below is a screen shot of proper folder management for this project:

Copy the contents of the turkey folder we downloaded from Google SketchUp and paste them into the Flash Project > assets > models folder. Copy the files we downloaded earlier and put them in the corresponding folders.

4) Next, we can make a marker for our project. The marker is what the camera will recognize and use as an x, y, and z axis for rendering our turkey. I created a 600 by 600  px image. Black background with a 450 by 450 px white square in the middle. I put the text “JWU AIGA” in the middle, but you can put whatever you wish. It is important to follow this black and white colorscheme or else the FLAR package won’t recognize our marker. Save this in the root of our Flash Project Folder as a jpg, gif, or png.

5 ) Next step is to print out, or load this new marker that you’ve made on your smart phone. Open up the ARToolKit Marker Generator that came in the zip file included above. If you hold up the marker you’ve made a red outline should appear around the marker. While this red line is there, click “Save Pattern” and save it as Flash Project > assests > FLAR > FLARpattern.pat.

6 ) Now it’s time to jump into flash. Create a new Actionscript 3.0 .fla, 30 fps with dimensions 640 by 480 px.

7 ) Create a new Actionscript File (.as). This is where we will adding all of our code, we will attach this .as file to our .fla in a later step. First thing we will do in our code is add all of our imports:

--------------------
package {
import flash.display.BitmapData;
 import flash.display.Sprite;
 import flash.events.Event;
 import flash.media.Camera;
 import flash.media.Video;
 import flash.utils.ByteArray;

 import org.libspark.flartoolkit.core.FLARCode;
 import org.libspark.flartoolkit.core.param.FLARParam;
 import org.libspark.flartoolkit.core.raster.rgb.FLARRgbRaster_BitmapData;
 import org.libspark.flartoolkit.core.transmat.FLARTransMatResult;
 import org.libspark.flartoolkit.detector.FLARSingleMarkerDetector;
 import org.libspark.flartoolkit.pv3d.FLARBaseNode;
 import org.libspark.flartoolkit.pv3d.FLARCamera3D;

 import org.papervision3d.lights.PointLight3D;
 import org.papervision3d.materials.shadematerials.FlatShadeMaterial;
 import org.papervision3d.materials.utils.MaterialsList;
 import org.papervision3d.objects.parsers.DAE;
 import org.papervision3d.objects.primitives.Cube;
 import org.papervision3d.render.BasicRenderEngine;
 import org.papervision3d.scenes.Scene3D;
 import org.papervision3d.view.Viewport3D;
--------------------

What this code does is imports all the external classes needed from the flash player, flash ar tool kit, and paper vision 3d needed for this project to work properly.

8 ) Next we will define our class, setup our class properties and variables

--------------------
public class AugmentedReality extends Sprite
 {

 private var video    : Video;
 private var webcam    : Camera;    

 private var flarBaseNode                : FLARBaseNode;
 private var flarParam                   : FLARParam;
 private var flarCode                    : FLARCode;
 private var flarRgbRaster_BitmapData    : FLARRgbRaster_BitmapData;
 private var flarSingleMarkerDetector    : FLARSingleMarkerDetector;
 private var flarCamera3D                : FLARCamera3D;
 private var flarTransMatResult          : FLARTransMatResult;
 private var bitmapData                  : BitmapData;
 private var FLAR_CODE_SIZE              : uint         = 16;
 private var MARKER_WIDTH                : uint         = 80;

 [Embed(source="./assets/FLAR/FLARPattern.pat", mimeType="application/octet-stream")]
 private var Pattern    : Class;

 [Embed(source="./assets/FLAR/FLARCameraParameters.dat", mimeType="application/octet-stream")]
 private var Params : Class;

 private var basicRenderEngine           : BasicRenderEngine;
 private var viewport3D                  : Viewport3D;
 private var scene3D                     : Scene3D;
 private var collada3DModel              : DAE;

 private var VIDEO_WIDTH                 : Number = 640;
 private var VIDEO_HEIGHT                : Number = 480;
 private var WEB_CAMERA_WIDTH            : Number = VIDEO_WIDTH/2;
 private var WEB_CAMERA_HEIGHT           : Number = VIDEO_HEIGHT/2;
 private var VIDEO_FRAME_RATE            : Number = 60;
 private var DETECTION_THRESHOLD         : uint   = 80;
 private var DETECTION_CONFIDENCE        : Number = 0.5;
 private var MODEL_SCALE                 : Number = 0.2;                

 private var COLLADA_3D_MODEL            : String = "./assets/models/models/bird.dae"; // make sure you rename the original .dae file
--------------------

9 ) Next we will setup the constructor. The constructor will do 4 things. It will prepare the webcam, the marker detection, and the rendering for papervision3D. Then, it will repeatedly loop to update the marker and the rendering of our turkey

--------------------
public function AugmentedReality ()
 {
 prepareWebCam();
 prepareMarkerDetection();
 preparePaperVision3D();
 addEventListener(Event.ENTER_FRAME, loopToDetectMarkerAndUpdate3D);
 }
--------------------

10 ) Next step we will define all of the methods listed in step 9.

--------------------
private function prepareWebCam () : void
 {
 video = new Video(VIDEO_WIDTH, VIDEO_HEIGHT);
 webcam = Camera.getCamera();
 webcam.setMode(WEB_CAMERA_WIDTH, WEB_CAMERA_HEIGHT, VIDEO_FRAME_RATE);
 video.attachCamera(webcam);
 addChild(video);
 }

private function prepareMarkerDetection () : void
 {
 flarParam = new FLARParam();
 flarParam.loadARParam(new Params() as ByteArray);
 flarCode = new FLARCode (FLAR_CODE_SIZE, FLAR_CODE_SIZE);
 flarCode.loadARPatt(new Pattern());

 bitmapData = new BitmapData(VIDEO_WIDTH, VIDEO_HEIGHT);
 bitmapData.draw(video);
 flarRgbRaster_BitmapData = new FLARRgbRaster_BitmapData(bitmapData);
 flarSingleMarkerDetector = new FLARSingleMarkerDetector (flarParam, flarCode, MARKER_WIDTH);
 }

private function preparePaperVision3D () : void
 {
 basicRenderEngine     = new BasicRenderEngine();
 flarTransMatResult     = new FLARTransMatResult();
 viewport3D             = new Viewport3D();
 flarCamera3D         = new FLARCamera3D(flarParam);
 flarBaseNode         = new FLARBaseNode();
 scene3D             = new Scene3D();
 scene3D.addChild(flarBaseNode);

 collada3DModel = new DAE ();
 collada3DModel.load(COLLADA_3D_MODEL);
 collada3DModel.scaleX = collada3DModel.scaleY = collada3DModel.scaleZ = MODEL_SCALE;
 collada3DModel.x = -30;
 collada3DModel.y = -530;
 collada3DModel.rotationX = 90;
 collada3DModel.rotationY = 0;
 collada3DModel.rotationZ = 0;

 flarBaseNode.addChild(collada3DModel);
 addChild (viewport3D);
 }

//and finally

private function loopToDetectMarkerAndUpdate3D (aEvent : Event) : void
 {
 bitmapData.draw(video);

 try {

 if(    flarSingleMarkerDetector.detectMarkerLite (flarRgbRaster_BitmapData, DETECTION_THRESHOLD) &&
 flarSingleMarkerDetector.getConfidence() > DETECTION_CONFIDENCE                            ) {

 flarSingleMarkerDetector.getTransformMatrix(flarTransMatResult);
 flarBaseNode.setTransformMatrix(flarTransMatResult);
 basicRenderEngine.renderScene(scene3D, flarCamera3D, viewport3D);
 }
 } catch (error : Error) {}
 }
 }
}
--------------------

11 ) This concludes the amount of code we need. Lastly, let’s jump back to the .fla we created and attach this Actionscript (.as) file to it.

12 ) That’s the last step, export the fla, grant the swf permission to use your camera and hold your marker up to the webcam.

All in all, augmented reality is still low in functionality and uses a large chunk of CPU power, however there is a pretty big “awe” factor involved. I put a custom marker of my own on the back of my business cards and it goes over extremely well during interviews. It shows that you are up-to-date on web trends and are there to impress.
And of course, I’ve included the source files with all of the work. However, in order to get the most out of this tutorial I recommend following the tutorial and only making references to the source for help if you get stuck. The source files are greatly //commented for this reason.

Thanks a lot for taking the time to read my blog post and tutorial. If you get the opportunity post your own AR experiments in a comment.

Also, if you have any questions please post them in a comment so everyone can benefit from them.

Robert Erskine
Member since 2009

This entry was posted on Tuesday, December 1st, 2009 at 1:49 pm and is filed under Design Inspiration, Student Work, Tutorials. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.