Using FLARToolKit, Papervision3D, and Video

By Howard Buksbaum filed under Code

February 13, 2013

Yesterday at around noon I got an email from Tom Beeby over at Beeby, Clark + Meyler asking me if I know anything about ARToolkit. He pointed me to this site from GE.  You can see a demo video of it here. So I printed the PDF as instructed and launched the GE demo. WOW!  This was the coolest thing I've seen in flash in a long time. Turns out it's not all that difficult to build your own experience. In this article I'll go over some of the basics I've learned during my 24 hour crash-course. I'll also provide some useful links I've found...

 

THE BASICS

Essentially the ARToolKit provides an interface for detecting visual markers from an image or video then determining it's position in 3D space. The FLARToolKit is an AS3 port of the ARToolKit developed by a Japanese programmer named SaqooshaPapervision3D is an open source realtime 3D engine for flash. Using FLARToolKit and Papervision3D together can create stunning results. I started by reading this very useful tutorial from Saqoosha.net. I also read through Mikko's tutorial.  For Papervision3D I found this tutorial. I have to say the concept of creating organic markers inside a video then mapping something to those markers is pretty cool. The nice thing about the tutorials I found was that the source code was relatively easy read. For the most part, customizing the code from the samples is a matter of changing values and making some minor tweaks. I was so impressed with the fact that I can make my own marker that I started there.  You can download my marker here. It was as easy as creating an 8.5"x11" document in Photoshop and drawing an asymmetric symbol in the center.  The purpose of the asymmetric symbol is to help FLARToolKit find your marker in 3D space.

 

BUILDING THE APP

Next I found this nifty little Adobe AIR utility that uses your webcam to create a pattern file from the pattern you've created in your marker.  You can also use this online marker generator.  Simply print out your marker.  Launch the app and hold your maker up to your webcam until the red box appears around the outside of your marker. Then click the Save Pattern button.  Make sure to add the extension .pat to the end of the filename.

 

You can find a list of helpful tutorials here. They all seem to start with an application called SimpleCube. I made some changes to SimpleCube and created SimpleVideo. Using SimpleVideo as an example, open the SimpleVideo.as file and look for the following line:

 

private static const PATTERN_FILE:String = "resources/vni_marker.pat"; 

 

You can replace the reference vni_marker.pat with your own filename. Because I used an FLV as the the 3D object that gets attached to the marker, I added the following code to tell the netstream object to loop the video.

 

private function statusHandler(eventObject:Object):void {
  if(eventObject.info.code == "NetStream.Play.Stop")  { 
    // loop video
    netStream.seek(0);	
  } 		
}

 

THE FINAL TEST

First print out this pdf of the marker I made. Then click here to try the demo of the SimpleVideo test. You'll get a warning from Flash about accessing your webcam. You can click allow. Make sure your webcam is configured as the video source. If not, right-click on the flash movie to modify your settings.  All you have to do is hold the marker paper in front of your webcam.  Notice how moving the paper close to and away from the camera changes the scale of the objects. If you rotate the paper along the z axis you will see the logo rotate to match the perspective.

 

download the vni marker pdf

download the source files

 

SCREENCAST

Here's a screencast of the SimpleVideo application in action.

CONCLUSION

There are many useful applications for this framework. Advertisers can use markers to add their brands to existing content or user created content.  Designers can use markers to enhance a 360° product demonstration by allowing users to control its position organically instead using a mouse and keyboard. Here's a great video showcasing how creative animations can be used to enhance a New Year's greeting card. Saqoosha has a demo here of a project he created that simulates cloth.

 

USEFUL LINKS:

source files

download FLARToolKit

download marker pattern generator

download multiple marker detector 

 

marker generator

use online marker generator

 

tutorials

Saqoosha.net

Mikkoh

Papervision2.com

Pv3d.org

Dehash

Multiple Markers

Transmote

Cinema 4D integration

Tarotaro.org

 

frameworks

FLARToolKit

Papervision3D