Setting scaleX to -1 on a display object

Of course, you can easily place the video object back to where it was by compensating for the flip and adding the video object’s width to it’s current x value. While this solution works for most applications, there is one annoying thing about this method. If you ever want to move the video object around the stage or play with it’s scaleX value, there’s going to be a lot of compensating going on and it can get messy quick depending on the complexity of your application.

So to make it easier, here’s a class that extends the Video object and adds a “flipped” property. I don’t think I need to explain what the property does, but the class overrides the x and scaleX properties of Video so that it takes care of all the positioning when flipping the display object. Check out the demo [webcam required]:

Even when flipped, the x property always references the left side of the video object and the scaleX remains the same. Here’s the source: MirrorDemo.zip

Camera.activityLevel

The most basic way to detect activity through a webcam in flash is by accessing the Camera object’s activityLevel property. This returns a number from 0 to 100, depending on how much activity is going on. While useful in simple applications, this doesn’t give us a way to check for specific gestures or a way to check a specific section of the video for activity.

BitmapData.getPixel()

A popular way to detect motion is by using the BitmapData object in flash. By taking “snapshots” of the video coming in from the webcam constantly (using either Event.ENTER_FRAME, or the Timer object), we end up with frames that we can compare by looping through each pixel and checking to see which has changed. This technique is useful for checking which areas of the video has activity. We can use it to interact with buttons or other display objects by simply waving our hand over it. Though it can be very intensive (specially when working with video that has large dimensions), it is one of the best and most reliable way to detect motion in flash.

Augmented Reality & Marker Tracking

As mentioned earlier, another option is to use saqoosha’s FLARToolkit. If you haven’t played with it before, you should definitely check it out…after reading my blog post to the end, of course. From what I understand (and correct me if I’m wrong), this neat library also uses the BitmapData object to take stills of the video from the webcam. It then manipulates the bitmaps in order to detect and extract the position of a marker in 3D space. You can then use flash 3D libraries like Papervision to apply those coordinates to a 3D object and display it on top of the marker as if it’s a physical object. This method requires the user to print out a specific symbol and display it in front of a webcam. The wicked thing about FLARToolkit is that it doesn’t restrict you to using just Papervision, you can use Away3D or no 3D at all!

Though it comes packaged with Papervision and Away3D, with a bit of “code spelunking”, you can strip out the 3D stuff and gain access to the marker’s x,y, and z properties. By doing this, you end up with a basic marker tracking library which you can use for pretty much anything your nerdy heart desires. In my case, my nerdy heart and I created a very basic game where you have to catch falling apples using only simple vector MovieClips. Check it out below:

Download Source

Moving MovieClips with your hand (or other limbs) using the webcam

Now after playing with the BitmapData motion detection and the FLARToolkit, I went on a mission. A mission to find/figure out a way to interact with display objects on the stage using the webcam, BUT without using markers. After a bit of searching, I was able to find this demo on youtube which is close to what I want but with more control over the display object’s reaction.

Using what I’ve learned, and some inspiration from the youtube demo, I threw a quick example together as a learning exercise for myself. By checking for activity where our “box” movieclip is located, we can check which side has the most activity and move the box in the proper direction as if it’s a physical box you can push around. Though it’s not perfect and could DEFINITELY use some more code optimization, I was very satisfied with the result.

Download Source

AS3 Real-Time Face Detection

Another technique worth mentioning is face detection in flash. Here’s a snippet taken from Mario Klingemann’s blog about the subject:

“Yesterday Seb-Lee Delisle pointed us via Twitter to this great example of a Flash based real-time face detection. It turns out that already more than half a year ago Ohtsuka Masakazu had been porting the face detection part of OpenCV to AS3 and added the source code to the Spark project – which is like a Actionscript candy box full of surprises.”

He then explains how it works and how to optimize it, it’s a good read for those interested in the subject.

So there you have it, we went from detecting basic general webcam activity in flash to complex face detection algorithms. Big kudos to the folks mentioned in this post as they’ve put in many hours coding in order for the rest of us to create awesome and engaging content with ease.

We also knew that we wanted to create both the book assembly and viewing tool in Flash as opposed to Director. The tool is written in AS1 and uses Screentime Media’s MProjector software to create a standalone tool for creating the bundles of assets and text that are loaded into the viewer which works equally well locally or online.

What we see now as a version 1.0 tool has been used successfully by the Toronto Public Library in a public touch-screen kiosk format at the Metro Reference Library and had it’s widest release just recently as the core of a jointly produced project called Ontario Time Machine.

At this year’s SXSWi and the Museums and the Web conferences, the “VirtualBook” or “Page Turner” (it needs a real name) was greeted with interest by quite a few people from libraries and museums and it got us thinking about the possibility of a version 2.0 which would build upon the functionality and strengths of the existing version.

We know some of our basic goals in working towards a new version:

• Author in AS3 (bitmap handling and more structured code just a couple of the reasons)
• Streamline the book assembly process and application (cross-platform desktop application and/or hosted process)
• Make the viewing interface relatively “skinnable”
• Expand the display/export options (Flash viewer, HTML view, text only, PDF, etc.)
• Open the process and the product up to benefit from outside knowledge, criticism, and development

The first step in this is this post. We know there are a number of projects with similar goals out there in varying stages of production and we certainly have our personal opinions of where those tools succeed and fail but we would love some direct public feedback on what we have done with version 1.0 and on an on-going basis as we work towards a new version.

As an aside, here are a few companies and organizations with related goals or technology: Open Library, Google Book Search, Zoomify, and Issuu, Any others?

P.S. Please excuse the extremely punny title of this post. I shall endeavour to keep humour reigned-in during all future internet ramblings. Some may say it has been suitable reigned in that they noticed no humour present herein. To them, point taken.