Setting scaleX to -1 on a display object

Of course, you can easily place the video object back to where it was by compensating for the flip and adding the video object’s width to it’s current x value. While this solution works for most applications, there is one annoying thing about this method. If you ever want to move the video object around the stage or play with it’s scaleX value, there’s going to be a lot of compensating going on and it can get messy quick depending on the complexity of your application.

So to make it easier, here’s a class that extends the Video object and adds a “flipped” property. I don’t think I need to explain what the property does, but the class overrides the x and scaleX properties of Video so that it takes care of all the positioning when flipping the display object. Check out the demo [webcam required]:

Even when flipped, the x property always references the left side of the video object and the scaleX remains the same. Here’s the source: MirrorDemo.zip

Camera.activityLevel

The most basic way to detect activity through a webcam in flash is by accessing the Camera object’s activityLevel property. This returns a number from 0 to 100, depending on how much activity is going on. While useful in simple applications, this doesn’t give us a way to check for specific gestures or a way to check a specific section of the video for activity.

BitmapData.getPixel()

A popular way to detect motion is by using the BitmapData object in flash. By taking “snapshots” of the video coming in from the webcam constantly (using either Event.ENTER_FRAME, or the Timer object), we end up with frames that we can compare by looping through each pixel and checking to see which has changed. This technique is useful for checking which areas of the video has activity. We can use it to interact with buttons or other display objects by simply waving our hand over it. Though it can be very intensive (specially when working with video that has large dimensions), it is one of the best and most reliable way to detect motion in flash.

Augmented Reality & Marker Tracking

As mentioned earlier, another option is to use saqoosha’s FLARToolkit. If you haven’t played with it before, you should definitely check it out…after reading my blog post to the end, of course. From what I understand (and correct me if I’m wrong), this neat library also uses the BitmapData object to take stills of the video from the webcam. It then manipulates the bitmaps in order to detect and extract the position of a marker in 3D space. You can then use flash 3D libraries like Papervision to apply those coordinates to a 3D object and display it on top of the marker as if it’s a physical object. This method requires the user to print out a specific symbol and display it in front of a webcam. The wicked thing about FLARToolkit is that it doesn’t restrict you to using just Papervision, you can use Away3D or no 3D at all!

Though it comes packaged with Papervision and Away3D, with a bit of “code spelunking”, you can strip out the 3D stuff and gain access to the marker’s x,y, and z properties. By doing this, you end up with a basic marker tracking library which you can use for pretty much anything your nerdy heart desires. In my case, my nerdy heart and I created a very basic game where you have to catch falling apples using only simple vector MovieClips. Check it out below:

Download Source

Moving MovieClips with your hand (or other limbs) using the webcam

Now after playing with the BitmapData motion detection and the FLARToolkit, I went on a mission. A mission to find/figure out a way to interact with display objects on the stage using the webcam, BUT without using markers. After a bit of searching, I was able to find this demo on youtube which is close to what I want but with more control over the display object’s reaction.

Using what I’ve learned, and some inspiration from the youtube demo, I threw a quick example together as a learning exercise for myself. By checking for activity where our “box” movieclip is located, we can check which side has the most activity and move the box in the proper direction as if it’s a physical box you can push around. Though it’s not perfect and could DEFINITELY use some more code optimization, I was very satisfied with the result.

Download Source

AS3 Real-Time Face Detection

Another technique worth mentioning is face detection in flash. Here’s a snippet taken from Mario Klingemann’s blog about the subject:

“Yesterday Seb-Lee Delisle pointed us via Twitter to this great example of a Flash based real-time face detection. It turns out that already more than half a year ago Ohtsuka Masakazu had been porting the face detection part of OpenCV to AS3 and added the source code to the Spark project – which is like a Actionscript candy box full of surprises.”

He then explains how it works and how to optimize it, it’s a good read for those interested in the subject.

So there you have it, we went from detecting basic general webcam activity in flash to complex face detection algorithms. Big kudos to the folks mentioned in this post as they’ve put in many hours coding in order for the rest of us to create awesome and engaging content with ease.

• Remember that somebody else (who may not know all the ins and outs of XCode) will be working on the project with you. Take the time to keep things neat and organized. Create a Classes folder with all your source code and a Resources folder with nib files, images, sounds, etc. Keep the organization of the project and the organization of the files on disk as close as possible, it’s less confusing for everyone. Give all of your files consistent descriptive names. Delete files that aren’t being used anymore (Tip: Make bundles for any resources that don’t need to be compiled, like images and sounds. XCode doesn’t keep track of the contents of bundle files, which means when you add/remove/rename files in the bundle XCode will automatically see the changes so you don’t have to remember to add and remove them from the project). Simple stuff, but it makes a big difference.
• Pushing updates to the App Store is a proven way of improving a game’s visibility, so make sure the project and the code are easy to update. Remember to slow down and do things well.
• Use Foundation and UIKit classes wherever possible — it helps keep user interface elements consistent and it’s easy for another developer to quickly see what’s going on and make changes without learning another 3rd party framework. For example, use a UIView with UIButtons for game screens. The exception to the rule here is the game engine itself, where the benefits of using OpenGL or other open source code are worth the trouble.
• Use Interface Builder wherever possible — this goes hand-in-hand with the last point. Using UIKit classes and Interface Builder makes it easy to adjust the layout of screens, labels for buttons, etc. without hunting through the code. Remember, the person making the adjustments may not be a programmer.
• Use UILabels for interface text, rather than embedding text in images. Keeping text out of images makes the project much easier to update and localize. Using native text for everything also helps keep things consistent visually – your high score screen is going to need dynamic text and the app will look better if it matches everything else.
• Memory management is very high priority on the iPhone. Use as little memory as possible. Don’t load things into memory until you need them. Release things from memory when you’re done with them. For example, rather than loading all of a games screens when the app launches and keeping them in memory for the life of the app, load each screen when it’s going to be shown and release it from memory after it’s hidden.
• Test the app regularly in Instruments during development to find memory leaks and avoid wasting memory:
  • Check using Leaks — This will help you find good, old-fashioned memory leaks.
  • Check using Object Allocations — An object in memory may not technically be a leak, but if it doesn’t need to be there it’s wasted memory. When you’re playing the game, are any of the objects associated with the other screens in memory? When you’re showing the game over screen, are any of the objects associated with the game engine in memory?
  • Check using Memory Monitor — Apple doesn’t specify a hard limit on the amount of memory an app can use, but the general consensus is that 15Mb to 20Mb is fairly safe. If your app is using 25Mb of memory you should be concerned. If your app is using 45Mb of memory you have a very big problem.
  • Check OpenGL Texture Memory — OpenGL on the iPhone has a soft limit of 24Mb for all textures. If you go above the limit, there’s a massive performance hit. See: How To Check iPhone Texture Memory Usage With Instruments
• Follow standard Objective-C naming conventions. They’re different from other languages, so take the time to read up on them. See: Coding Guidelines for Cocoa, Cocoa Style for Objective-C: Part I, Cocoa Style for Objective-C: Part II
• Put the dealloc method at the top of the @implementation block for every class you write. dealloc is the most important method in every class, so show it some love.
• Use accessor methods for everything — it’s the best way to avoid simple memory management bugs. Use accessor methods within classes, too. You can define private accessor methods in a category at the top of your implementation file. You can redefine a readonly property in the public interface as a readwrite property in the private category.
• Mark properties as nonatomic unless you have a good reason not to.
• Use square bracket syntax (i.e. [self foo], [self setFoo:@"bar"]) instead of dot syntax (i.e. self.foo, self.foo = @"bar") for accessing properties. Using one syntax keeps your code consistent, and makes it easier to see where you’re accessing a property of an Objective-C object as opposed to a member of a C struct or C++ class (which use dot notation).
• Avoid subclassing UIViews. Build the view hierarchy in Interface Builder and put your logic in a UIViewController. If you have a lot of UIView subclasses, you should be concerned.
• Remember to mark properties as IBOutlets and methods as IBActions wherever it makes sense.
• Organize related methods and properties into groups in the source code (i.e. the UIViewController subclass for a game screen might have groups like button properties, action methods and overridden UIViewController methods). Mark the groups in both the header and the implementation using #pragma mark Label. Organize methods within a group alphabetically. You’ll spend a bit of time organizing things and a lot less time looking for things.
• Use NSAssert() liberally. If you intentionally stop the app when something unexpected occurs, it’s easier to find where the problem started and fix it. It takes a bit of time to write the assertions, but I guarantee it takes less time than tracking down a bug later on.

The list above is by no means definitive, but it covers the basics. Let me know if I’ve left out anything obvious, or if there’s something that rubs you the wrong way. I’m always happy to learn a better way to do something.

The instructions below assume you’re a registered iPhone developer and have:

• Mac OS X 10.5
• XCode 3.1.2
• iPhone SDK 2.2.1
• Box2D 2.0.2
• A valid app id and development provisioning profile that you can use to build the project for a device.

So, for my future reference and yours….

1. Create a new XCode project. I’m creating a View-Based Application called MyGame on my desktop.

   

   

2. Open the terminal and export Box2D 2.0.2 from the Box2D svn repository into your project folder:
   svn export https://box2d.svn.sourceforge.net/svnroot/box2d/tags/Box2D-2.0.2 ~/Desktop/MyGame/Box2D-2.0.2
3. Right click on MyGame under Groups & Files and choose Add > Existing Files…

   

4. Browse to the your project directory, select the Box2D-2.0.2 folder and click Add.

   

5. Make sure the Copy items into destination group’s folder checkbox is unchecked and click Add.

   

6. Expand the Box2D-2.0.2 group and delete everything except the Include and Source groups. Whether you move the files to the trash or just delete the references is up to you.

   

   

7. Expand the Source group and delete the Makefile.

   

   

8. If you haven’t already, set the app id (in Resources/Info.plist) and the Code Signing Identity in the project build settings (in Project > Edit Project Settings > Build). Make sure you have Device 2.2.1 and Debug selected in the menu at the top left corner of XCode (anyone know what this menu is called?). Finally, build the project (Command-B).
9. Open up the build results (Command-Shift-B), and you’ll see a whole bunch of errors, along the lines of:

   error: 'finite' was not delcared in this scope

   

10. Open the project build settings (Project > Edit Project Settings > Build) and find the Other C Flags setting. Add the following flag, which tells Box2D that it’s being compiled for the iPhone so that it knows where to find the standard C++ libraries.

    -DTARGET_OS_IPHONE

    

11. Build the project again (Command-B). It will build cleanly, but we’re not quite out of the woods yet.
12. Open MyGameViewController.m and add the following line at the top of the file:

    #import "Box2D.h"

    

13. Build the project again (Command-B). Yay, more errors:

    error: initializer element is not constant

    

14. Select all of the .m files in your project (yes, all of them).

    

15. Click the Info button in the toolbar, go to the General tab and find the File Type dropdown.

    

16. Change the filetype from sourcecode.c.objc to sourcecode.cpp.objcpp. This tells XCode to compile the files as Objective-C++ (mixed Objective-C and C++ code) instead of plain Objective-C code.

    

17. Build the project again (Command-B). That’s it!

So there you go, you’re ready to start using Box2D on the iPhone. Just remember to change the file type from sourcecode.c.objc to sourcecode.cpp.objcpp for any .m files you add to the project.

When developing iPhone games, one of the biggest performance optimizations you can make is to use OpenGL ES for rendering (instead of CoreGraphics or UIKit). For a 2D game like Arctic Shuffle, this means loading all of the animation into Open GL as textures, and drawing them onto simple rectangular shapes.

There are two caveats to this approach. First, disk access on the device is dirt slow, so you can forget about loading textures on the fly – all of the textures need to be loaded before gameplay begins. Second, OpenGL ES on the iPhone has a texture memory limit of 24MB. If you try to load more than 24MB of textures (including any buffers, like the framebuffer), the device has to start swapping them in and out of memory, and you go over a performance cliff, so to speak.

With that in mind, it’s very helpful to know how much texture memory is being used by your game – this is where Instruments comes in (the instructions below assume that your device is plugged into your development machine and you already have a working debug build of your game on the device).

1. Open Instruments (/Developer/Applications/Instruments.app) and choose the “Blank” template.

2. In the bottom-left corner of instruments, click on the gear icon, then choose Add Instrument > Memory Monitory. Click the gear icon again and choose Add Instrument > OpenGL ES.

3. Click on the OpenGL ES instrument’s info icon, and make sure that GARTResident Object Size is checked.

4. Choose the device and application you wan to check. My device is named Barracuda and the application is ArcticShuffle2.

5. Click the record button. This will launch the application on the device, and start monitoring it’s memory and OpenGL ES usage. The GARTResident Object Size column shows the amout of texture memory used, in bytes. Divide that by 1024 (i.e. amount / 1024 / 1024) to convert the number to megabytes. Notice how the Frames Per Second drops dramatically while the textures are loading.

Happy Optimizing!