Marijn Speelman

At the moment I’m working on a webcam game as part of my Master thesis research. There are many Computer Vision libraries available in C/C++ (for example with optical flow implementations or face detection), but C++ isn’t very suited for rapid prototyping in respect to interface and game graphics.

Flash is very useful for this. Also, reading the webcam input is a lot easier in Flash than in C. So, what’s the idea: Flash reads the webcam, sends the image data to C++. Then the C++ application sends back the meta information about the images / frames, which Flash uses to draw the game graphics and execute actions in the game.

Zinc
My plan was to use Zinc. With Zinc you can convert a .swf file (Flash) to a .exe (Executable). Zinc is actually a container like a browser, using an integrated Flash Player. What’s nice about Zinc is that you can shape the window of your application like you want, and they provide additional API functionality. With the extra API calls you can access low level system commands like rebooting the PC, or information like the CPU speed. It also provides database connectivity with MySQL and Access.

Using the Zinc API you can also access external DLL’s. So, what I tried to do was use this DLL access to analyse the webcam data in an external C++ application that uses existing Computer Vision libraries.
To test the DLL connection, I wrote tiny test DLL (source). It consists of two functies, Init and Process. Init is a pure dummy function, Process returns a small test string.

Flash
Calling the external DLL in Flash (AS3):

// Create the Bitmap
var bmd:BitmapData = new BitmapData(30, 30, false);
var rect:Rectangle = new Rectangle(0, 0, 30, 30);
var bytes:ByteArray = bmd.getPixels(rect);

// DLL
var myDLL:DLL = new DLL("TestDLL.dll");
parameterIndex = myDLL.addParameter("integer", "120");
parameterIndex = myDLL.addParameter("integer", "120");
myDLL.call("none", "Init");
myDLL.clear();

// Send data to DLL
parameterIndex = myDLL.addParameter("unsigned char*", bytes);
var result:String = myDLL.call("unsigned char*", "Process");
myDLL.clear();
myDLL.close();

Result
And we’re done! You would say… Unfortunately, some time profiling shows even the Init dummy function takes 30ms to execute. Sending a 120×120 bitmap takes 180ms. As 10FPS can be considered a minimum you want for analysis, we can conclude this is too slow to be useful.
The next and final attempt (because of time constraints) is to use a C++ server and use a Socket to communicate between Flash and the server.