Face detection using Flash and C++, revisited
Over the past few months I’ve been asked several times for the code for face detection in Flash. So I’ve taken some time to make a short demo application, grabbed from my original Virtual Mirror game (see my two previous blogs).
Here’s what it does: Flash grabs the webcam image every 60ms, and sends it using a socket connection to a separate C++ server that runs on your localhost. The server uses the OpenCV software library to detect faces, using a haar classifier cascade. When it detects one or more faces it sends the result (coordinates) back to Flash. There, Flash draws a transparent box or image at the coordinates of the face, like this:
What makes this cool is that you can perform quite complex image analysis (at least something you can’t easily do in Flash), and use this data on a platform that is highly suitable for multimedia development (animation, sound, nice graphics, etc).
The demo only handles one face now, but this could be easily expanded.
Download:
At the moment there are no installation instructions included. If you need help you can contact me, or wait untill I create the instructions (which I will) :)
A few quick pointers to get you past some obstacles:
- Binary: You probably have to add the folder you run the .swf from as trusted directory in your Flash settings. Otherwise you get a security / sandbox violation error.
- Binary: Start the server before starting the .swf (client). The .swf tries to connect to the server when it starts.
- Binary: The needed OpenCV dll’s and .xml file for face detection are included in the same folder.
- Source: (Server) If I remember correctly, only pthread is a required dependency. UPX is a file packer for releasing, Visual Leak detector can be turned off if you don’t want to use it.
- Source: (Server) Additional dependencies you should include when compiling/linking: wsock32.lib cv.lib cxcore.lib highgui.lib
Last note: This was both my first AS3 and C++ project, I’m always open to suggestions, improvements and cool new ideas. Also, when you find this code useful, I would appreciate it if you’d drop me a line.
March 26th, 2008 at 1:39 am
you mentioned in a previous post that c++ is not well suited to rapid prototyping, and game development. have you looked at openFrameworks. oF is a c++ framework that wraps a lot of openGL, QT, and openCV functions making it quick and easy to write effective programs (prototypes).
check out http://openframeworks.cc/download for a copy. [actually looks to be down right now, but im sure it will return soon!] v0.5 should be released soon.
best
-a
March 28th, 2008 at 10:38 am
Thanks andrew, very interesting. I agree with you, a framework like that is probably also suitable.
April 9th, 2008 at 4:16 pm
hi, is ther a version on mac os x for the server ?
July 23rd, 2008 at 2:35 am
hi Marijn.
tried your files and got a “Bad record received : 23
Any idea? maybe something quite obvious to but C++ is sort of like chinese to me…
ty and keep up the good work
j
August 11th, 2008 at 1:54 pm
Hi Marijn, I’ve taken this a bit further, check it out:
http://www.itu.dk/~friism/blog/?p=70
October 27th, 2008 at 2:41 pm
That looks very cool!
I wasn’t able to get it working here – when I ran the server, and then ran the .SWF, I got this error in the Server console…
“Bad record received ( should be 76800 ): 17520″
How do you start communication with the server? I can try to connect up with it using Director.
November 16th, 2008 at 6:42 pm
Thanks to the fact that Marjin was nice and has provided his code; I think I can shed some light on the “Bad record recieved” issue. I’m pretty sure that everyone that try’s this demo will get that error. (I got it as well)
The problem lies with the microphone and Camera authorization window that flash will pop up when you first run the swf. The swf can make a socket connection to the c++ application right away, but can only start sending images once you authorize use of the microphone and webcam.
Looking in the c++ code, the socket server ensures that the data its recieving is always a size of 76800 bytes. Due to the fact that the connection is made right away, but no image data is sent from the swf until authorization is granted, the data will always initially be less then 76800 bytes
December 16th, 2008 at 11:12 am
Can any body tell us what to do with the Console.as ??
I have set the ASFace.as as the Document class of my Flash project
but dont know what to do with the Console.as
January 29th, 2009 at 12:16 am
You guys should check out the open source library “Marilena”, which allows you to do the face detection directly in Flash… Might not be as smooth as a C++ application, but it works… Check out a very basic example here: http://www.artofrawr.com/2009/01/28/face-tracking-in-flash/
May 28th, 2010 at 10:34 am
Great informative post!
June 7th, 2010 at 12:35 am
I know this was some time ago now, but cool work, i’m looking at crossing it into unity3d until they finally get the webcam functionality sorted, I originally tried sending images through javascript base64 but it was to slow and limited, so i’m now seeing if this is a fast and viable route and working on the integration, but if this is a sucess will keep you posted and share the code. Thanks for the head start!
September 4th, 2010 at 9:46 pm
7zuKt8 vvixyvxrwoee, [url=http://yrxvveppsffq.com/]yrxvveppsffq[/url], [link=http://gowmzhgmvllo.com/]gowmzhgmvllo[/link], http://ywiamqnpyzgz.com/
December 6th, 2010 at 4:37 am
J9QXAc lzeexdzrdysm, [url=http://aadpqniwlvbv.com/]aadpqniwlvbv[/url], [link=http://bfqtynmkphnp.com/]bfqtynmkphnp[/link], http://zhogwjcmxfmx.com/
December 6th, 2010 at 4:38 am
emailemailJ9QXAc lzeexdzrdysm, [url=http://aadpqniwlvbv.com/]aadpqniwlvbv[/url], [link=http://bfqtynmkphnp.com/]bfqtynmkphnp[/link], http://zhogwjcmxfmx.com/