Discussing the nuts and bolts of software development

Monday, April 02, 2012


Real-time face detection with Android

In this post I will show you how to use the still image FaceDetector class from the Android SDK to achieve (mostly) real-time face detection.  The following method was used in the creation of the MonkeyCam application.

Since its first release, the Android SDK has made available the FaceDetector class, which “identifies the faces of people in a Bitmap graphic object”.  By using the FaceDetector on preview frames from the camera, we approximate the real-time face detection.  For example in the MonkeyCam application we achieved a rate of slightly over one second per frame which is in the same range as other solutions found around the Internet using more complex solutions like third party computer vision libraries.  Short of using the NDK to do GPU programming for the face detection algorithm this was a very usable compromise.

The Camera.PreviewCallback interface provides the onPreviewFrame callback to deliver copies of the preview frames as they are displayed.  The default image format is NV21 and I strongly recommend to stick with it as many cameras don’t support other formats and silently accept a different format from setPreviewFormat() only to return blank frames.  

Now that we have the preview frame in the NV21 format we need to convert it to a bitmap format so we can pass it to the FaceDetector.  Fortunately, since Android 2.2 we can use the YuvImage class to wrap our NV21 frame and then convert it to RGB_565 using its compressToJpeg method and decode the resulting stream into a Bitmap object using BitmapFactory.decodeStream().  For previous versions of the Android SDK you will have to do the conversion manually.

At this point all that is left to do is passing the newly created Bitmap frame to FaceDetector.findFaces and remember to re-add the callback buffer to the Camera object so onPreviewFrame gets called again.  At this point we now have a real-time face detection mechanism, it’s now time to let the creativity flow and add your own touch to make it into something special.

Edit [2012-12-21]: The source code for MonkeyCam as been released under Apache License 2.0 and can be found on github.

Labels: , , , , , , ,

Nice but slow. Because of few redundant pixel conversions and copying.
FaceDetector internally converts Bitmap to grayscale, camera offers YUV, which already contains grayscale buffer, but FaceDetector doesn't accept anything else but RGB in Bitmap.

And you even mention conversion to/from JPG, which is overkill for realtime detection.

Shame, but FaceDetector is not good choice for this task to be fast.

Fast would be directly passing chroma values from YUV to detector lib.
Its Nice but we want portrait mode using front camera detect the face. Please give suggestions.
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?