Some Face Tracking With Kinect: Thesis Update #2

See the previous post in this series: Facial Expression Analysis With Microsoft Kinect : Thesis Update #1

I’ve been doing some more work on my emotion recognition project and I have a few preliminary pictures to show. The current status of the project is that I can detect faces and see the wire mesh over them, but it’s not exactly as accurate as I would like. For instance, I have noticed that wearing glasses seems to confuse the eye placement. My eyes in the wire mesh are consistently hovering above the top rim of my glasses. If I take them off, this effect is lessened. Another limitation that I noticed is that if someone is wearing a hat it will often not detect the face at all. I brought the equipment to my parent’s house last weekend and let them try it out. My dad always wears a baseball cap, and the Kinect could not recognize his face until he removed it.

Below you will see some facial expression “archetypes” that I have developed using my own face. You will notice that “sad” is not included in the list. Because of the inaccurate eye and eyebrow placement, I cannot get it to show the upturned “sad eyebrows” that I wanted. In addition, as much as I frown, it looks like the wire mesh is making a kissing face.

Definitely some things to work out but check out the faces I’ve worked on so far. I plan to get multiple people to make faces at the Kinect and see if I can get them into groups that the program can further “learn” with.

normal

“Normal face” – a baseline

Angry face - note the eyebrows

Angry face – note the eyebrows

Happy Face - smile!

Happy Face – smile!

Surprised face - heightened eyebrows and open mouth

Surprised face – heightened eyebrows and open mouth

As you can see, eyes are much better placed without glasses...

As you can see, eyes are much better placed without glasses…

UPDATE: See the next posts in this series:

Advertisements

5 comments on “Some Face Tracking With Kinect: Thesis Update #2

  1. John Aceti says:

    That was awesome! Very good read!

  2. […] as the semester winds down, I am trying to prepare a final product to present. As I described in my last thesis update, I had been able to test a sample product which overlays a 3D mesh on the user’s face and […]

  3. Hi,
    Thanks for the sharing. It’s help me in my project.
    I’ve download your code that you have share in https://github.com/nelsonam/thesis-facetracking
    and try to run it using 1.7 SDK and Microsoft Visual Studio 2010 Express on a Win 7 32 machine.
    I managed to get the code run without any error.
    Based on your project, I should be able to get the wire mesh on the face as a result of the face has succeed been track as explained in this page. However, I noticed that my output can detect the face only without the wire mesh on the face.
    Can you give some ideas to me for troubleshooting my problem? 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s