Facial Expression Analysis with Kinect: Thesis Update #1

UPDATE: some things in this project have changed since writing this post, please see the series of posts I wrote on this below:

  1. Some Faces : Thesis Update #2
  2. Animation Units for Facial Expression Tracking : Thesis Update #3
  3. Kinect Face Tracking — Results : Thesis Update #4

It’s my final semester of undergraduate work in computer science, and as such I will be completing an undergraduate honors thesis on a topic of my choosing. I plan to post updates here on my progress and things I am working on, both for my own benefit and tracking and also in the hopes that someone else may find my work interesting. The project that I am working on is an extension of a project started by a former graduate student at my university. Using a Kinect sensor and a Bioloid humanoid robot he was able to get the Kinect to track movements of a person standing in front of the robot and then send that information to the robot and have it imitate these movements. If you are interested what this looks like you can find a video here.

What I plan to do with my project is use the Kinect Face Tracking API to analyze facial expressions using the Kinect sensor. The face tracking API tracks a number of points on the face as shown here: (image from msdn)

We can track the angles and positions of each of these points on the face and use them to determine whether the subject has a “happy” face or a “sad” face, among other different expressions. What I plan to do afterwards is to send this data to the Bioloid robot. The robot will then be able to react in a manner appropriate to the expression he sees. For example, if the robot detects a happy face, he may make a clapping motion, while if he sees a sad face, he will react in a different manner. Applications of emotion recognition are springing up across the board. With this project I hope to both learn some new things and have a fun, interactive project at the end.


4 comments on “Facial Expression Analysis with Kinect: Thesis Update #1

  1. […] up times and the time I spend working on my thesis (if you’re curious about that, see here.) I know for me, implementing self tracking into my life has really opened my eyes to a lot of […]

  2. […] been doing some more work on my emotion recognition project and I have a few preliminary pictures to show. The current status of the project is that I can […]

  3. […] ongoing project concerning Facial Expression Analysis with Kinect is making progress, and as the semester winds down, I am trying to prepare a final product to […]

  4. […] Facial Expression Analysis With Microsoft Kinect : Thesis Update #1 […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s