UPDATE: some things in this project have changed since writing this post, please see the series of posts I wrote on this below:
- Some Faces : Thesis Update #2
- Animation Units for Facial Expression Tracking : Thesis Update #3
- Kinect Face Tracking — Results : Thesis Update #4
It’s my final semester of undergraduate work in computer science, and as such I will be completing an undergraduate honors thesis on a topic of my choosing. I plan to post updates here on my progress and things I am working on, both for my own benefit and tracking and also in the hopes that someone else may find my work interesting. The project that I am working on is an extension of a project started by a former graduate student at my university. Using a Kinect sensor and a Bioloid humanoid robot he was able to get the Kinect to track movements of a person standing in front of the robot and then send that information to the robot and have it imitate these movements. If you are interested what this looks like you can find a video here.
What I plan to do with my project is use the Kinect Face Tracking API to analyze facial expressions using the Kinect sensor. The face tracking API tracks a number of points on the face as shown here: (image from msdn)
We can track the angles and positions of each of these points on the face and use them to determine whether the subject has a “happy” face or a “sad” face, among other different expressions. What I plan to do afterwards is to send this data to the Bioloid robot. The robot will then be able to react in a manner appropriate to the expression he sees. For example, if the robot detects a happy face, he may make a clapping motion, while if he sees a sad face, he will react in a different manner. Applications of emotion recognition are springing up across the board. With this project I hope to both learn some new things and have a fun, interactive project at the end.