More Kinect: skeletons and thoughts

This is fun! Today I created a C# application to view skeletons on my PC monitor. Below is the result (available in HD res):

I’m just doing some silly moves in front of the camera like lifting my arms etcetera, so nothing spectacular there; it’s the skeleton being displayed on the monitor what’s spectacular, and fun!

Every time I did some tests, the kids came running from the livingroom and started jumping, waving their arms and doing all kinds of crazy stuff 🙂

At the end of the video, where I walk back towards my keyboard and the distance to the Kinect reduces to 50 cm or so, you can see that the Kinect is getting a hard time tracking all the joints; it becomes a big mess. But that’s okay.

After tinkering with the Kinect and the Kinect SDK for 8 hours or so, my first impression is better than I expected, acually. The way speech is being recognized, how well a skeleton is being tracked is better then I had expected. Ready for use! Is it, or…?

My goal is to use the Kinect for speech and gesture recognition; for that I want to develop a small, lean and mean app that does just that and that can interface with my Domotica system in some way. Either by adding a TCP server to the app, or by uploading information with XMLRPC, or whatever – haven’t really thought about that yet. But I also need an application to ‘record’ gestures and an easy way to define new speech commands this app should recognize; and not just “light on”, but also commands with variables in them, like “we will be back at 16:00”. My system can handle this kind of ‘dynamic events’, so the Kinect should too, right?

But some other things give me second thoughts about the usability of the Kinect.

First, while I was capturing this video, the CPU usage on my Intel E8200 Core 2 Duo (2.66 GHz) Windows 7 Ultimate PC with 8GB RAM was around 50-60%; that’s rather high. Well, I guess most of the CPU cycles were consumed by displaying the skeleton on the screen (which is something I won’t need in the end), but I really need to know what the remainder of the current 50-70% will be when I strip this app to what it is supposed to do: simply recognizing gestures and speech without UI. Cause I’m not that fond of spending that much processing power (and the power consumption that comes with it) on this.

Second, the SDK is limited to Windows 7 (and above, I guess). That’s too bad; now that I think of it; wouldn’t I be better off with something more versatile, platform independent? Wouldn’t it be ideal to connect a Kinect to some local, low-power processor unit (Raspberry Pi, xyz Plug, etc) and let that device take care of the processing and only communicate “the stuff that matters” to the central system? Hmm, maybe I should have a look at some alternatives as well.

Third, the Kinect uses USB to connect to the PC. What if I want to use more than 1 Kinect, say 2 or 3, throughout the house? How do I fix that, USB and max 5 meters of cable length?

So although I’m impressed by the Kinect, there’s still a huge pile of things to do, discover, explore and.. decide!

Tagged , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *