When you think of hacking, school assignments probably aren't the first thing that come to mind. But then again, you're probably not in Golan Levin's Interactive Art & Computational Design course at Carnegie Mellon University.
Earlier in the week, we covered a couple projects from the class involving hacking Microsoft's Kinect controller for Xbox 360. While Kinect hacking has been a popular topic in hacking and DIY circles (and here at GeekTech), it isn't something one normally thinks of as a topic of study, but Professor Levin worked the Kinect into his curriculum as a way for students to get creative--and to learn something in the process.
In late January, Levin assigned his students to "[create] a system that responds to signals or information interactively, in real-time." That is, the students were to write a program that could analyze the content of a video feed, and insert interactive elements depending on the video's content.
This is basically the concept behind augmented reality (AR), but not all AR software goes quite this far (many AR smartphone apps, for example, just overlay location information over video captured by the phone's camera): Levin's students would have to write programs that could recognize human forms or objects, in addition to simply overlaying items over video.
Levin had his class break up into groups, and he had a dozen Kinects for his students to use. The students then had roughly two and a half weeks to complete their projects.
Why the Kinect
Levin cites Kinect hacks--many of which we've covered in this very blog--as the inspiration for this assignment.
"The Kinect sensor is an ideal teaching tool. It makes altogether new forms of interactions possible, it is inexpensive and it is readily available," Levin told me in an e-mail. "I was inspired by all of the numerous artistic projects and research investigations produced by individual hackers and coders all over the world--projects which, in my opinion, are more interesting than any of the commercially available software 'intended' for the Kinect. "
His students used various open-source programming toolkits to complete their projects, including the Processing programming language (which has been used to program various other Kinect hacks), as well as OpenFrameworks and Cinder. In the case of the project we showed you yesterday, some students also used other related tools, such as the ofxKinect add-on for OpenFrameworks.
Above: "Comic Kinect" by Maya Irvine, Ward Penney, Emily Schwartzman, and Mark Shuster
Tools Aren't Quite There...Yet
But while these programming tools can be used to hack a Kinect, as Levin notes, they're less than ideal: "The main problem has been the unavailability of proper drivers and APIs for the Kinect. Microsoft has promised a Kinect SDK for release at an unspecified time this spring, but they're late to their own party. As of this moment, most of the Kinect libraries that we had to use were essentially hacked together by individual artists and hackers in the FLOSS community."
We here at GeekTech have been impressed by what hackers have been able to do with the Kinect with kludged-together, unofficial programming tools, and it might be just a taste of what's possible. As Levin put it, "The Kinect is a game-changer for all applications that use computer vision. So it (and other depth cameras) will have a massive impact in that area."
In this case, Levin says that the "class's objective was not (necessarily) to be practical, but rather, to expand the vocabulary of human interactions," but we've seen a number of other research projects that could one day prove practical, such as the Minority Report-style interface from MIT researchers, and the Magic Mirror hack, which could help medical students visualize the parts of a human body.
Although the Kinect itself is just a gaming peripheral, the underlying technology is capable of much more, and we can't wait to see what lies ahead.
Update March 5, 2011; 10:44PM PST: The story as originally published stated that Microsoft released the non-commercial SDK in late February, but that it came too late for Professor Levin's class. This is not the case: Microsoft announced in late February that the non-commercial SDK will be available sometime this Spring. We apologize for the confusion, and regret the error.
Like this? You might also enjoy...
- Kinect Hack Class Project Makes You See Ghosts
- Comic Kinect Turns Human Interaction Into a Comic Book Fight Scene
- Kinect Powered Robot Could Help Earthquake Victims