Microsoft's new Kinect motion-sensing system for the Xbox One game console is so good that it's creepy. Using its sensors, the Kinect can detect your heartbeat, simply by perceiving the pulsing of your skin.
That's one of the cool - and scary - technologies that Microsoft's new Xbox One, announced yesterday at a press event I attended in Redmond, Wash., can do. It is one of things that could move Kinect from a novelty to a necessary part of next-generation gaming and entertainment. It's no longer a science project. Welcome to the age when your game console - and Microsoft - knows if you've been bad or good. It knows more about you than you know about you.
Games can use it to process physical feedback from you, increasing or decreasing the difficulty as needed. And you can use Kinect's gesture and voice commands to control TV functions and navigate menus.
The new Kinect is a superbly upgraded version compared to the original Kinect launched in 2010. That system couldn't detect a lot of things like your finger movements, and it was limited to viewing movements about 10 feet away or so.The new Kinect is accurate down to millimeter measurements, said Scott Evans (pictured at top), group product manager for Kinect.
The new Kinect has a 60 percent wider field of view so it can see most of a room. Kinect can now detect six skeletons, compared to two before. That means as many as six players can participate in a game at the same time. Phil Harrison, corporate vice president at Microsoft, said in an interview with GamesBeat that it is up to developers as to what they do with such technology.
The new system also has a color high-definition camera that can capture your image in a recording for Skype calls. The ultra-wide angle allowed it to pick up a bunch of people sitting in a faux living room that I visited in a lab on Microsoft's campus on Tuesday.
"Xbox One demonstrations were impressive, particularly the improvements to Kinect for greater detail and higher sensitivity that have potential to add to the hardcore game experience," said Billy Pidgeon, a longtime industry analyst.
And Kinect can see you in the dark because it has an infrared sensor that detects things alongside the primary camera. While Kinect previously relied on the system's main processing chips, the new Kinect has its own dedicated silicon. Those custom chips (located in the Xbox One) handle audio processing for better voice recognition and image processing for visuals.
In the past, Kinect used "structured light," or dots of light, to detect motion in a room. The new technology uses "time of flight," or sending out light and detecting how long it takes for it to bounce back to the source.
Now Kinect has enough fidelity to correctly model your skeleton, capturing details such as whether your limbs are rotated. It can see how many fingers you are holding up and whether you are smiling or not. I stood in front of it and could clearly see from the image map that Kinect could see that my thumb was pointing up. You can see the shrug of the shoulders and other subtle movements that can be interpreted into responses to game situations.
The physics of your body is also recorded. Kinect sees which muscles you are using and how much pressure you are putting on them. If you jump, for instance, the muscle map of your body shows the parts that are stressed - your legs - in bright red, while the parts that aren't show up in green. If you move your hand forward, Kinect can detect how much force you are using. If you punch forward at the camera, a big white circle shows up on the image map, indicating you are using a lot of force.
All of this data could overwhelm a lesser system. But Kinect is designed to capture several times more data in an area that is several times larger than before. It then sends that data to the Xbox One for game developers to interpret and use in their games.
The audio system still has four microphones with echo cancellation. The technology is directional. That means that it knows who is speaking in a room and it filters out other noises so that it can focus on what the speaker is saying. That makes it much better and faster at recognizing speech commands. If you are not looking at the screen, it will not register a command from you.
One of the coolest things is that it knows if you give the controller to somebody sitting next to you. Kinect will then record that person as the player, making a switch that would otherwise befuddle any other game system.
Six of us got in front of the camera and we were all moving about. The Kinect muscle maps showed accurately what every one of us was doing, with no delays in putting the imagery on the screen.
The heartbeat demo was the coolest feature, but it was just one measurement that Kinect could do that showed it could really understand what I, the gamer, wanted to do. Another demo showed how the force-feedback technology on the game controllers that you hold in your hand could also deliver the feeling of a heart beating. Between Kinect and the force-feedback controller, the possibilities for pulse-pounding, intense games seem very exciting.
