The Xbox One’s UI May have sold me on the Kinect pack-in

The Xbox One’s UI May have sold me on the Kinect pack-in

I’m sitting in a small room with Jeff Henshaw, who is the Engineering Manager on the Xbox One platform, and I’m here to take a look at the stuff that’s supposed to be a broken mess. The user interface, the voice commands, the swapping from app to app… according to Internet rumor none of this stuff works, and everyone is in a panic.

Henshaw laughs when I bring this up, and asks what forums or sites I’ve been reading. He seems laid back, and excited to show off the work.

That’s not to say that the team isn’t working hard to get this out the door. I asked how many people are crunching on getting the software finished. “Everybody,” he said.

“We have several hundred people that are cranking away on both the console client and the Live service as well as all the support for game developers. You will probably hear a few quotes that we have been throwing around the last few days,” he explained.

“All of our launch titles are now in certification. Or out of certification.  We have some that are being pressed onto DVDs now. That was a huge magic moment. We have a bell that we ring whenever a title ships.  And to have the Xbox One bell being rung, as titles come out of certification and are being manufactured on BluRay discs was a huge, huge moment for us.”

The Kinect has a towel over it, and Henshaw removes the blinder and takes a few steps back. The interface changes as it recognizes him, and brings all his content to the home screen. You can see his individual things he has pinned to the home screen, including The Walking Dead so he can always catch the latest episode. There is a frozen instance of Forza 5 that he can jump in and out of instantly.

There is an impressive level of granularity here: If you pin a show like The Walking Dead you can pin the series in general, or a specific season, or a single episode. With music you can pin a station, a playlist, an artist, a specific album, or even just a song.

The system can keep one game “hot” at any time, so you can pause the game and move to a television show or a Skype call or anything else and the game sinks into the background until you call it back up. There is no loading time, no need to save the game. It simply goes away into the background, and you can return via a voice command at any time.

“Kinect actually has an array of microphones in it that can do a tech called Beam Forming.  It’s able, to within about five inch accuracy, which is about the width of a human mouth, pinpoint the source of a spoken command, so when it hears someone say the command that we just said, it knows which skeleton it came from. And it has mapped that skeleton to a biometric profile on the console,” Henshaw said.

So when I say “Xbox, show me my stuff,” it matches the voice, but also my relative size in the room. My son can walk in and give the same command, and it will match the voice and his size and bring up his content. If he does so and tries to play an M-rated game and his account is locked down, he won’t be able to play.

“We actually want to be pretty liberal about letting people play as other people,” Henshaw said. “If I can't get past a level,  I want to be able to hand you the controller and get your help. So we stayed liberal on leaving the account logged in and letting other people use it.  We stayed conservative on if you log out because you don’t want your son… if you don’t want anyone playing as you, your son can’t get around it.”

So if I leave myself logged in and a kid comes in and picks up the controller, the system won’t boot him because it only sees a small skeleton, but if I set it the parental controls and log out it will look for both my voice and a figure of my size before it lets anyone else log in.

“I don't know the exact percentage, but it needs to see a percentage of your skeleton to map it to you to biometrically ID you. We actually do a really smart blending of facial characteristics and overall skeletal structure, and we combine those two into a profile,” Henshaw said. “That profile is what we use to authenticate you.  So you can't just take a glossy 8-1/2 by 11 and hold it up in front of the camera.  We did think about that.  We had college dorm room visions of people trying to hack that. So we were pretty smart about it.”

Seeing all this in action is impressive, and it’s pointed out to me that you don’t have to yell; you just speak conversationally. He brings up the television feed, Skype, Internet Explorer, music, and the system switches between each application nearly instantly, and moves back to the main screen just as quickly. We jump back into the game, and then pause the game to take a Skype call. I use a few voice commands to capture a scene in Forza and we edit it into a little video.

I try to speak quietly to try to slip up the system, but it recognizes my commands. I ask him to go to a few different things in different order just to make sure this isn’t canned. 
Henshaw leans back, and reminds me that I had believed none of this was working when I walked in.

The science behind the magic

“So we just switched between 8, 9 apps.  Forza is still hot. And this is really key,” Henshaw said. “Because on any other platform, and you're going to be talking to all the big boys shipping new consoles this year. On any other platform, if you exit the game to go check the sports score in a game on TV, or to go browse the web for a hint on your console, you just lost your progress unless you saved.  You just lost the state of your game.  Not on Xbox One.” This is their goal, to keep the game always ready to go, no matter what else you do, or what other apps you use. The game is always ready for you to return, even if you’re using the Xbox One for other media.

How the hell does that happen?

“So around the E3 time frame, there was a lot of noise about how Xbox One incorporates like 3 operating systems,” he told the Report. “And technically, that is true.  We have a hypervisor at the very bottom of our stack that was written by Dave Cutler and I don’t know if the name Dave Cutler means anything to you, he’s the guy who wrote VMS and wrote the Windows NT kernel.”

“So the dude… he’s legendary in the industry.  He knows his core operating system shit like no one else.  Dave wrote our hypervisor.  We have two LS virtual machines.  One that handles the dedicated game, and one that handles the round-robinning of all the apps.  Those are completely separated,” Henshaw continued.  “Your game always has a fixed view of the world of hardware resource,  it’s running any hypervised dedicated VM.  And that was why there’s always one that’s hot at a time. The apps run in a much more Windows 8-like round robin VM.  So you'll always have 2, 3,4, 5 of those hot, depending on what they do.  Those are managed separately and they can be reset separately.”

This is important, because you have don’t have to worry about cleaning up after the system. Even your smart phone likely benefits from a hard reset every few days, but the Xbox One has been designed to always stay on, and to run the entirety of your home theater.

“Generally that whole VM reboots whenever you start a new game. It’s cleaned by virtue of watching a new title,” Henshaw said. “And the apps that we are flipping through so quickly and doing all this stuff, they have automatic clean up that happens in that VM and that whole VM can be restarted without ever affecting the game.”

And this is what Microsoft has done such a poor job of communicating: They're spent a significant amount of time and money making sure that the Xbox One can run your home theater, but they also made sure to keep the gaming aspect of the hardware front and center. You can step away from the game at any time, and the system will keep the game frozen, ready to jump back into action when you're ready to play again.

The PlayStation 4 does this on a system level; you'll be able to put the system in standby mode and walk away, but the Xbox One allows you to do this for the game itself and continue using the console for anything else. It's a powerful feature that doesn't really sink in until you see it in person, and goes a long way to justify Microsoft's claims that this should be the middle of your entertainment experience.

I'll be testing all these issues in detail in my own home very soon, but this meeting at least proved that yes, this exists. It works. And it's pretty damned cool.