We picked up one of the Microsoft Mixed Reality headsets more than a year ago. We have played a lot of games, and I have hooked up up to Unity to test out some of the development tools for VR, but It always felt a little wonky. There are a lot of claims that the inside out tracking on the headset might not be up to snuff, but I can safely say that the headset was never the problem. The headset has two wide angle cameras that look out at the world in front of them. By testing for changes in the position of static objects around it and comparing that information to internal gyroscope and accelerometer data, the headset can figure out where it is, if it moved, and where it moved too. It does this at a rate that is much faster than you could physically move your head, and because of this, the virtual world displayed in the headset stays pretty rock solid. If you think that can’t possibly be accurate enough you’re going to be blown away by how optical mice have been working for the last 30 years.
Having tested the other two major VR headsets several times, I can safely say that, for regular use, the inside out tracking solution works just as well as the other positioning systems.
My problem wasn’t with the tracking, it was with the controllers. The same system that tracks where the headset is in space, keeps track of the two controllers, but rather than being wired to the computer they connect over bluetooth. They aren’t the most comfortable of all of the current batch of VR controllers, but they aren’t the worst either. For several months though, they were super flakey.
Early on, the controllers would regularly freak out and stop registering, often when you were moving them quickly or turning around. Most games became a test of patience as you would watch one controller that had been behaving well start to twitch and move to positions that it would be impossible to get your actual hands into. In the Unity editor where things don’t always react as quickly as you would like anyway, the controllers could become downright useless. I would wait with my hand hovering over an interaction point that I had set up until the controller decided to show up and snap to that same spot.
It’s worth noting, that while this was happening the headset continued to work properly. It even kept up when I would do things in the Unity editor with lights and physics that would slow down my framerate and cause pauses in the simulation. Through all that the headset continued to track properly and even respond to quick movements.
One day I was connecting and disconnecting USB cables, as you do, and I thought to swap the XBOX controller receiver with the bluetooth adapter. The XBOX receiver had been finicky from day one and it only seemed to work properly when I plugged it into a USB extender so that it could get some distance from the PC case. I supposed that the case, or something in the case, had caused some interference with the wireless signal between the receiver and the controller so giving it that foot or so of separation from the PC had solved the issue. Instantly the problems I had with XBOX controllers went away and I could reliably use the controllers up to a room away even with a wall in between.
I’m not sure why it hadn’t occurred to me for months, but it seems the same issue was plaguing the bluetooth adapter. I put the adapter on the short USB cable so that it sat a small gap from the PC case and that changed everything.
Now the controllers follow your hands solidly and consistently. They react accurately to fast movements, like throwing or catching or punching. When the controllers move outside the view of your headset, like when you are reaching behind you or far out to the side, the accelerometers and gyroscopes take over and usually do a pretty good job. As soon as the cameras pick up the controllers, if they had strayed at all they instantly snap to their real locations.
We have tested (played) a lot of our favorite VR experiences again and they have a whole new life. When you can count on the controls, games are so much more enjoyable and engaging. I suppose that isn’t really a revelation. I have yet to test out the more demanding interactions in the Unity editor again, but I’m pretty confident they would work much better now.
I suppose the moral of this story is, when you are trouble shooting some problem, it’s often easiest to blame the newest factor, or the most exotic component of a system. If there is a part that you don’t really understand, like the camera based inside out tracking of this VR headset, it becomes very easy to lay the blame there and move on. In this case the problem was simple radio interference, something that should be familiar to anyone living on earth since the early 20th century, and usually solved by moving one of the antennas. Bad TV signal? Put the antenna on your roof. Lost GPS connection? Get out of the parking garage that’s between you and the sky. VR controllers aren’t working right? Move the bluetooth antenna a foot to the right.
Cool. I’m gonna have to set VRTK back up and test it out in Unity again.
Having tested the other two major VR headsets several times, I can safely say that, for regular use, the inside out tracking solution works just as well as the other positioning systems.
My problem wasn’t with the tracking, it was with the controllers. The same system that tracks where the headset is in space, keeps track of the two controllers, but rather than being wired to the computer they connect over bluetooth. They aren’t the most comfortable of all of the current batch of VR controllers, but they aren’t the worst either. For several months though, they were super flakey.
Early on, the controllers would regularly freak out and stop registering, often when you were moving them quickly or turning around. Most games became a test of patience as you would watch one controller that had been behaving well start to twitch and move to positions that it would be impossible to get your actual hands into. In the Unity editor where things don’t always react as quickly as you would like anyway, the controllers could become downright useless. I would wait with my hand hovering over an interaction point that I had set up until the controller decided to show up and snap to that same spot.
It’s worth noting, that while this was happening the headset continued to work properly. It even kept up when I would do things in the Unity editor with lights and physics that would slow down my framerate and cause pauses in the simulation. Through all that the headset continued to track properly and even respond to quick movements.
One day I was connecting and disconnecting USB cables, as you do, and I thought to swap the XBOX controller receiver with the bluetooth adapter. The XBOX receiver had been finicky from day one and it only seemed to work properly when I plugged it into a USB extender so that it could get some distance from the PC case. I supposed that the case, or something in the case, had caused some interference with the wireless signal between the receiver and the controller so giving it that foot or so of separation from the PC had solved the issue. Instantly the problems I had with XBOX controllers went away and I could reliably use the controllers up to a room away even with a wall in between.
I’m not sure why it hadn’t occurred to me for months, but it seems the same issue was plaguing the bluetooth adapter. I put the adapter on the short USB cable so that it sat a small gap from the PC case and that changed everything.
Now the controllers follow your hands solidly and consistently. They react accurately to fast movements, like throwing or catching or punching. When the controllers move outside the view of your headset, like when you are reaching behind you or far out to the side, the accelerometers and gyroscopes take over and usually do a pretty good job. As soon as the cameras pick up the controllers, if they had strayed at all they instantly snap to their real locations.
We have tested (played) a lot of our favorite VR experiences again and they have a whole new life. When you can count on the controls, games are so much more enjoyable and engaging. I suppose that isn’t really a revelation. I have yet to test out the more demanding interactions in the Unity editor again, but I’m pretty confident they would work much better now.
I suppose the moral of this story is, when you are trouble shooting some problem, it’s often easiest to blame the newest factor, or the most exotic component of a system. If there is a part that you don’t really understand, like the camera based inside out tracking of this VR headset, it becomes very easy to lay the blame there and move on. In this case the problem was simple radio interference, something that should be familiar to anyone living on earth since the early 20th century, and usually solved by moving one of the antennas. Bad TV signal? Put the antenna on your roof. Lost GPS connection? Get out of the parking garage that’s between you and the sky. VR controllers aren’t working right? Move the bluetooth antenna a foot to the right.
Cool. I’m gonna have to set VRTK back up and test it out in Unity again.