Skip to main content

HTC Vive Pro cameras can be used to track hands, controllers

htc-vive-pro
Image used with permission by copyright holder
Beyond the uprated screen resolution over its predecessor, the HTC Vive Pro’s most obvious change is in the form of a secondary external camera on the front. Although speculation suggested that the stereoscopic array could enable augmented reality, it turns out that the cameras are used for depth sensing, which can improve the safety-orientated chaperone feature and enable basic controller and hand tracking.

One of the major difficulties faced by virtual reality companies in headset and controller development is occlusion. If something gets in the way of the sensors, suddenly the system powering the headset doesn’t know where it is and that could get uncomfortable for the wearer. The HTC Vive’s Valve-developed Lighthouse trackers had less trouble with that than the Oculus Rift’s cameras, but it still wasn’t perfect.

Recommended Videos

HTC/Valve’s Lighthouse 2.0 tracking solution, due to release later in 2018, will improve this with support for four tracking stations, but the Vive Pro’s cameras could provide a secondary layer of anti-occlusion technology too. In an interview with HTC Vice President of VR technology Raymond Pao, Engadget learned that the stereoscopic cameras can track hands independently of controllers or Vive trackers.

Please enable Javascript to view this content

That wasn’t its original intention, however. HTC’s initial plan was to use them to augment its chaperone feature which provides a digital wall to prevent VR users from hurting themselves and damaging hardware. The cameras would be able to provide a more nuanced chaperone tracking system, thereby showing objects which could be knocked over, or potentially pets or children getting in the way.

That is because the cameras are only capable of “low VGA” resolution (640 x 480), so wouldn’t be fun to look through for augmented reality purposes. However, because of their depth-sensing capabilities, early developers playing around with the Vive Pro found that they could provide basic hand-tracking. Although simple, it was enough to provide precise tracking for fingers, making it possible to play VR games without motion controllers.

That form of “inside-out” tracking does have its limitations — putting your hands outside of the view of the camera would break tracking — but it shows interesting potential. Even then it could be used as a backup tracking solution for controllers should the wearer’s body or another object block the line of sight to the mounted Lighthouse sensors.

Jon Martindale
Jon Martindale is a freelance evergreen writer and occasional section coordinator, covering how to guides, best-of lists, and…
Vision Pro could take ‘four generations’ to reach ideal form
The front visor of the Vision Pro on display at an Apple Store.

Apple employees working closely with the new Vision Pro mixed-reality headset believe it could take “four generations before the device reaches its ideal form,” according to prominent Apple tipster Mark Gurman.

In his weekly Power On newsletter for Bloomberg, Gurman cited his source as “some people in [Apple’s] Vision Products Group" who work directly on the headset.

Read more
You really do not want to forget your Vision Pro passcode
An Apple employee shows a person how to use a Vision Pro headset at an Apple Store.

Owners of Apple’s new Vision Pro headset are advised to make a strong mental note of the passcode they create for their new device.

Why? Because if they forget it, it appears that the only way to start using it again is by taking the Vision Pro to an Apple retail store to get it reset. Or by shipping it to AppleCare customer support.

Read more
Vision Pro durability test turns up a few surprises
Apple's Vision Pro headset.

Apple Vision Pro DURABILITY Test! Worth $3500?

A durability test performed on Apple’s new Vision Pro headset revealed many things we weren't expecting.

Read more