Spectator View provides a looking glass for all to see HoloLens holograms

By Kevin Parrish Published February 14, 2017

microsoft hololens spectator view dslr — Image used with permission by copyright holder

As we saw 30 minutes into Microsoft’s recent Windows 10 Creators Update press event, the company can display what a HoloLens user sees in real time to a general audience. For instance, Microsoft “experience designer” Taj Reid pulled a holographic chair out of a floating web browser and positioned it on the stage. While he experienced all of this in first-person, the audience could see the same objects in a third-person view though the event’s large displays or via the online broadcast. Now Microsoft wants to bring this “spectator view” to all HoloLens developers.

During October’s press event, Microsoft brought out a special camera that captured Taj Reid’s demo in real time while simultaneously rendering the holograms into the live feed. Microsoft wants developers to create a similar camera by offering documentation based on its camera design, aka the Spectator View Camera. The use cases for this setup include snapping images of HoloLens wearers as they interact with holograms, live demonstrations, and video capture.

Recommended Videos

“Since a HoloLens can only be worn by one person, how do you open the view so everyone can see what that person sees? To solve this problem, we developed Mixed Reality Capture (MRC) to aid in the visualization of holograms that the wearer is seeing from a first-person perspective,” said Microsoft’s Brandon Bray in a blog on Monday.

For photo and video capture, Microsoft tested its Mixed Reality Capture technology on the Canon EOS 5D Mark III camera ($2,500) while it tested the Blackmagic Design Production Camera 4K for live demonstrations ($3,000). However, any camera with HDMI output should work, as should any add-on camera lens. Microsoft uses the Canon EF 14mm f/2.8L II USM Ultra-Wide Angle Fixed Lens ($2,100).

The camera setup consists of a tripod, a DSLR camera, a hot shoe fastener to mount a hot shoe adapter, a 3D printed adapter to connect the HoloLens mount to the camera’s hot shoe, and a machined aluminum bracket holding a HoloLens unit. The whole setup can connect to a capture card installed in a PC to preview the composite scene. Microsoft has already tested the setup using the Blackmagic Design Intensity Pro 4K capture card ($190).

On the software side, here is what developers will need:

Spectator View software (GitHub)
Blackmagic Capture Card SDK
Blackmagic Desktop Video Runtime
OpenCV 3.1 to capture without the Blackmagic card
Optional Canon SDK for Canon cameras
Unity for app development
Visual Studio 2015

Instructions on how to build the camera can be found here. Microsoft also provides an online Holograms 240 class showing developers how to create a shared experience through HoloLens and Windows 10 apps via Microsoft’s Universal Windows Platform. Of course, shared experiences require at least two HoloLens headsets, which cost $3,000 each.

The ultimate purpose of Spectator View is to provide a stable, third-person composite view of the HoloLens user and the holograms. Spectators can follow along with the experience without suffering through a nausea-causing “shaky cam” viewpoint created by head-mounted cameras.

Topics

Former Digital Trends Contributor

Kevin started taking PCs apart in the 90s when Quake was on the way and his PC lacked the required components. Since then…

Computing

I tried the RayNeo Air 2s glasses and they’re on sale for Black Friday

RayNeo Air 2s on custom Steam Deck - Briley Kenney Digital Trends_edited

With the holidays coming, I've been trying a spat of unique VR and AR devices. One pair I got my hands on, called the RayNeo Air 2s, basically gives you a portable 201-inch display that you can put on and use anytime, anywhere. They work with Android, Mac, Nintendo Switch, PS5, and -- my favorite -- Steam Deck. Our team has used the RayNeo Air 2 previously and also gave them high marks. Fun Fact I learned from reading that, RayNeo is actually a TCL brand. As for what I think of them, we'll get to that. For now, I want to talk about the crazy RayNeo Black Friday deals that have just dropped.

RayNeo Air 2 -- $184, was $380 51% off

Computing

At basically $105, the Ryzen 5 7600X is the best gaming CPU to buy right now

The Ryzen 5 7600X sitting among thermal paste and RAM.

I don't usually get my hopes up for Black Friday CPU deals, but I found one that's just too good to pass up. Right now, you can get the Ryzen 5 7600X -- still one of the best processors for value-focused gaming -- for basically $105. No, that's not the actual price listed on Newegg where you'll find the deal, but there's a lot going on with this sale.

For starters, the CPU itself is marked down by 24%, bringing the $299 list price down to $225. Not a great deal for a last-gen chip. However, you can save an additional $30 by using the promo code BFEDY2A33, and more importantly, you'll get a free Kingston NV3 1TB hard drive with the order. That's a PCIe 4.0 SSD that normally costs $90.

Computing

This Asus laptop with Copilot+ is $350 off at Best Buy

Asus Vivobook S 15 CoPilot+ front view showing display and keyboard.

You can do quite a bit of gaming on the go these days, thanks to all the handheld consoles and gaming laptops that are on the market. Regarding the latter, we’re always on the lookout for top discounts on the gaming gear we all want to own, which leads us to this wonderful discovery:

For a limited time, when you purchase the Asus Vivobook S 15 with Copilot+ at Best Buy, you’ll pay $550. At full price, this model sells for $900. We tested this PC earlier this year, and our reviewer said the following: “The Asus Vivobook S15 is the best large-display Copilot+ laptop so far in an old-school form factor.”