Sunday, April 9, 2017

The Unique Requirements of Public VR

A good treadmill for home use costs around $1000. A treadmill for use in a health club could be ten times more expensive. Why would club owners agree to pay so much more? Because they understand that the home equipment would not withstand the heavy use in a club.

The same is true for VR goggles. Goggles for home use are not suitable for sustained use in arcades and amusement parks. Both VR vendors and attraction operators need to understand and address these differences.

Durability is one key issue. A goggle in an amusement park is subject to both accidental and intentional abuse. A kid might try to see if a lens can pop out. Someone else might wish to take a component as a souvenir.

Hygiene is also important. VR experiences can be intense, and parks can be in hot and humid areas. Whether it is sweat, suntan lotion or something else, a guest does not wish to wear a soaked or dirty goggle. This is true not only for the face mask, but for any built-in headphones.

Sweat and humidity might also cause fogging on the lenses. If a guest has to take off the goggles to defog them, the break in VR immersion degrades the experience.

Depending on the attraction, visitors can be of all ages. Good physical fit is important, regardless if the visitor is a small kit or a large man. Goggles must accommodate a wide range of head sizes, and eye separation. Eyeglasses also present a design challenge. Visitors prefer to keep them on, so goggle vendors try to make room for them.

Beyond the customer-facing experience, there are important operational considerations. Attractions make money by providing a unique experience to a large number of visitors. Every minute spent by a guest adjusting straps in a roller-coaster, is a minute lost. Some have reported a 30% decrease in guest throughput after upgrading a roller coaster to VR. A larger park crew assisting guests or maintaining headsets means larger operational costs.

There are several different types of public VR experiences, each with unique challenge. A free-roam experience (e.g. Zero Latency) needs to address backpack PCs and controllers. Themed experiences such as The Void have accessories that supplant the story. These accessories have many of the same challenges. A small attraction in a shopping mall cannot afford a large operating crew. A VR roller coaster might need a chin strap to keep the goggles on the head. If a VR roller coaster relies on standard phones, they might overheat or need to recharge often.

Often, the first instinct of those building VR attractions is to do everything. They might try to build their own goggles, create content, or even a tracking system. Over time, they focus on their core competencies, bringing external vendors for everything else.

The first generation of these solutions show the immense promise of public VR. VR Coaster, for instance, has deployed GearVR-based roller coaster experiences in over 20 parks. These use a special tracking system to determine the position of each car at any time. The Void and Zero Latency use backpack PCs to allow guests to explore unique spaces. Talon Simulations provides flight and driving simulations in malls. IMAX opened a VR center where guests can try a variety of 10-minute VR experiences. Most of these first-generation solutions use consumer-grade hardware. Operators realize that many problems still need solutions. At the same time, consumers learn what they like and dislike in current solutions.

HMD vendors are also rising to the challenge. ImmersiON-VRelia has developed phone-based goggles that feature a reinforced plastic construction. Sensics released goggles with detachable face mask to address both hygiene and operational efficiency.

What's missing? Low-latency wireless video solutions to get rid of cables. Faster ways to adjust goggles for guest. Better methods to clean goggles between guests. Phones the don't overheat. Multi-user experiences with a stronger social aspect. The passage of time to see what works and what doesn't. VR standards to help integrate new devices into compelling experiences.

I am excited what public VR experiences could be. My excitement comes both from a user standpoint - these experiences are fun! - but also from a problem-solver view - the problems are challenging.

Try these experiences next time you can. Going to a movie theater provides a different experience than watching at home. Going through a good public VR experience is beyond what VR at home provides.

Monday, April 3, 2017

Suffering, Art and VR Standards


Think about a great work of art: a classic book, a timeless painting, a symphonic masterpiece. What's common to many of these creations?

They were all the result of great suffering.

Tolstoy, Van Gogh, Mozart - they did not have easy lives. Many of the greats suffered from oppression, mental or physical illness, or hunger.


If you don't have drama in your life, how could you summon drama for your art?

People ask me "what made you want to work on VR standards?" My answer: it's the suffering.

No, not my personal suffering. I'm no Amaedus or have never considered cutting off my earlobe to express love.

But in many years of working with customers on their VR systems, I saw a lot of technical suffering:

The suffering of integrators that need to chase the latest API again and again. That don't know if the equipment they design for today will be available to buy in a year.

The suffering of device manufacturers that need just one more driver to support them.

The suffering of end-users that wonder if today's software will work on tomorrow's devices.

That's why we need efforts like OSVR or OpenXR to make it easy for everyone to work together. It wouldn't be as timeless or profound as "War and Peace", but it will help a lot of people.

Saturday, April 1, 2017

Unholy Alliance

A few weeks ago, we were approached by a VR porn site seeking a partnership.

It is no secret that adult entertainment is an early adopter of many new technologies and virtual reality is no exception.

We respectfully declined as we decided a long time ago that we won't participate in this market.

Besides, I'm no longer as good looking as I used to be.

Monday, March 27, 2017

Virtual Reality Standards: too early or long overdue?

This article originally appeared on Mar 22nd at ReadWrite
You bought a new printer for the office. You unpack and connect it to your PC. You install its demonstration software and see the printer works well. Then, an unfortunate surprise: your word processor cannot work with this printer. You'll need to wait until the maker of the word processor releases a new version. When will this version be available? Should you return the printer? Should you change to a different word processor?

If this sounds like the 1980's, it's not too far from where VR is today. VR programs are often hard-coded to one set of hardware devices. Only a particular HMD, coupled with its tracking system and controller will work. Every device manufacturer has a different API. If you want to use this VR experience with a different set of hardware, you might not be in luck. At best, you'll need another version. Worst case, you just won't be able to do it. The problem of different vendors having different APIs is often called 'API fragmentation'

VR standards can help solve this fragmentation problem. In the PC world there are a few basic device types: keyboard, mouse, printer, scanner, and so forth. Likewise, basic VR device types include an HMD, tracker, controller, and a few others. A standard way for a program to interface with these VR devices would help solve this problem.

If software could work across different hardware combinations, almost everyone would benefit:
  • Consumers could mix and match devices to their liking. I may have a Dell computer but I don't always want a Dell printer to go with it. Furthermore, consumers would be confident that their investments are future-proof. A 2017 game would likely work with 2018 or 2019 hardware. 
  • Game publishers and other experience creators would have a larger addressable market. Today, they hard-code their game to a particular set of hardware devices. Tomorrow, they would support any device that has a conforming 'driver'. 
  • Manufacturers that support a standard would have their devices work with lots of content. This would allow even small players to enter the market, and promote innovation. 
Some claim it is too early for a VR standard. VR is new, they say. Let a couple of years pass and then we will know what to standardize. Yet while consumer VR is new, VR has been in academia and industry for decades. Labs and factories deployed HMDs, head and eye trackers, and controllers for many years. Such devices were expensive and perhaps more clunky, but still performed the same functions. Software frameworks like UNC's VRPN (http://www.vrpn.org) provided device independent access for many years.

The resistance to a standard sometimes stems from the competitive strategy of a company. A vendor relying on a 'walled garden' approach often wishes to control the entire stack. The ability to swap out hardware, or use a different app store might be not what they had in mind.

In VR, there are often two standard interfaces that need to defined. The first is the device interface. This defines how to configure devices of particular type and how to extract data from them. Printers have different capabilities but share the same basic functions. The same is true for VR devices. The second standard interface is the application interface. It describes how an application or a game engine renders its content and get data. Inbetween the applications and devices there is often a middleware layer. That middleware is the software intermediary between applications and devices.

One effort that adapts this approach is OSVR. Started by Sensics and Razer, it is an open-source software platform for VR. OSVR implements both a device interface layer as well as an application layer. OSVR supports over 200 devices, and most of the OSVR code is free and open-source.

Another effort is OpenVR which is an open API (though not open source) from Valve. Building on the success of SteamVR, OpenVR allows HMDs to work with SteamVR content. There is some compatibility between these efforts. An OSVR plugin for OpenVR, allowing OSVr devices to work with SteamVR content.

In January, the Khronos group (known for OpenGL standards) launched a new VR initiative. The initiative, called OpenXR, brings together a wide range of companies. Industry leaders including Google, Oculus, Valve, Sensics and Samsung are part of this effort. OpenXR aims to combine lessons learned from building OSVR, OpenVR and proprietary APIs. It aims to create both a device interface as well as application interface. It is unclear how soon this effort will mature. Khronos standards take an average of 18 months. It is also unclear what capabilities will be part of the first standard. What is clear is that these companies felt enough pain to want to work on standards.

I am encouraged that so many participants are coming together to work on a standard. Other interested parties are also invited to contribute. Standards are sometimes boring, but they are important. They will make the consumer experience better and promote innovation.
This article originally appeared on Mar 22nd at ReadWrite

Wednesday, October 19, 2016

Peeking inside the Sensics Goggles for Public VR

Earlier this week, we made the Sensics Goggles for Public VR available for purchase on the OSVR Store. This is a limited pre-production run as we gear up for production of larger quantities.
We designed this product to address the needs of those that operate VR in public places such as theme parks, entertainment venues and shopping malls.
Goggles for public VR have different requirements than goggles for home use just like an exercise treadmill at a gym or health club needs to be different than a treadmill at home. Specifically, goggles for public VR need to be:
  • Durable, so that they withstand use by a large number of people. Unlike users of a VR goggle at home, users of a VR goggle at a public place might care less about handling it carefully.
  • Easy to clean, so that every user can get a clean, fresh feeling when wearing the goggles, regardless of who wore it before them.
  • Easy to maintain, in case something breaks.
  • Designed to allow maximum throughput of guests so as to maximize the number of people that can experience the attraction.
At the same time, the visual experience needs to be at least as good as goggles for home use because guests typically expect an experience beyond what they can get at home.
To achieve these goals, we used mass-produced 2160x1200 90 Hz OLED screens, high-quality dual-element optics with individual focusing mechanism, accurate 9-axis orientation tracker and incorporated them into a novel, patent-pending design. Below are some of the highlights of this design. To illustrate them, we mostly use the CAD drawings because they make it easy to show internal parts.
Here is CAD model of the entire unit (each part is colored differently in this model to make them stand out) next to the actual unit:
The back side of the unit has a cable clip to allow easy insertion and removal of cables as required. This ensures that cables don’t get in the way of the user.
The front of the unit includes a window that is transparent to IR. This allows inclusion of a Leap Motion camera inside the unit to facilitate natural interaction with the hands. The fact that the controller is embedded inside the goggle eliminates the need to route cables externally. This approach is superior to external mounting of the controller because when mounted externally, the controller might be easier to detach from the goggles. Note that in the CAD model, the IR window has been removed so that the Leap Motion unit is clearly visible.
To fit a wide range of users, the goggles were designed with adjustable optics. These allow people that normally wear eyeglasses to take them off and still see an excellent picture. Individual knobs — highlighted by the arrows in the CAD drawing from a bottom view perspective
- allow focusing of each eye independently. It is also possible to design optics that have a large enough eye relief to accommodate glasses but we chose adjustments in this particular design.
The face mask — the part that touches the user’s face — is easily removable and replaceable. It is designed with a groove (not shown in the picture) that allows an operator to quickly and accurately replace the mask when needed without requiring any special tools.
VR experiences can be very intense. For instance, guests to SEGA Joypolis run around in a special warehouse and shoot zombies. It is important to keep these guests cool and dry. That’s why we the public VR goggles include dual silent fans that whisk away humidity and heat.
The diagram has arrows pointing to an air vent (one in each side) and the holes through which it exits the goggles. An important feature in the Sensics design is the ability to separate the “passive part” of the goggles (facemask and head strap) from the “active part” (electronics, optics, etc.). This feature provides several important benefits:
  1. It allows guests to don the passive part while waiting in line. They can adjust the fit to their heads, and make sure the strap is comfortable. While doing so, the front part of the passive unit is completely open so guests can still see the real world, take a selfie with the strap. Only when the activity is about to begin does the operator attach the active part to the passive part.
  2. It permits various cleaning strategies for the passive part — the part that touches the head. For instance, an attraction operator can have many more passive parts than active parts and then clean the passive parts in batch at the end of the day.
  3. Separating the face mask from the active part of the goggles allows for multiple sizes of the face mask to fit kids, different facial structures and so forth.
The two parts of the goggles — active and passive — are illustrated in the photos below by Sensics team member Yaron Kaufman.
Detaching the passive part from the active part is done by pressing two button — one in each side of the goggles. The button is shown as yellow highlighted by the arrow in the diagram. The clasp holding the two parts together is made of metal, and thus designed for numerous grab/release cycles.

Two additional parts are highlighted in the diagram: configurable buttons on the top right side of the goggles serve as programmable user-interface controls. This could be to increase/decrease volume, to pause the game, select a menu item or any other function. The mechanical design allows for one, two or three buttons per the preferences of the customer.
Last, an audio output jack appear on the bottom. The goggles can also support a permanent audio solution which attaches where the large yellow ellipse is shown in the diagram to the right. We put a lot of thought into designing this product. We hope you will get a chance to try it and appreciate its suitability to public VR applications.

Tuesday, August 9, 2016

Why did Sensics launch the OSVR Store?



Last week, the OSVR Store came on-line. It offers a range of OSVR-related products, services, accessories and components. It also contains useful information, most of it adopted from this blog.

But why did the Sensics team launch it?

The first answer that comes to mind is “to make money”. That’s an obvious reason, as Sensics is a for-profit company. We invest a lot in developing OSVR and would love to see returns on our investments.

But that’s not the only reason, nor perhaps the most important one. Here are some others.

We wanted the OSVR Store to be helpful to the VR enthusiast and hacker. That’s why we offer components: optics, tracking boards from various vendors, IR camera. More components are coming. Some will use those to upgrade an existing system, others to build a new one.

We wanted a place for hardware developers, a platform to market their innovations. If you make something OSVR-related, we invite you to sell it on the OSVR Store. It can be an OSVR-supported HMD. It can be an accessory or component that can help OSVR users. It can even be OSVR-related services. We strive to offer fair and simple terms. If you can build it, we can help you promote it. Drop us a note at hello@osvrstore.com to get started.

To me, OSVR has always been about choice. About democratizing VR. Not forcing users to buy everything from the same vendor. Encouraging applications to run on many devices. Support more than one operating system.

The OSVR Store is one more way to give everyone choice. Check it out.

Monday, August 1, 2016

OSVR - a Look Ahead

Introduction

OSVR is an open source software platform and VR goggle. Sensics and Razer launched OSVR 18 months ago with the intent of democratizing VR. We wanted to provide an open alternative to walled-garden, single-device approaches.

It turns out that others share this vision. We saw exponential growth in participation in OSVR. Acer, NVIDIA, Valve, Ubisoft, Leap Motion and many others joined the ecosystem. The OSVR goggle – called the Hacker Development Kit – has seen several major hardware improvements. The founding team and many other contributors expanded the functionality of the OSVR software.

I’d like to describe how I hope to see OSVR develop given past and present industry trends.

Increased Device Diversity leads to more Choices for Customers

Trends

An avalanche of new virtual reality devices arrived. We see goggles, motion trackers, haptics, eye trackers, motion chairs and body suits. There is no slowdown in sight: many new devices will launch in the coming months. What is common to all these devices? They need software: game engine plugins, compatible content and software utilities. For device manufacturers, this software is not a core competency but ‘a necessary evil’. Without software, these new devices are almost useless.

At the same time, content providers realize it’s best not to limit the their content to one device. The VR market is too small for that. The more devices you support, the largest your addressable market becomes.

With such rapid innovation, what was the best VR system six months ago is anything but that today. The dream VR system might be a goggle from one vendor, input devices from another and tracking from a third. Wait another six months and you’ll want something else. Does everything need to come from the same vendor? Maybe not. The lessons of home electronics apply to VR: you don’t need a single vendor to make all your devices.

This ‘mix and match’ ability is even more critical for enterprise customers. VR arcades, for instance, might use custom hardware or professional tracking systems. They want a software environment that is flexible and extensible. They want an environment that supports ‘off-the-shelf’ products yet extends for ‘custom’ designs.

OSVR Implications

OSVR already supports hundreds devices. The up-to-date list is here: http://osvr.github.io/compatibility/ . Every month, device vendors, VR enthusiasts and the core OSVR team add new devices. Most OSVR plugins (extension modules) are open-sourced. Thus, it is often possible to use an existing plugin as baseline for a new one. With every new device, we come closer towards achieving universal device support.

A key OSVR goal is to create abstract device interfaces. This allows applications to work without regards to the particular device or technology choice. For example, head tracking can come from optical trackers or inertial ones. The option of a a “mix and match” approach overcomes the risk of a single vendor lock-in. You don’t change your word processor when you buy a new printer. Likewise, you shouldn’t have to change your applications when you get a new VR device.

We try to make it easy to add OSVR support to any device. We worked with several goggle manufactures to create plugins for their products. Others did this work themselves. Once such a plugin is ready, customers instantly gains access to all OSVR content. Many game engines – such as Unity, Unreal and SteamVR- immediately support it.

The same is also true for input and output peripherals such as eye trackers and haptic devices. If developers use an API from one peripheral vendor, they need to learn a new API for each new device. If developers use the OSVR API, they don’t need to bother with vendor-specific interfaces.

I would love to see more enhancements to the abstract OSVR interfaces. They should reflect new capabilities, support new devices and integrate smart plugins.


More People Exposed to more VR Applications in More Places

Trends

Just a few years ago, the biggest VR-centric conference of the year had 500 attendees. Most attendees had advanced computer science degrees. My company was one of about 10 presenting vendors. Today, you can experience a VR demo at a Best Buy. You can use a VR device on a roller coaster. With a $10 investment, you can turn your phone into a simple VR device.

In the past, to set up a VR system you had to be a geek with plenty of time. Now, ordinary people expect to do it with ease.

More than ever, businesses are experimenting with adopting VR. Applications that have always been the subject of dreams of are becoming practical. We see entertainment, therapy, home improvement, tourism, meditation, design and many other applications.

These businesses are discovering that different applications have different hardware and software requirements. A treadmill at home is not going to survive the intensive use at a gym. Likewise, a VR device designed for home use is not suitable for use in a high-traffic shopping mall. The computing and packaging requirements for these applications are different from use to use. Some accept a high-end gaming PC, while others prefer inexpensive Android machines. I expect to see the full gamut of hardware platforms and a wide variety of cost and packaging options.

OSVR Implications

“Any customer can have a car painted any color that he wants so long as it is black”, said Henry Ford. I’d like to see a different approach, one that encourages variety and customization.

On the hardware side, Sensics is designing many products that use OSVR components. For instance, our “Goggles for public VR” use OSVR parts in an amusement park goggle. We also help other companies use OSVR components inside their own packages. For those that want to design their own hardware, the OSVR goggle is a good reference design.

On the software side, I would like to see OSVR expand to support more platforms. I’d like to see better Mac support and more complete coverage of Android and Linux platforms. I’d like to see VR work well on mid-range PCs and not limited to the newest graphics cards. This will lower the barriers to experience good VR and bring more people into the fold. I’d like to see device-specific optimizations to make the most of available capabilities. The OpenCV image processing library has optimizations for many processors. OSVR could follow a similar path.

Additionally, it is important to automate or at least simplify the end-user experience. Make it as close to plug-and-play as possible . The task of identifying available devices and configuring them should be quick and simple.

Simplicity is not limited to configuration. We’d like to see easier ways to choose, buy and deploy software.

Reducing Latency is Becoming Complex

Trends

Presence in VR requires low latency, and reducing latency is not easy. Low latency is also not the result of one single technique. Instead, many methods work together to achieve the desired result. Asynchronous time warp modifies the image just before sending it to the display. Predictive tracking lowers perceived latency by estimating future orientation. Direct mode bypasses the operating system. Foveated rendering reduces render complexity by understanding eye position. Render masking removes pixels from hidden areas in the image.

If this sounds complex, it is just the beginning. One needs to measure optical distortion and correct it in real-time. Frame rates continue to increase, thus lowering the available time to render a frame. Engines can optimize rendering by using similarities between the left- and right-eye images. Techniques that used to be exotic are now becoming mainstream.

A handful of companies have the money and people to master all these techniques. Most other organizations prefer to focus on their core competencies. What should they do?

OSVR implications

A key goal of OSVR is to “make hard things easy without making easy things hard”. The OSVR Render Manager examplifies this. OSVR makes these latency-reduction methods available to everyone. We work with graphics vendors to achieve direct mode through their API. We work with game engines to provide native integration of OSVR into their code.

I expect the OSVR community to continue to keep track of the state of the art, and improve the code-base. Developers using OSVR can focus away from the plumbing of rendering. OSVR will continue to allow developers to focus on great experiences.


The Peripherals are Coming

Trends

A PC is useful with a mouse and keyboard. Likewise, A goggle is useful with a head tracker. A PC is better when adding a printer, a high-quality microphone and a scanner. A goggle is better with an eye tracker, a hand controller and a haptic device. VR peripherals increase immersion and bring more senses into play.

In a PC environment, there are many ways to achieve the same task. You select an option using the mouse, the keyboard, by touching the screen, or even with your voice. In VR, you can do this with a hand gesture, with a head nod or by pressing a button. Applications want to focus on what you want to do rather than how you express your wishes.

More peripherals mean more configurations. If you are in a car racing experience, you’d love to use a rumble chair if you have it. Even though Rumble chairs are not commonplace, there are several types of them. Applications need to be able to sense what peripherals are available and make use of them.

Even a fundamental capability like tracking will have many variants. Maybe you have a wireless goggle that allows you to roam around. Maybe you sit in front of a desk with limited space. Maybe you have room to reach forward with your hands. Maybe you are on a train and can’t do so. Applications can’t assume just one configuration.

OSVR implications

OSVR embeds Virtual Reality Peripheral Network (VRPN), an established open-source library. Supporting many devices and focusing on the what, not the how is in our DNA.

I expect OSVR to continue to improve its support for new devices. We might need to enhance the generic eye tracker interface as eye trackers become more common. We will need to look for common characteristics of haptics devices. We might even be able to standardize how vendors specify optical distortion.

This is a community effort, not handed down from some elder council in an imperial palace. I would love to see working groups formed to address areas of common interest.

Turning Data into Information

Trends

A stream of XYZ hand coordinates is useful. Knowing that this stream represents a ‘figure 8’ is more useful. Smart software can turn data into higher-level information. Augmented reality tools detect objects in video feeds. Eye tracking software converts eye images into gaze direction. Hand tracking software converts hand position into gestures.

Analyzing real-time data gets us closer to understanding emotion and intent. In turn, applications that make use if this information can become more compelling. A game can use gaze direction to improve the quality of interaction with a virtual character. Monitoring body vitals can help achieve the desire level of relaxation or excitement.

As users experience this enhanced interaction, they will demand more of it.

OSVR Implications

Desktop applications don’t have code to detect a mouse double-click. They rely on the operating system to convert mouse data into the double-click event. OSVR needs to provide applications with both low-level data and high-level information.

In “OSVR speak”, an analysis plugin is the software that converts data into information. While early OSVR work focused on lower-level tasks, several analysis plugins are already available. For example, DAQRI integrated a plugin that detects objects in a video stream.

I expect many more plugins will become available. The open OSVR architecture opens plugin development to everyone. If you are an eye tracking expert, you can add an eye tracking plugin. If you have code that detects gestures, it is easy to connect it to OSVR. One might also expect a plugin marketplace, like an asset store, to help find and deploy plugins.

Augmenting Reality

Market trends

Most existing consumer-level devices are virtual reality devices. Google Glass has not been as successful as hoped. Magic Leap is not commercial yet. Microsoft Hololens kits are shipping to developers, but are not priced for consumers yet.

With time, augmented-reality headsets will become consumer products. AR products share many of the needs of their VR cousins. They need abstract interfaces. They need to turn data into information. They need high-performance rendering and flexible sensing.

OSVR Implications

The OSVR architecture supports AR just as it supports VR. Because AR and VR have so much in common, many components are already in place.

AR devices are less likely to tether to a Windows PC. The multi-platform and multi-OS capabilities of OSVR will be an advantage. Wherever possible, I hope to continue and see a consistent cross-platform API for OSVR. This will allow developers to tailor deployment options to the customer needs.

Summary



We designed OSVR to provide universal connectivity between engines and devices. OSVR makes hard things easy so developers can focus on fantastic experiences, not plumbing. It is open so that the rate of innovation is not constrained by a single company. I expect it to be invaluable for many years to come. Please join the OSVR team and myself for this exciting journey.

To learn more about our work in OSVR, please visit this page





This post was written by Yuval Boger, CEO of Sensics and co-founder of OSVR. Yuval and his team designed the OSVR software platform and built key parts of the OSVR offering.