TL;DR: Despite the recent spike in adoption, VR (Virtual Reality) is still a new medium. Many VR apps handle user authentication poorly, leaving users frustrated when trying to sign in to new apps. As part of our research for improving the developer and the user experience for VR authentication, we propose an incremental improvement approach for user authentication experience for VR and open the discussion on what it will take to achieve better IdP-neutral experiences.
VR Is Accelerating; AR Is Next
It’s not very common for sci-fi tropes to actually come true in our lifetimes (“where is my flying car?”), but it looks like it’s happening to Virtual Reality (VR).
Thirty years ago, VR was a thing on the pages of Neal Stephenson or at the movies. The Oculus Quest 2, a standalone VR headset priced at less than half of a modern gaming console, was one of the must-have gadgets of the holiday season 2021 — accelerating a run that has been going strong since its release date. With Meta’s increasing investment in the Metaverse, this adoption is only likely to continue.
The distribution and price point have been an absolute boon for developers, in particular for game developers, who are increasingly targeting VR users with their products. The COVID-19 pandemic resulted in forced isolation, social distancing, and increased remote work, all of which inspired a brand new wave of immersive collaboration and social networking tools (Workrooms and Horizon are just two examples of the many apps in those new categories). Job training, fitness, even therapy are among the other categories emerging as applications that can really benefit from the new medium.
There are signs suggesting that this is only the beginning. Meta seems to be on top of today’s VR market, but it’s clear that they will soon face formidable competitors. Apple, Google, and Snapchat are reported to be working on VR and augmented reality (AR) headsets of their own.
AR has the potential to be an even bigger revolution than smartphones, bringing a semantic layer on top of everyday reality, and today’s development tools for VR can be used to target AR as well. It’s no wonder that new VR applications are proliferating in greater numbers every day.
Authentication in Today’s VR Apps
At first glance, one might think that developing apps for VR is a big technical challenge. As it turns out, developing an app that can run on a VR headset is surprisingly easy.
VR headsets were first popularized and remain primarily used as gaming devices with a blossoming developer community. The gaming industry created extraordinary tools such as the Unity game engine and development platform. Unity began in 2005 and is now a comprehensive ecosystem with creator tools, a vibrant community, and abundant content that makes it easy (and free for indie developers) to create 3D games targeting PC, consoles, iOS, and Android.
VR headset manufacturers such as Meta and Google (Google pioneered the cardboard headset class) leveraged Unity’s popularity, releasing SDKs that make it very easy to retarget existing and new apps to their hardware and expose the special capabilities of the VR environment through primitives compatible with the Unity API.
Having the tools to easily make an existing app run in VR, however, doesn’t necessarily mean that its experience in VR will be good as-is! Let’s pick a concrete example: the authentication experience.
Games and apps originally designed to run on a desktop can rely on a vastly different arsenal of affordances: full keyboards, password managers, the ability to switch between your app and a system browser... all those things are either problematic or impossible in VR at the moment.
A typical experience today: you put your headset on, use one controller or basic hand tracking to open an app for the first time, and you are prompted by a floating username and password dialog. Imagine you have a large poster in front of you, right under that dialog, with the picture of a keyboard. To type, you have to point at each key with a laser pointer and pull a trigger. Think about the length and complexity of the passwords you normally use and imagine having to type those using that technique. What’s worse, if you use a password manager, you are unlikely to even know the password. The result is a very tedious process requiring several cycles of lookup and error-prone input, which can leave the user frustrated right when the enthusiasm for a new game should be at its highest.
It is very likely that some of those VR dialogs are literally the same classes developed for the desktop versions of the app, given that the password characters are shielded by asterisks — a measure that makes no sense in VR, given that no one will be peeking above your shoulder. As an industry, we can definitely do better than that.
“It is abundantly clear that a viable vendor-neutral authentication experience for VR apps is necessary to maximize market reach and simplify the user experience.”
Tweet This
Apps developed for a particular VR ecosystem can leverage the accounting system imposed by the manufacturer (e.g., Oculus users must use a Facebook account to interact with the store, notifications, etc.), and in that case, the app can usually SSO with that account. On the other hand, apps are often multiplatform, requiring the use of more portable identities — and many users won’t tolerate being forced into specific account choices, as illustrated by the fierce Facebook backlash. When they announced the aforementioned policy. It is abundantly clear that a viable vendor-neutral authentication experience for VR apps is necessary to maximize market reach and simplify the user experience.
An Incremental Improvement: The OAuth 2 Device Grant
No full keyboard, no password managers, no browser… does that sound familiar? That’s right, Smart TVs and set-top boxes! As an industry, we already faced the problem of performing user authentication on input-constrained devices when Smart TVs began natively supporting apps like Netflix, Prime Video, and other meant-for-the web streaming services. A popular solution that emerged in that scenario is the OAuth 2 device grant. You can find a simple explanation of the flow here.
In a nutshell, the input-constrained device (let’s call it A) that needs the user to authenticate provides a random code and a URL and starts polling an authentication service in the background. The user grabs a fully capable device (typically a smartphone or a laptop; let’s call it B), opens a browser, navigates to the supplied URL, enters the code, and performs whatever authentication is required without the restrictions that would have made the operation hard or impossible on device A. As soon as the user is authenticated on device B, device A automatically receives the necessary tokens, stops polling, and voila! Your Smart TV can now access your Netflix subscription on your behalf.
Our Auth0 Lab team wondered if we could use the same mechanism to perform authentication in VR, where A is the VR headset and B is your phone or computer. We tried it out and learned that, yes, we can! Given that the OAuth2 device grant is an established standard, the solution could also work with any service supporting it- nicely aligning with our vendor-neutral goals.
Here’s a quick video demonstrating the approach in action:
Pretty cool, huh? There is, unfortunately, a minor complication: whereas your smart TV and your phone (let’s pick that as our sample authentication device) are both in your living room, your VR app and your phone live in quite literally different realities. The code and URL prompt of the device grant are displayed by your headset, but while you are wearing it, you can’t really see your phone, not well enough to open a browser, navigate to a URL and enter a code.
To do all that, you have to lift the headset and look at the phone directly. And given that neither the URL nor the code are particularly short, you might need to do it more than once before you’ll be done. That is very far from ideal.
However, let’s remember what our baseline is: having to laser-pointer in VR a long, complicated password we don’t know by heart. That would also require us to peek outside of the headset multiple times to read that password from our phone, probably more times given the length and random nature of passwords, and enter it through a far more awkward input method (e.g., every special character or uppercase letter requires at least lasering the shift key before hitting the actual key) than you’d do entering a short URL and code in your phone. It bears restating: the device code grant in VR isn’t an ideal, ultimate solution, but we believe that it could offer an incrementally better experience than the current use of passwords.
The Experiment: The Auth0 Unity SDK
In the typical pragmatic Auth0 style, we decided we wanted to get our hands dirty with the approach and make it easy for you to do the same. Our Auth0 Lab team put together an experimental SDK (Software Development Kit), the Auth0 Lab Unity SDK, that makes it easy to add device grant style authentication to your VR apps (and to Unity apps in general).
Here’s what the development experience looks like:
- Let's assume you already have your Unity app with the Oculus integration in place.
- You start by downloading the unity package from Github: https://github.com/auth0-lab/auth0-unity.
- Then you import it as a Custom Package (see Option 2 of Setup in the Github Readme).
- Once imported, you'll find the Auth0 Lab Unity SDK content in your assets under the AuthO folder.
Put simply; the Auth0 Unity SDK packs the Auth0 .NET client SDK, some scripts that make it easy to handle the device flow and token persistence/lifecycle via Unity, and ready to use prefabs that make it a breeze to add the authentication experience in VR.
The easiest prefab to use is
DeviceFlowPrompt
The
DeviceFlowPrompt
From the identity setup perspective, all you need to do is to configure the SDK with the domain of your Auth0 tenant and the client id of your app. You do so by filling the blanks in the
AuthManager.cs
...and that’s all there is to it!
“In just a couple of steps, you can initiate authentication from within your VR app, signing in with any identity provider you configured for your app in the Auth0 tenant.”
Tweet This
The Auth0 Lab Unity SDK will persist your tokens and make them available, alongside user claims, whenever you reopen your app — the SDK will ensure that tokens are still valid, will automatically retrieve fresh ones via refresh tokens, and in general, keep offering identity related functions until you hit sign out.
If you want to exercise more control over the look and feel of the authentication dialog, or if you want to target non-VR Unity apps, all you need to do is to use the other prefab,
DeviceFlowRawNext Steps
VR and AR are very important platforms, with huge disruption potential for both consumer and enterprise markets. Although it’s still relatively early, the development tools used to target those platforms are well established — hence we wanted to start a conversation about authentication in that space in very concrete terms by offering the experimental Auth0 Lab Unity SDK for you to experience firsthand opportunities and limitations of tooling today.
The Auth0 Lab has a discord server where we have conversations about our experiments, and we hope you will join us there to discuss IdP-neutral authentication in VR and in Unity in general.
The device grant is an incremental improvement in respect to having to laser-point a password you don’t know on a virtual billboard, but it’s also clear that it ought not to be the endgame when it comes to authentication experience in VR. Entering a validation URL and a code in the phone is not a great experience, even if you manage to do so without taking the headset off by looking through the nose gap.
We can expect improvements in the VR password experience, such as roaming password managers and better system browser integration, but those will be partial solutions as well.
Alternatives available today, such as magic links, codes, etc., all suffer from the same flaw- they don’t enforce proximity between the headset and the authenticating device, raising the risk of phishing-related abuses.
Bluetooth and (eventually) NFC are technically viable to initiate authentication from the headset to an authenticating device, but without a standard profile that would require a dedicated app on the authenticating device, limiting adoption and interoperability. The pressure toward proprietary, integrated stack solutions is very strong in this phase of the technology foundations of the metaverse, but we believe —as always — in the power of open standards as a tool to help everyone benefit. Our hope is that by providing the tools to easily add authentication in VR apps with what’s possible today, we will facilitate the requirements gathering and discussion of what we want the experience to be tomorrow.
Now the ball is in your court. Is IdP-neutral authentication needed in VR? How should authentication in VR evolve? Let’s start a conversation. We can’t wait to see what innovative, amazing experiences you’ll create in this new medium!
About the author
Vittorio Bertocci
Principal Architect