How Apple’s virtual reality will interact with your lived experience

So much is written around AR and VR, and yet if you look at the numbers it’s evident that consumer and enterprise users remain to be convinced that these virtual experiences add value to their lives. We’re living through tough times, and we’ve become a tough crowd.

All the same, Apple will we think introduce its take on altered reality at some point this year. Here is how it may interact with your lived experience.

Hey, is this thing on?

First, we think the first generation Apple device will be relatively expensive and built for use at home for short periods, rather than for porting while out and about.

Indeed, more recently we heard Apple may have had to put plans for a ‘wear everywhere’ device on hold, partly because the tech to make that happen affordably doesn’t yet exist.

Maybe.

Bloomberg’s Mark Gurman wrote an extensive report explaining lots about the purported device. I thought I’d go through it to explain how it currently seems these things will work – though development is ongoing, and I get the impression the final experience isn’t nailed down.

Which is why I think these products won’t ship till later in the year, though we may get teased and titillated by an early glance in spring and a developer kit at WWDC. We’ll wait and see how that pans out. So, how will we interact with these things.

Ski goggles for special people

Think ski goggles with little slots for prescription lenses and external battery packs you carry with you. The headset will be festooned with cameras designed to track your hands and what’s around you, along with sensors inside the gadget to read your eyes.

The eyes have it

Reading your eyes means you’ll be able to control the device by gaze – presumably that implies that if you stare at the dock along the bottom of your screen (presumably, though that is a guess) you’ll be able to invoke an app or service. Like pointer devices, you should be able to navigate these things by staring, coupled with gesture, pinch, and voice. I guess EyeID will be part of this equation, too.

Clench and bump

Gesture will extend to the kinds of things you can already almost do (but not quite well enough, though perhaps that is just me) on Apple Watch. Pinch thumb and index figure together to make something happen, fist bumps, clenches and the like.

Speak gently to me

You think Siri will be part of this? I think Siri will be part of this.

A mobile Home

Gurman speculates on a grid-based virtual Home screen (like iPads or iPhones) with an icon-based view. I can easily imagine widgets will be part of this also

Reach out and touch me

You’ll have a little touch in the form of a Digital Crown. This will let you switch functions between fully immersive VR experiences and useful augmented reality sessions. In those sessions I guess your lived experience will be boosted with useful information on demand. Your apps and services will interact with what’s around you. That’s the information piece, but it also suggests you’ll be able to use apps in this mode while remaining aware of your surroundings.

Speak to me, baby

FaceTime will ‘realistically render a user’s face and full body’ as avatars in VR. You’ll be able to communicate as if you are in the same room. These will be realistic avatars, rather than cartoons. I’m guessing (but don’t know) that apps like Freeform will exist in these environments, which means even people without access to these things will be able to interact usefully with those who do.

A Mac you wear like sunglasses

I think the direction of travel is a Mac you wear like sunglasses. These devices will work as an external display for your Mac. You’ll see this in VR but control it with your mouse, trackpad, or keyboard. Just like Universal control lets you use one keyboard and or mouse to control multiple Macs and iPads.

Master of the universal (control)…

It will also work with external control devices such as mice on its own account. That may sound a little weird, but I suspect might make it easier for third party developers to offer their apps for these devices with less initial investment while the water is tested for heat. Apple is working on a virtual keyboard, but that’s not ready yet, Gurman says.

But it’s a mobile device

Gurman speculates the xrOS will introduce 3D experiences that echoes some of those you get on iPhone and iPad, including Safari, photos, and Mail.

With an App Store

You’ll have an App Store from which you can purchase apps. You’ll also get Apple’s media services, so you can sit back and watch a movie in a deeply immersive way on the plane or train.

You’ll be able to try before you buy

Apple Stores will have dedicated areas in which you can try these things out for yourself. The company is also apparently aiming for a million sales in the first year and doesn’t anticipate an early profit on these devices. Though it does think its existence will help spike sales of its other products.

What’s this thing wear?

The report claims Apple has designed something classy that echoes the AirPods Max – think aluminium, glass, cushions. A curved screen will show a wearer’s eyes, there will be a headband and speakers on the side. A lot like this.

When will this happen?

I think this is going to be the start of a trail for Apple. We’ve already seen other big tech firms enter the space with solutions that don’t quite resonate.  The burning question will be precisely that. When people use these things do they feel like they’ve been using them forever, or will they remain forever in the shallows around novelty island with no one quite convinced?

We can only wait and see. But I predict a bit of a tease on our journey to release day.

Please follow me on Mastodon, or join me in the AppleHolic’s bar & grill and Apple Discussions groups on MeWe.