When Apple announced the Apple Vision Pro at WWDC of 2023 it brought an end to years of rumors and expectations. The headset closely matched the rumors, including a key feature that Apple was apparently keen to offer — controller-free, gesture-based controls. While similar headsets used controllers, Apple’s relied simply on hands and fingers. Some see it as a feature, some as a liability that will limit the Apple Vision Pro’s gaming capabilities. A bit like it or not, the gesture-based controls might be on their way elsewhere.
The iPhone is touch-first, as is the iPad, while the Mac is cursor-based which requires a mouse or a trackpad. But Apple is clearly working on a potential input change that will see the Apple Vision Pro technology used across the rest of the company’s lineups, and a new patent proves that even if the headset doesn’t hit the mark with consumers, they could soon be using its technologies instead of mouse or a finger.
It’s still very early days for the Apple Vision Pro and it’s expected to set the foundations for Apple’s spatial computing initiatives for years to come. But even if the headset never captures the public’s imagination the way the iPhone did all those years ago, Apple will want to take as many learnings and technologies from its development as possible. And where best to use those things than its most important products?
Touch-free computing
A new patent, first spotted by AppleInsider, hints at such a move to a touch-free world. Titled “Devices, Methods, And Graphical User Interfaces For Using A Cursor To Interact With Three-Dimensional Environments,” the patent suggests that Apple’s future plans are all about waving hands and fingers around as a form of input.
There are of course similarities between the iPhone and Apple Vision Pro already. The technology behind Face ID can already bring eye-tracking to the iPhone, but the Mac doesn’t have such a thing just yet.
Apple’s patents are always as broad as possible which means they can be read in a variety of ways. But if the Apple Vision Pro brought a Minority Report-like experience to those who bought one, there are now hints that we could be controlling more devices using similar gestures in the future. Could we one day wave our hands at the TV to interact with tvOS, for example? Microsoft already did something similar with the ill-fated Kinnect, but the tvOS Home Screen is already suited to such a control scheme.
Expand that to the rumored Apple TV-turned-HomePod device and we could one day control an Apple smart display without touching it.
All of this could be a little way off becoming reality, of course, and there is always the usual caveat with these things. Apple patents a ton of ideas each and every year, and not all of them turn into products that we can buy at Apple Stores. But some do, and I’m absolutely here for a future where I don’t have to ever touch a Siri Remote again.