ep. 11: Signals of the future - spatial computing themes to watch in 2024
8 min read
Each year, we read an industry trend report, predicting that this will be spatial computing’s year. 2024 is no exception: the International Data Corporation has reported that AR headset shipments are forecast to reach 845,000 this year, up 85.6% relative to 2023. AR/VR headset shipments are expected to grow 46.4%.
I’m taking more stock in these predictions than in the past. Why? The signals of the future we saw in 2023 suggest that we’re gaining traction, both in delivering user value in spatial computing devices and increasing accessibility of these devices. I’ll cover both what these signals were, and the key themes they point to in 2024.
Signal #1: AI and spatial computing became more deeply intertwined, lighting the way from novelty to retention for smart glasses and headsets.
I was standing backstage at Augmented World Expo (AWE) 2023, the leading spatial computing conference. Returning as track chair, I got to introduce keynote speakers before they took the stage. The opening keynote was delivered by AWE co-founder Ori Inbar. I remember deeply resonating with something Ori said early on in his presentation: extended reality (XR) is the interface for AI. That is, our interactions with AI will be less about text and prompts, and more about spatial content and natural human-computer interaction.
Ori’s words struck me because they succinctly articulated something I’d been feeling diffusely since 2021, reading David Rose’s book, SuperSight. Rose described how cloud computing, 5G, and AI computer vision algorithms are all building blocks that come together to set the foundation for a new way to see the world - through AR glasses that can detect what we’re looking at. ChatGPT’s release and subsequent, exponential growth of gen AI accelerated how quickly AI would enable spatial computing’s value for both creators and consumers.
Since AWE, we’ve continued to see further intertwinement of AI and spatial computing. For spatial content creators, we have a growing list of gen AI tools to help create 3D assets that can be leveraged in XR experiences. This will help unblock the content creation bottleneck that has plagued spatial computing. Better (and more) content opens the door to more engaging user experiences. For natural interactions, we’re seeing this actualized via multimodal LLMs that can see and hear, and touchless controls, from headset UIs to ambient computing products like Humane’s AI Pin.
2023 also saw the release of the well-received Ray-Ban Meta smart glasses. These glasses, set to integrate with Meta’s AI tools, have the potential to help people learn more about what they’re looking at, unlocking - what I would argue will be retentive - use cases like recognizing landmarks and seeing real-time information. In a similar vein, Brilliant Labs raised $3M for gen-AI-based AR glasses in October, hinting at an AI assistant use case. We’ve heard buzz that Sam Altman, Sir Jony Ive and Masayoshi Son are working on an AI wearable device, and Samsung filed trademarks in the US and UK for “Samsung Glasses” and “Galaxy Glasses”.
As we enter 2024, we can only anticipate a deepening spatial computing - AI connection. AR Insider released 2024 Predictions: AR & AI Get Hitched, “XR can be the face of AI, while AI is the brains of XR”. Two ways this is predicted to materialize are the continued growth of creators using Gen AI to create 3D models that speed up XR prototyping and iteration, and LLMs enabling more natural, conversational UX in XR devices.
While I’m bullish about this ‘hitching’, I offer a caveat: to truly make AI-powered functionality useful and usable for people in spatial devices, we’ll need to decrease the latency between user query and gen AI output. This might take the form of 5G and/or processing generative AI models on-device (granted, both of these might be harnessed by spatial computing devices sooner than later).
2024 theme to watch: Increased growth and early adoption of spatial computing wearables powered by gen-AI, to help people learn about the world around them and access ChatGPT-style assistants.
Signal #2: Augmenting our existing screens with smart glasses and headsets signaled clear consumer value - a big win for technology that has struggled to show an appealing mainstream use case.
In 2023, we saw XReal and Rokid reveal new glasses to provide a wider-screen, heads-up alternative to laptop and console monitors. Apple highlighted how their Vision Pro can “Free your desktop. And your apps will follow”, and rolled out spatial video recording for Vision Pro with the iPhone Camera. Disney Plus content like sports games, shows, and movies will be available in Vision Pro. These developments, more akin to the release of a next-gen TV than a deeply embodied AR Glasses North Star, cracked the door open to something practical and desirable you can do with these devices. This seemingly small win will be important scaffolding for the next generation of devices to come (see Signal #1).
2024 theme to watch: Wider adoption of screen extension use cases for gaming, entertainment and productivity, accelerated by increased use of smart glasses and Apple Vision Pro (shipping Feb 2, in case you missed it).
Signal #3: Privacy concerns around AI and sensor technologies’ maturation moves from discussion to law.
In 2023, the FTC issued a policy statement addressing emerging technologies (especially in relation to consumers’ biometric data) that might harm consumers and violate the FTC act. At least 15 biometric privacy law proposals also emerged across 11 states, imposing new requirements on companies’ collection, handling, protection, use, and dissemination of biometric information. Examples include retina or iris scans, fingerprints, voiceprints, and scans of hand or face geometry - all data you could expect to collect on a spatial computing device.
In September, the UK passed the Online Safety Bill, requiring companies to assess the likelihood of users encountering illegal content and of children encountering content that could be harmful to them, and how to mitigate these risks. Virtual spaces were in the bill’s scope, with Lord Stephen Parkinson, member of the Parliament's upper house, specifically naming the metaverse being in scope of the bill.
2024 theme to watch: Increased legislation and formation of independent oversight bodies to set stronger privacy guardrails, especially for technology at the wearables-AI intersection.
Signal #4: Multisensory inputs make for more natural interactions, reducing friction around spatial computing device use.
Eye tracking & hand tracking & contextual awareness, oh my!
Apple’s Vision Pro promises “a high-performance eye tracking system of LEDs and infrared cameras that projects invisible light patterns onto each eye.” Both Vision Pro and Meta’s Quest 3 utilize direct user input (“direct gesture” and “direct touch”, respectively). This means a person can interact with a virtual object using a direct gesture, such as touching the object with their finger. Direct interaction unlocks more natural user interfaces, reducing layers of interface abstraction by allowing us to use our hands in daily life. Apple’s Spatial Audio is as much a contextually-aware input as output: “The technology models the sonic characteristics of a person’s surroundings, automatically making audio sound natural in their space”. Humane’s AI Pin sees and hears its user’s context, using AI to help people communicate with others in a different language or learn more about their environment.
2024 theme to watch: Spatial computing devices, leveraging multimodal LLMs, will continue to build out their perceptual systems, becoming more contextually aware. The goal: more accurately anticipating user intent to deliver more relevant outputs.
Signal #5: Early growth of cross-platform development and interoperability.
In 2023, we saw the launch of several tools that help creators develop cross-platform spatial experiences. Following the Vision Pro announcement, Unity opened their beta program for creating spatial experiences on the visionOS platform - aka Polyspatial. Unity also launched AR Foundation, allowing developers to create multi-platform AR apps. Its supported provider plug-ins include Meta OpenXR for Meta Quest, as well as support for developing on HoloLens 2, Android and iOS.
Both Pico 4 and Magic Leap 2 announced that they were supporting Open XR, the open standard that enables cross-platform AR and VR development. The list goes on (see #12 in Tom Emrich’s excellent 2023 spatial computing lookback).
These tools all help you develop and deploy cross-platform experiences. Now, how should you adapt a given experience for each platform? For example, if you’re an iPhone app developer, how do you adapt your content for Apple Vision Pro? What does responsive design look like for spatial experiences?
Last year, I shared my Spatial Translation Framework, to help creators decide when and how to translate 2D content into 3D, embodied content, for use in spatial computing headsets. With both Apple Vision Pro and Meta Quest 3 in consumers’ hands this year, we’ll see an increased need for - and output from - the creator community around best practices for spatial responsive design.
Last but not least, there’s interoperability. How do our new spatial computing devices play nice with the tools we’ve already adopted? It’s not like we’ll all receive our shiny new smart glasses or headsets and chuck our smartphones into the garbage can. Apple has elegantly displayed strong interoperability between Vision Pro and their device ecosystem - for instance, showing how a Vision Pro user can glance at their MacBook, connecting the two so that the vision Pro can serve as a Mac display (see #2 in this newsletter). Ensuring interoperability between our existing tech ecosystem (e.g., smartphone, smartwatches, laptops) and smart glasses/headsets will be critical for delivering a strong user experience and developing return use.
2024 theme to watch: Defining responsive design for spatial content, and designing for interoperability (e.g., between mobile and headset).
New Year’s News Roundup
My top three favorite pieces around 2024 predictions in the realm of spatial and AI:
24 Augmented Reality Trends to Keep an Eye on for 2024 by Tom Emrich
2024 Predictions by AR Insider
The State of AI in 2024 by The AI Breakdown podcast
Is your team working on launching a 0-to-1 product in an emerging space like spatial computing and AI? Sendfull can help you build for user needs and evaluate your solution. Reach out at hello@sendfull.com
That’s a wrap 🌯 . More human-computer interaction news from Sendfull next week.