ep. 9: What does it mean to augment human capabilities with technology?
7 min read
As I read Professor Fei Fei Li’s new book, The Worlds I See, I find myself coming back to a key theme across her work: AI should augment, not replace, human capabilities.
Professor Fei-Fei Li driving home a key point of her talk at October’s BayLearn Machine Learning Symposium.
This phrase reminds us that we need to build tools that solve real human problems that positively extend our abilities. For example, helping people see what they would be otherwise unable to see (think leveraging AI tech to help detect and prevent medical errors). However, this also means we’re delegating increasingly advanced “thinking tasks” to machines (e.g., using ChatGPT to write a book summary). We’ll address what this delegation means for human cognition and behavior, and offer three questions to ask yourself as you develop a new product, to keep you focused on building tools that augment human capabilities.
What is cognitive offloading, and is it bad for people?
This phenomenon of ‘thinking task delegation’ is called cognitive offloading. It is often viewed as inherently bad, making us less intelligent. For example: take away Google Maps, and I lose the ability to navigate even in a familiar city. Cognitive offloading goes into ‘10x’ mode with AI technologies that can do “thinking things” like carry on conversation, summarize books or or schedule our meetings. Is cognitive offloading all bad, something to be avoided?
Not exactly.
Cognitive offloading from the brain to different technology tools. Image created in Image Creator from Microsoft Designer.
When we adopt a new technology tool, it changes what we focus on (and relatedly, our behavior). We can understand this shift using the framework of distributed cognition: that is, cognition doesn’t only exist in our individual minds, but across the members of a social group, between people and their material environments (including their technology tools!), and through time. Returning to the Google Maps example: I have a mental representation of my destination, and distribute the ‘navigation’ part of cognition to Google Maps to get me there.
This change in what we focus on is seen as positive by some. In Design of Everyday Things, Don Norman states, “Does the fact that I can no longer remember my own phone number indicate my growing feebleness? No, on the contrary, it unleashes the mind from the petty tyranny of tending to the trivial and allows it to concentrate on the important and the critical”. In Things That Make Us Smart, Norman posits that “the power of the unaided mind is highly overrated. It is things that make us smart”.
My point of view is that cognitive offloading, while often useful to people, should not be accepted as inherently positive, with no further questions asked. We must consider the downstream effects of new technologies on how we move through the world. The smartphone unlocked incredible human capabilities. However, it also turned us into people hunched over our screens. Is the juice worth the squeeze for users? For society?
A path forward
As the people building new technology tools, we must consider what capabilities we’re unlocking for people, and what they're no longer focused on doing as a result of using what we build. One of the reasons I’m excited about spatial computing wearables is that, if designed well, we can unlock human capabilities - for example, seeing what is currently invisible (e.g., AR glasses providing information about what building you’re looking at). This allows you to learn more from your context than you would without AR glasses, and refocuses the user on the building rather than looking down at their phone to look up the information.
We can also think about how we can have people and technology work together in new ways to create value. We can see an example of this in Stanford professor Erik Brynjolfsson’s remarks at a 2012 meeting of the National Academy of Engineering: the best chess player in the world is a team of humans and computers working together, rather than just a computer or a human. Garry Kasparov, chess grandmaster, describes the results of a 2005 freestyle chess competition: Human strategic guidance combined with the tactical acuity of a computer dominated over Hydra, a chess-specific supercomputer. In other words, the winners were those who used the unique skills of humans and computers to complement each other.
Relatedly, should consider what human challenges we can face better with technology than alone. Dr. Li offers more examples of this, from the mundane (cleaning up after a wild party - one of the top three tasks people wanted a robot’s help on) to AI helping address the medical labor shortage, serving as another set of eyes to help healthcare professionals improve patient outcomes.
Takeaways
Emerging technologies, and especially AI, should augment, not replace, human capabilities.
To do this, consider the following questions as you develop a new product:
What human challenges can we face better with your product than alone (or, than with existing products)?
What capabilities does your product unlock for people - something they couldn’t do as well before using your product? Think cognitively, such as how your product impacts attention, decision making and memory.
What capabilities does your product replace, and what does that mean for the user and the people around them?
Are you building a 0-to-1 AI product and thinking about how you can better unlock human capabilities? Sendfull can help. Reach out at hello@sendfull.com
Human Computer Interaction News
Mental image reconstruction from human brain activity (Neural Networks): AI is getting better at “reading” your brain. This academic article proposes a novel framework for integrating Bayesian estimation and semantic information assistance, improving the ability to reconstruct both seen and reimagined images from brain activity relative to past methods.
Uncovering the AI industry: 50 most visited AI tools and their 24B+ traffic behavior (Writerbuddy, Sujan Sarkar): In this study of 3,000+ AI tools, authors scraped data from various directories that list AI tools. Isolating the top 50 most visited tools (which reflect over 80% of the AI industry’s traffic) from September 2022 to August 2023, key takeaways included: The top 50 AI tools attracted over 24 BB visits, leading with ChatGPT, Character AI and BARD. 63% of AI tool users accessed via mobile devices.
50 most visited AI tools, September 2022 to August 2023.
ChatGPT one year on: who is using it, how and why? (Nature): Seven scientists share what they have learned about how ChatGPT should and shouldn’t be used. Insights include: “Fix, don’t amplify, biases in health care”, “thing about whether it should be used at all”, and “use it for structure, not content”.
How Does AR Impact SEO? (AR Insider, Justin Scott): Learn three tips for AR and search engine optimization: generate high-quality product images via 3D/AR, include AR and virtual try-on language in descriptions and alt text, and optimize calls to action to encourage AR discovery.
2023 Design Tools Survey (UX Tools): This annual survey of 3281 UX professionals from around the world provides a snapshot into the State of Design. Across respondents, notable findings include:
Figma was the most popular tool used for UI design, basic and advanced prototyping, and design systems.
22% of respondents don’t do user testing, primarily because it’s not part of their team’s process. (Putting this more optimistically, 78% do conduct user testing).
65% of respondents already are using AI in their work. 84% of these respondents use ChatGPT, followed by Bard and Bing.
Meta and IBM launch ‘AI Alliance’ to promote open-source AI development (The Guardian): Per Meta’s press release, the AI Alliance is “focused on fostering an open community and enabling developers and researchers to accelerate responsible innovation in AI while ensuring scientific rigor, trust, safety, security, diversity and economic competitiveness”. The 50+ founding members of the alliance include Hugging Face, Intel, and NASA, as well as numerous universities such as Dartmouth, Berkeley, and Yale. Unsurprisingly, Microsoft, OpenAI and Google are not on the list.
2024 Predictions: AR & AI Get Hitched (AR Insider, Mike Boland): Learn why extended reality (XR) and AI go together to make each other stronger: “XR can be the face of AI, while AI is the brains of XR”. This will manifest in two ways:
Generative XR: Just as we’ve seen AI create 2D art from text prompts, people building XR experiences can generate 3D models to speed up prototyping and iteration, helping unblock the content creation bottleneck that has plagued the field. (This reminds me of Brilliant Labs raising $3M for generative AI-based AR glasses in October).
Conversational AI x XR: LLMs will enable more natural, conversational UX in XR devices.
These predictions mirror a key theme at this year’s Augmented World Expo, XR is the interface for AI.
That’s a wrap 🌯 . More human-computer interaction news from Sendfull next week.