ep. 44: Designing AI to Think With Us, Not For Us
10 min read
Last week, I shared live updates from the 20th annual EPIC conference, the premier international gathering for practitioners using theory and methods from the social sciences to advance business and organizations.
The theme of this year’s conference was Foundations, Displacement and Generation. I presented a paper, Designing AI to Think With Us, Not For Us, where I describe how generative AI (genAI) challenges the foundations of how human-centered practitioners work, and shared a new framework for building human-centered AI.
Today, I share a summary of my presentation (the published paper will be available this November). Are you working on genAI tools, especially developing new ideas, features and market positioning ? Let’s chat about how we can work together on a case study. Reach out at stef@sendfull.com
The Foundations of the Design Process
Human-centered design practitioners, such as designers and design researchers, typically start the design process by seeking to understand existing problem spaces. Examples include learning about a potential audience’s current context, goals and pain points.
Following a thorough search of problem space, we identify “people problems” and opportunities to address with a design solution. Solution space follows from problem space. We advise our teams against leading development with a given technology solution. This problem-space-first approach is key to the reasoning processes and subsequent problem framing that experienced practitioners engage in (more on that in episode 30).
This approach is modeled in classic design frameworks such as the The Double Diamond, where we diverge and converge to land on the problem we’re solving, and again, to identify and refine the solution to that problem.
GenAI is Challenging the Foundations of How We Design
Enter genAI, a class of machine learning that generates text, images or sound output, based on user input across these same modalities. GenAI sprung into mainstream consciousness in November 2022, with the release of Open AI’s ChatGPT. ChatGPT became the fastest growing application in history, reaching 100 million monthly active users within two months. Since then, we have observed unprecedented, exponential and worldwide growth of genAI.
GenAI isn’t just disrupting markets - it’s disrupting the foundations of how we work. This quote by Paul Graham, co-founder of influential venture capital firm and startup accelerator, Y-Combinator, sums up the disruptive innovation force gen AI is exerting.:
GenAI is “the exact opposite of a solution in search of a problem. It's the solution to far more problems than its developers even knew existed.” - Paul Graham, co-founder, Y-Combinator.
GenAI - this rapidly evolving technology solution - is unlocking a new range of problem spaces to which it can be applied. The diagram below shows how genAI solution space is increasing in size from left to right. A corresponding, increasing number of problem spaces are expanding against this backdrop.
This isn’t the first time we’ve seen a disruptive innovation exert this solution-first push unlock new problem spaces. For example, we needed Apple’s iPhone (solution) to unlock the potential of mobile-first use cases (problem spaces) - for instance, sharing photos via social media (e.g., Instagram), ridesharing (e.g., Uber) and mobile payments (e.g., Venmo). However, the timescale of solution space advancement was longer and more incremental than what we’re seeing when it comes to genAI development.
GenAI’s fast-paced, solution-first dynamic creates a foundational challenge for human-centered design practitioners, whose playbooks tend to focus on solving for existing problem spaces, and starting from problem space before moving to solution space. We need a new framework that addresses this foundational challenge.
The Problem-Solution Symbiosis Framework
I offer the Problem-Solution Symbiosis (PSS) framework to help resolve the tension between current human-centered approaches and the dynamics of genAI’s solution-first development.
The PSS framework models the co-existence of problem space (solid purple line), and solution space (dotted orange line). When we embark on designing a new product built on genAI, problem and solution space exist in parallel. They converge over the course of product development, leading up to launch. This convergence is driven by touchpoints at which each space can symbiotically inform and evolve the other. This is represented by the arrows between the two spaces.
There tends to be team functions representing each line - often, the product team for problem space, and machine learning engineers and research scientists driving solution space development. This symbiosis requires someone to lead orchestration between these two spaces and their respective groups. Human-centered practitioners (e.g., design researchers) are uniquely well-situated here, with expertise in building coalitions between different perspectives between cross-functional team members.
Three Aspects that Make GenAI Solutions Uniquely Hard to Design For
The PSS Framework models how we can address the foundational challenge presented by genAI’s rapidly-moving, solution-first momentum. Amplifying the challenge are three aspects that make genAI solutions uniquely difficult to design for: emergence, AI-first strategy and 10x cognitive offloading. I’ll cover what each of these challenges are, and offer the PSS Toolkit - three tools that address these aspects, and make the PSS Framework actionable.
Challenge #1: Emergence
Emergence is when genAI models exhibit unexpected capabilities. For instance, when engineers make a small change to one part of the model to better solve certain types of problems (e.g., math problems), new behaviors emerge as a result of the system’s complexity. As model complexity increases with each new generation, so does the difficulty of predicting their behavior before the system is built.
Challenge #2: AI-First Strategy
Business leaders are pushing to apply gen AI solutions - or take an “AI-first strategy” - to ride the wave of this disruptive innovation, and keep up with competitors. This is a downstream effect of genAI’s solution-first momentum. AI-first strategy often looks like adding genAI to an existing tool, perhaps with an existing end user in mind - something like this:
Challenge #3: 10x Cognitive Offloading
GenAI is capable of cognitively offloading increasingly complex “thinking tasks” with each subsequent generation of foundation model.
Cognitive offloading is the delegation of our cognition to technology. This concept itself is not inherently associated with genAI. For instance, take away Google Maps, and I lose the ability to navigate even in a familiar city. This goes into 10x mode with GenAI.
Some of us already use GenAI to retrieve or process information that our brains otherwise would have handled - such as transcribing meeting notes in real time, and providing a summary with action items after the call (e.g., using tools like Otter or Fireflies).
The type of “thinking tasks” genAI can perform is only increasing. The technology industry widely accepts that artificial general intelligence (AGI) - roughly defined as “AI systems that are generally smarter than humans”, is an inevitable end goal.
We’re offloading tasks people don’t want to do, right? Writers, designers and increasingly, other professions may disagree, seeing genAI solutions infringing on what makes us, us. At the same time, there is potential to offload tedious, “uncreative” parts of our workflows to genAI.
The Problem-Solution Symbiosis Toolkit
PSS Toolkit helps human-centered practitioners guide genAI development, making the PSS Framework actionable. It consists of three tools that map to the unique challenges of designing for genAI:
Building genAI intuition as a team, to address emergence
Mapping problem space ecosystems, to address AI-first strategy
Cognitive Offloading Matrix, to address 10x cognitive offloading
Tool #1: Build GenAI Intuition as a Team
The first tool is building genAI solution space intuition to identify use cases and strengths. This starts by spending time interacting with existing frontier models. Ethan Mollick, Wharton Associate Professor of Management, suggests spending at least 10 hours of using one of the frontier models (Claude’s Anthropic, Open AI’s ChatGPT) to develop a sense of the use cases to which genAI could be applied in your product space.
Next, bring together teams to playtest. This looks like setting goals or tasks that are relevant to your product area, to be performed using a genAI model - ideally the one you’re using for your solution. Playtesting together builds a shared understanding of what the genAI solution can and cannot do, allowing the team to form a point of view on your solution’s unique strengths (and weaknesses).
This approach is informed by an approach I’ve used in my own practice in an analogous, emerging technology space of spatial computing (e.g., augmented reality). Leading hands-on playtesting helped teams move beyond the hype and see the realities of what solutions realistically could and could not do. This ultimately helped us identify applications that were both desirable and technically feasible.
Tool #2: Map Problem Space Ecosystems to Uncover Opportunities and Risks
The second tool is mapping problem space ecosystems. Building for GenAI, we are contending with significant push from business leadership to apply genAI, often to an existing customer base. This presents two problems - development is heavily influenced by the values and incentives of the stakeholder, and we are often limited to focusing on one end user, despite genAI being a complex system that impacts numerous stakeholders.
This tool encourages taking an ecosystems-level view. This looks like mapping the different stakeholders within a genAI landscape, identifying each stakeholder’s values and incentives. Workshopping on this with your team can identify risks, opportunities and assumptions. Start by identifying all relevant stakeholders. Then, map how they interact (e.g., one stakeholder per bubble, and show how bubbles overlap). At this point, you can discuss what each stakeholder values, what their incentives are, and how they’re interconnected.
Tool #3: The Cognitive Offloading Matrix
The last tool is the Cognitive Offloading Matrix, to help us choose which tasks to delegate to genAI that can yield useful, desirable genAI-based solutions for people - tools that think with us, not for us.
The x-axis includes strengths of humans versus genAI. This is rooted in the Fitts List, a tool created by psychologist and human factors researcher Paul Fitts in the early 1950s, which delineated identifying what people versus humans excel at (e.g., people better at anomaly detection, machines better at detecting things beyond perceptual thresholds, such as infrared).
Human strengths are are informed by secondary and primary research. For solution strengths, you can use Tool #1. Some existing guidebooks, like Google’s People + AI Guidebook list of “when AI is probably better” can also serve as a help starting point (remember to adapt based on the genAI tool you’re considering for your product). Ideate on the strengths of your genAI solution you’ve identified from intuition building.
The y-axis includes tasks - what people prefer doing versus not doing, desirable versus undesirable, which you’ve learned via ethnography. Generating examples for each quadrant is best done collaboratively, as a team workshop activity, to foster alignment. The sweet spot of the matrix is the bottom right, where you have tasks that are undesirable for your audience to perform AND leverage genAI’s unique strengths.
Let’s consider an example scenario, in which a team is considering applying genAI to a music streaming tool. Assume you have some understanding of your audience’s contexts, goals and music streaming behavior (e.g., from design research). You’ve also applied Tool #1 to build intuition about the genAI solution at hand.
Strengths:
Human strength (informed by research): making recommendations based on cultural nuances or non-obvious connections.
GenAI strength (informed by intuition building from Tool #1): reliably generating large-scale, real-time personalized recommendations.
Tasks:
Desirable tasks for your target audience: Your audience enjoys building custom playlists for loved ones (research finding). They will take the time to do this, and it’s important to both parties (playlist creator and recipient) that the playlist has a personal touch.
Undesirable tasks for your target audience: Having to tweak music to match context. For example, you’ve learned from your research that people are frustrated when the streaming service recommends the same low-energy music people want to hear when winding down in the evening, while embarking on an early-morning drive.
Mapping these strengths and tasks onto the Cognitive Offloading matrix, we see this:
We need to be extra mindful that we’re not offloading tasks that are desirable - something people enjoy doing - to genAI. This “danger quadrant” is in the upper right of the matrix at top left. At the very least, don’t prioritize building and marketing a feature that boasts “fully automated custom-playlist building for loved ones”.
It’s more strategic to offload tasks people don’t enjoy doing and leverage genAIs’s unique strengths (and ideally, not just relative to humans, but also something that no other technology solution can do!). These opportunities live in the bottom right quadrant of the matrix.
Solutions that use genAI’s unique capabilities to help people do things they don’t like doing are more likely to be useful and desirable - and therefore, adopted. In the streaming example, this could be having genAI recommend music based on your context (e.g., it’s 6am and in you’re in a car, and the last time you were in this context, you played energizing dance music - so recommend an uptempo dance playlist).
Marketing Applications
The matrix can be applied early in product development, to prioritize which ideas and features to pursue. However, we also need to consider how we position genAI capabilities in marketing materials. In my EPIC talk, I shared a recent ad for genAI tool Google Gemini, which was recently pulled after backlash because of what it depicted: a father using Gemini to help his daughter write a letter to her Olympic hero, American hurdler Sydney McLaughlin-Levrone. If genAI is supposed to give us back time for more genuine human interaction (like writing a heartfelt letter with your child), this ad showed the opposite, outsourcing this special task to genAI.
Given that public trust in AI has been declining, us folks building these technologies must give extra intentionality to what parts of the human condition we show as “offloadable” to genAI systems. We can use the Cognitive Offloading Matrix for this purpose. In the case of Google Gemini, co-writing a heartfelt letter with your child” would fall into the red “avoid offloading to genAI” quadrant.
Through conversations at EPIC, there are also opportunities to use this matrix to identify where genAI technologies can be maximally helpful within teams. For example, a team of design researchers could playtest different genAI tools given their goals and workflows, map what they would consider offloading with the Cognitive Offloading Matrix, and consider how tool adoption can impact relationships with their stakeholder ecosystem.
📣 A Call to Action
The next step for the PSS Framework is to get more teams using it to help them navigate genAI product development and marketing. Are you working on genAI tools, especially developing new ideas, features and market positioning ? Let’s chat about how we can partner on a case study. Reach out at stef@sendfull.com
That’s a wrap 🌯 . More human-computer interaction news from Sendfull next week.