To me, the issue is not “AI causes cognitive offloading and should therefore be banned,” but rather “AI companies have shown they are not addressing cognitive offloading.” Soon, I think, it will be “AI companies cannot be trusted to make responsible products and therefore should be regulated.” Social media companies externalized their responsibility and are kind of being held accountable. But the effects of whole generations of cognitively illiterate people is terrifying enough to require regulation. The companies are not doing this.
Yes, we’re in the next rev of The Anxious Generation. In addition to detrimental cognitive offloading, there are also the effects on emotional intelligence that I didn’t touch on here. If you cut your socialization teeth on sycophantic LLMs, you’re likely going to struggle with the complexities of human-human relationships.
I’m not confident about forthcoming regulation given that we’re only starting to see legal action against corporations for social media addiction now. There *are* a number of bills on AI in education, but mostly at the state level and mostly on integration and literacy: https://www.future-ed.org/legislative-tracker-2026-state-ai-in-education-bills/ I worry whatever comes will be too little, too late.
In the meantime, AI system design feels like an area where there are a few degrees of freedom. The prompt bar and current AI output format have parallels to the social media feed and the like button; they need to be more widely seen as such. The former invites unwarranted trust and detrimental offloading, whereas the latter invites addictive engagement.
Putting on my pitch deck hat, these patterns are also lazy design. It’s early days and UX/UI can be rethought to be a differentiator.
TLDR: I’m writing AI system design because it feels like there’s a readily available leverage point here. However, I acknowledge it’s still relatively close to the fulcrum. This is a wicked problem and needs multiple categories of intervention to affect systemic change.
I'm not confident in that the regulatory changes would be comprehensive, standardized, and truly international either. So, in effect, as you say -- "too little, too late."
However, people want controls on AI. This may show up as social norms against AI usage, for example (we already see AI shame).
Hi! I recently quit my job as a product researcher at Google to work independently at an industry level to address this exact issue. I truly believe these are relatively easy design problems to tackle and companies will address them with enough external pressure, because (some of them) want to be a platform appealing to high school and college students . Given Anthropic has documented evidence from their own internal research that supports what academics found and there is now precedent that the harm can be in the design, they have a lot of reasons to take action on this immediately. At this point not addressing this would be considered negligence on their part. I also believe there is emerging evidence that the current UX of chatbots is leading to harmful results in other domains as well and there need to be a broader reckoning around this. I wrote about a this here https://earlyinsightsclub.substack.com/p/the-ux-of-ai-chatbots-is-risking?r=6ju2pa&utm_medium=ios
"What’s especially notable about these two categories of harm is that they are being experienced by the two earliest adopters of AI: students & tech workers" - yes!
I keep coming back to why this isn't getting built. The 10x cognitive offloading potential of this tech was evident early on. Transcending the chatbot could be a key differentiator. Are alternatives seen as too friction-filled, at risk of negatively impacting DAU? Is the space just moving so fast that "slowing down" to rethink the interface feels, well, too slow? What would need to be true to incentivize better design?
I think there's a big disconnect between the work happening in the labs ( that could have predicted this years ago) and the feature teams building UIs. A few other speculations: 1) companies are primarily focusing on pleasing early adopters (who can keep the hype cycle going), who I think will push back against anything they think is restricting the product (I've seen this when Claude prompts people to go to bed). 2) those same early adopters have essentially created a community around teaching each other the best prompts/workarounds on these issues, so while these UX feels obvious to someone someone who is thinking about 'designing for all', early adopters blamed these failures as "skill issue" on the user 3) AI companies are heavily benefiting from keeping the system undesigned for as long as possible to see how people use it, so it can inform what to build. This is why I think Anthropic prides it self in launching as many features as possible. I don't think they are thinking about designing a cohesive product, just one that pushes capabilities as much as possible.
The design question is 100% on point--and the writing tool you're describing is a genuine improvement over "here's your essay," right? But I keep thinking about how making the boundary visible isn't the same as making it legible. For a student who doesn't yet have the perceptual equipment to know what an "argument" worth articulating looks like, "write your two sentences first" might function as a compliance step rather than a judgment exercise--something to clear before getting to the formatting help. Seems like the scaffold works beautifully for the student who already has partial formation: it names what they need to do, and then gets out of the way. .. but for the student who lacks that formation, I'm betting the two-sentence exercise itself surfaces the gap--which means the tool ends up triaging the formation problem, rather than solving it. That might actually be the strongest design goal: not replacing prior formation, but making visible where it's absent. Which hands the student back to something slower, probably. Thanks for pressing this, Stef--the exchange clarified something for me, too.
The intervention that caught my attention: teaching students to protect "higher-order work" (analysis, evaluation, reflection) while delegating lower-order tasks to AI. The design logic is compelling, but I can't help wondering whether the very distinction between what's lower and what's higher is itself a perception one develops through the struggle you're now asking students to skip. Seems like the ability to recognize "this is the part that requires my judgment" is exactly what early, unglamorous practice builds--which would make the lower/higher line only legible to learners who've already (at least partially) done the work. If so: the intervention reaches farthest for students who already have some prior formation, and least far for those who need it most. I think that's the equity risk buried in the research--not just access to structured tools, but access to the prior formation that makes the structure intelligible. Genuinely useful framing here, Stef.
You've surfaced an assumption baked into most current AI use: that users can become systems thinkers without putting in the reps. The lower/higher distinction is earned through practice. Without that formation, AI widens the gap between students who already have it and those who need it most.
The design question this raises for me: can AI systems pre-bake the delegation structure rather than leaving it to the user? I think so. Consider a writing tool that, instead of generating a full draft on command, first asks the student to articulate their argument in two sentences, then offers to handle citations and formatting. The system is making the boundary visible: this part requires your judgment, that part doesn't. The student who can't yet distinguish the two gets the scaffold; the student who already can moves faster. That's a different design posture than "here's your essay."
The neutrality here is rare and worth noting. Following evidence rather than a predetermined conclusion is harder than it looks when the topic is this charged.
The performance paradox framing is the sharpest thing in the piece. Output up, capability down. But it raises a question the article gestures at without fully landing: is the offloading problem individual, or is it educational? And does something have to go wrong during school years before AI becomes a crutch?
The studies suggest that structured interaction with AI produces good outcomes. But those interventions all assume a baseline capacity for self-regulation and meta-awareness. The ability to notice you're outsourcing. To pause when prompted. To evaluate rather than just consume.
That capacity is supposed to come from school. But educational systems were quietly offloading that same work long before AI arrived. Multiple choice exams train recognition, not recall. Heavily scaffolded assignments remove the productive confusion that precedes real understanding. The system optimized for measurable output and got exactly that — performance that doesn't necessarily indicate capability.
So when AI offers a smoother bypass, it isn't introducing a new vulnerability. It's exploiting one that was already there.
The equity point compounds this. A student with strong foundations uses AI as an accelerant. A student without them gets a convincing simulation of having foundations. Both look fine. The gap widens invisibly, without the feedback loop that would normally correct it.
And the teacher who might set the guardrails the article recommends came out of that same system. Was validated by it. May not experience the gap as a gap at all.
The interventions that work require someone who already thinks clearly about the difference between performance and capability. The system doesn't reliably produce those people. It occasionally produces them despite itself, and then makes their work harder.
AI in education is a solvable problem technically. The problem underneath it is older and considerably less tractable.
Agreed, AI didn't create the tendency to optimize for measurable output over durable learning. It made that existing gap faster and harder to detect. You picked up on a point I ended up removing from this piece for sake of length. It tied back to episode 88: AI is an accelerator and amplifier for whatever system it's attached to (https://sendfull.substack.com/p/ep-88-ai-is-an-accelerant-what-are). That includes widening the gap between students with and without strong foundations. "Better" design alone won't fix the systemic problem, though it can help stop (or slow) the tools from making it worse.
To me, the issue is not “AI causes cognitive offloading and should therefore be banned,” but rather “AI companies have shown they are not addressing cognitive offloading.” Soon, I think, it will be “AI companies cannot be trusted to make responsible products and therefore should be regulated.” Social media companies externalized their responsibility and are kind of being held accountable. But the effects of whole generations of cognitively illiterate people is terrifying enough to require regulation. The companies are not doing this.
Yes, we’re in the next rev of The Anxious Generation. In addition to detrimental cognitive offloading, there are also the effects on emotional intelligence that I didn’t touch on here. If you cut your socialization teeth on sycophantic LLMs, you’re likely going to struggle with the complexities of human-human relationships.
I’m not confident about forthcoming regulation given that we’re only starting to see legal action against corporations for social media addiction now. There *are* a number of bills on AI in education, but mostly at the state level and mostly on integration and literacy: https://www.future-ed.org/legislative-tracker-2026-state-ai-in-education-bills/ I worry whatever comes will be too little, too late.
In the meantime, AI system design feels like an area where there are a few degrees of freedom. The prompt bar and current AI output format have parallels to the social media feed and the like button; they need to be more widely seen as such. The former invites unwarranted trust and detrimental offloading, whereas the latter invites addictive engagement.
Putting on my pitch deck hat, these patterns are also lazy design. It’s early days and UX/UI can be rethought to be a differentiator.
TLDR: I’m writing AI system design because it feels like there’s a readily available leverage point here. However, I acknowledge it’s still relatively close to the fulcrum. This is a wicked problem and needs multiple categories of intervention to affect systemic change.
I'm not confident in that the regulatory changes would be comprehensive, standardized, and truly international either. So, in effect, as you say -- "too little, too late."
However, people want controls on AI. This may show up as social norms against AI usage, for example (we already see AI shame).
Yes! Related thought on social norms: another branch of this is rejecting AI altogether, as linked to a larger allergic reaction to screen time and algorithmic feeds (#AnalogLife; genZ's re-valuing of undivided attention, https://fortune.com/2026/04/01/gen-z-analog-economy-5-billion-market-nostalgia/)
Hi! I recently quit my job as a product researcher at Google to work independently at an industry level to address this exact issue. I truly believe these are relatively easy design problems to tackle and companies will address them with enough external pressure, because (some of them) want to be a platform appealing to high school and college students . Given Anthropic has documented evidence from their own internal research that supports what academics found and there is now precedent that the harm can be in the design, they have a lot of reasons to take action on this immediately. At this point not addressing this would be considered negligence on their part. I also believe there is emerging evidence that the current UX of chatbots is leading to harmful results in other domains as well and there need to be a broader reckoning around this. I wrote about a this here https://earlyinsightsclub.substack.com/p/the-ux-of-ai-chatbots-is-risking?r=6ju2pa&utm_medium=ios
"What’s especially notable about these two categories of harm is that they are being experienced by the two earliest adopters of AI: students & tech workers" - yes!
I keep coming back to why this isn't getting built. The 10x cognitive offloading potential of this tech was evident early on. Transcending the chatbot could be a key differentiator. Are alternatives seen as too friction-filled, at risk of negatively impacting DAU? Is the space just moving so fast that "slowing down" to rethink the interface feels, well, too slow? What would need to be true to incentivize better design?
I think there's a big disconnect between the work happening in the labs ( that could have predicted this years ago) and the feature teams building UIs. A few other speculations: 1) companies are primarily focusing on pleasing early adopters (who can keep the hype cycle going), who I think will push back against anything they think is restricting the product (I've seen this when Claude prompts people to go to bed). 2) those same early adopters have essentially created a community around teaching each other the best prompts/workarounds on these issues, so while these UX feels obvious to someone someone who is thinking about 'designing for all', early adopters blamed these failures as "skill issue" on the user 3) AI companies are heavily benefiting from keeping the system undesigned for as long as possible to see how people use it, so it can inform what to build. This is why I think Anthropic prides it self in launching as many features as possible. I don't think they are thinking about designing a cohesive product, just one that pushes capabilities as much as possible.
The design question is 100% on point--and the writing tool you're describing is a genuine improvement over "here's your essay," right? But I keep thinking about how making the boundary visible isn't the same as making it legible. For a student who doesn't yet have the perceptual equipment to know what an "argument" worth articulating looks like, "write your two sentences first" might function as a compliance step rather than a judgment exercise--something to clear before getting to the formatting help. Seems like the scaffold works beautifully for the student who already has partial formation: it names what they need to do, and then gets out of the way. .. but for the student who lacks that formation, I'm betting the two-sentence exercise itself surfaces the gap--which means the tool ends up triaging the formation problem, rather than solving it. That might actually be the strongest design goal: not replacing prior formation, but making visible where it's absent. Which hands the student back to something slower, probably. Thanks for pressing this, Stef--the exchange clarified something for me, too.
The intervention that caught my attention: teaching students to protect "higher-order work" (analysis, evaluation, reflection) while delegating lower-order tasks to AI. The design logic is compelling, but I can't help wondering whether the very distinction between what's lower and what's higher is itself a perception one develops through the struggle you're now asking students to skip. Seems like the ability to recognize "this is the part that requires my judgment" is exactly what early, unglamorous practice builds--which would make the lower/higher line only legible to learners who've already (at least partially) done the work. If so: the intervention reaches farthest for students who already have some prior formation, and least far for those who need it most. I think that's the equity risk buried in the research--not just access to structured tools, but access to the prior formation that makes the structure intelligible. Genuinely useful framing here, Stef.
You've surfaced an assumption baked into most current AI use: that users can become systems thinkers without putting in the reps. The lower/higher distinction is earned through practice. Without that formation, AI widens the gap between students who already have it and those who need it most.
The design question this raises for me: can AI systems pre-bake the delegation structure rather than leaving it to the user? I think so. Consider a writing tool that, instead of generating a full draft on command, first asks the student to articulate their argument in two sentences, then offers to handle citations and formatting. The system is making the boundary visible: this part requires your judgment, that part doesn't. The student who can't yet distinguish the two gets the scaffold; the student who already can moves faster. That's a different design posture than "here's your essay."
The neutrality here is rare and worth noting. Following evidence rather than a predetermined conclusion is harder than it looks when the topic is this charged.
The performance paradox framing is the sharpest thing in the piece. Output up, capability down. But it raises a question the article gestures at without fully landing: is the offloading problem individual, or is it educational? And does something have to go wrong during school years before AI becomes a crutch?
The studies suggest that structured interaction with AI produces good outcomes. But those interventions all assume a baseline capacity for self-regulation and meta-awareness. The ability to notice you're outsourcing. To pause when prompted. To evaluate rather than just consume.
That capacity is supposed to come from school. But educational systems were quietly offloading that same work long before AI arrived. Multiple choice exams train recognition, not recall. Heavily scaffolded assignments remove the productive confusion that precedes real understanding. The system optimized for measurable output and got exactly that — performance that doesn't necessarily indicate capability.
So when AI offers a smoother bypass, it isn't introducing a new vulnerability. It's exploiting one that was already there.
The equity point compounds this. A student with strong foundations uses AI as an accelerant. A student without them gets a convincing simulation of having foundations. Both look fine. The gap widens invisibly, without the feedback loop that would normally correct it.
And the teacher who might set the guardrails the article recommends came out of that same system. Was validated by it. May not experience the gap as a gap at all.
The interventions that work require someone who already thinks clearly about the difference between performance and capability. The system doesn't reliably produce those people. It occasionally produces them despite itself, and then makes their work harder.
AI in education is a solvable problem technically. The problem underneath it is older and considerably less tractable.
Agreed, AI didn't create the tendency to optimize for measurable output over durable learning. It made that existing gap faster and harder to detect. You picked up on a point I ended up removing from this piece for sake of length. It tied back to episode 88: AI is an accelerator and amplifier for whatever system it's attached to (https://sendfull.substack.com/p/ep-88-ai-is-an-accelerant-what-are). That includes widening the gap between students with and without strong foundations. "Better" design alone won't fix the systemic problem, though it can help stop (or slow) the tools from making it worse.