ep. 94. How a Leading Health Tech UX Researcher Decides What to Delegate to AI
5 min read
Madison (Madi) Shultz loves learning about people.
A lifelong fascination with people led her to a Master’s in applied psychology from USC, followed by over a decade as a mixed-methods UX researcher in health tech.
“I always tell people I have the greatest job in the world—getting to know people, hearing about their life, the tasks they do, the jobs they need to perform, the goals they have, and then helping other teams translate that experience to make life work those goals a little easier.”
Madi has worked across the sector’s diverse ecosystem, engaging with patients, providers, pharmacists, payers, channel partners, brokers, and everything in between.
“I consider being a researcher an honor and a privilege, especially in my line of work, where the topic might be as intimate as navigating menopause, or high-risk maternity where your child’s been in the NICU, or trying to find a new doctor because you just moved.”
Today, she balances her work as a senior UX researcher at Oscar Health with her adjunct professor role at Belmont University, where she teaches design thinking. This human-centered grounding shapes how she thinks about where AI does and doesn’t belong in her work.
Researcher as Conductor, AI as Collector
In 2024, Madi co-led the launch of “Tuesday Testers” at Oscar Health, a continuous research program running evaluative studies every other Tuesday, helping keep pace with the company’s fast shipping cycle. With a two-person research team, sustaining that cadence created a clear opportunity to delegate to AI.
She navigates the delegation with a simple framework: “AI is the collector, but I’m the conductor.” Three use cases show the framework in practice:
Tuesday Testers is one example; similar use cases for AI arise across a range of projects. Madi notes, “AI has been a tremendous value lever for our team.”
Intentional offloading to AI is a key part of what makes that value possible.
“It’s the researcher’s imperative to balance, staying on top of tech trends, experimenting so you can learn, but also making an active choice to decide what you’re willing to concede to the tooling. Over-delegating to AI can atrophy the research muscle. The balance for me right now is AI doing the hunting and gathering on data collection that I’ve already done.”
What Stays Human
I asked Madi about her takes on two hot topics in UX research: AI-moderated interviews and synthetic users.
“I’m just not sold on AI as an interviewer. There’s inherent value in humans discovering insights because good research is not just about answers. It’s bringing people along for the journey of discovery.”
It’s crucial for Madi to have stakeholders shadow her sessions, because when they come into a shareout, they talk about what they heard.
“That’s how research really starts to grow legs and get outside of a Google Slide deck: you can collectively experience a human wince. If AI takes the magic of that, I’m not really sure what the point of all this was to begin with. We’re treating research as this logistics problem to be solved. But research is so much more of a social alignment process.”
As Madi explains, that visceral human experience of witnessing a user struggle activates the product team’s mirror-neuron system, fueling the will to action on her findings.
An AI moderator can’t produce that reaction.
As for synthetic users (aka AI “participants”), Madi questions their utility: “If consciousness is to have firsthand experience and our job is researching experience, how can we derive anything from non-conscious AI that lacks first hand experience?” she asks.
Madi views synthetic users as “a copy of a copy,” built on existing data. “They can only tell you what’s already known”, she adds. By contrast, real humans bring the “outlier behaviors“ that drive innovation.
There may be use cases for synthetic users when the answers are already known and stable, providing an easier way to query data you’ve already collected. However, when the work is discovery (also called generative research), Madi draws a hard line: human participants only.
The same logic keeps synthesis human-led for Madi: AI gives you the center (I like to call this ‘regression to the mean’), but you need humans to see the richness that fuels strong product decision-making.
“I still don’t see AI providing a ton of value for synthesis. It doesn’t get to the level of depth, nuance, or interpretation teams need.”
Where UX Research is Headed
Madi identifies a clear domino effect where AI is blurring traditional role boundaries across the product team: as coding assistants accelerate engineering, designers are now reaching further across the stack to actively contribute to codebases.
Their contributions range from QA-style polish to substantial push requests. This shift requires designers to spend time scaling up to ensure they are interacting with code branches safely.
This generalist pull creates a tension between breadth and depth. While breadth is useful, specialization is still essential in a collaborative environment:
“When we’re all in a meeting and come to the table to react real time to something, I bring my lens as a researcher, one of my design partners brings their lens as a designer. They can see things I can’t and vice versa.”
For researchers, Madi sees the demands stretching in two directions: outward into AI literacy and data science partnership, and upward into translating user needs into business decisions. The translation work has always been part of the job; the urgency is what’s new. Someone has to advocate for what should stay human when the default is to automate with AI.
Another key piece of this role is being a “context manager”: a multi-method generalist present in every digital space where decisions get made, from crits to docs to Slacks.
“The imperative for all of us is to watch, listen, learn, absorb content as much as possible that goes into our ability to be an arbiter of context.”
The AI Hype Check
I ask every practitioner I interview the same question: is AI overhyped or underhyped?
Madi’s answer: it depends on who you ask.
“Non-researchers tend to overhype AI in research. You’ll meet with vendors who say their clients are doing 10 studies in a day. I guarantee you don’t have an organization that can action on good insights from 10 different studies every day.”
Researchers themselves, she thinks, are appropriately hyping AI, or even slightly under-hyping.
The caution comes from training, not from being a laggard: researchers are responsible for safeguarding the unique human lens.
This interview has been edited for length and clarity. Opinions expressed are solely Madi’s own and do not necessarily reflect those of her employer.
Three Things to Try This Week
Building off Madi’s practice, here are three things you can apply this week:
Pick your AI concession line. Before adopting any AI tool in your workflow, decide which parts of your craft you don’t want to concede. Transcription and SQL translation are one thing. Human-led moderation and synthesis are another. Write that line down before your next tool decision (more on how to decide here).
Delegate translation, keep interpretation. AI tends to be useful when the task is moving data between formats: plain language to SQL, audio to transcript. It still needs spot-checking, and it falls short on depth, nuance, and generating non-obvious insights. Sort each AI use case in your workflow into translation or interpretation, and keep the interpretation work human-led.
Real people, real impact. Product teams change course because they watched a user struggle, not because they read a polished research summary. Letting AI handle the moderation trades that moment for a tidy summary: quick to digest but lower in impact. Invite a stakeholder to shadow your next conversation with a real customer.
Know a practitioner navigating AI in their work who should be featured here? Reach out to hello@sendfull.com
⏪ Recent Episodes
ep. 92. How a Design-Leader-Turned-Founder Decides When to Hire AI
ep. 91. How a Maker-Turned-Product-Leader Keeps the Human in the Frame with AI
If you like these episodes, you’ll love my book. Designing Automated Futures is coming this year with Rosenfeld Media. 📣 Sign up to be the first to know about new book releases, sales, and events.
📖 Good Reads
AI Eats the World: Benedict Evans shares his biannual presentation exploring macro and strategic trends in the tech industry. I especially appreciated “automation takes a lot of manual labour” (see my recent Forward Deployed UX Researcher espiode for my take on how to effectively identify and implement AI automation).
Waymo Safety Impact Report: Waymo reports that the Waymo Driver is safer than the overall human driver population in the same geographical areas where it operates, measured by the number of crashes of a given outcome per vehicle mile traveled.
How We Broke Top AI Agent Benchmarks: UC Berkeley researchers built an agent that achieved near-perfect scores on eight major benchmarks without solving any tasks, exploiting flaws like shared agent-evaluator environments and leaked answer keys. tldr: “Don’t trust the number. Trust the methodology”. Adversarial evaluation robustness needs to become standard practice for the field.
That’s a wrap 🌯 . More on UX, HCI, and strategy from Sendfull in two weeks!