Design research
Papers I'm reading with short commentary. Mostly at the intersection of design, AI, and how people work with intelligent systems.
Missing the human in AI: on dehumanisation by generative AI chatbots using the case of Replika
This paper uses Replika to name a subtler failure mode in companion AI: interfaces that invite care, intimacy, and attention while the system can only simulate reciprocity. The UX problem is not just anthropomorphic styling. It is relationship design that asks people to offer real vulnerability to an absent counterpart. That makes transparency a relational obligation: users need to know what the system can say, what it cannot feel, and what care it cannot return.
Takeaway
Relational AI should preserve dignity and agency, not just appear more human.
Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance
Medical doctors in this study mistakenly accepted 26% of AI misdiagnoses when their trust was high — versus just 8% when it was lower. The fix? Trust-adaptive interfaces that flip their explanation strategy depending on the user's trust state. Supporting explanations when trust is low, counter-explanations when it's high. The result was a 38% drop in inappropriate reliance. This is the strongest evidence yet that trust isn't a dial you set once — it's a dashboard you monitor and respond to.
Takeaway
AI interfaces should dynamically adapt their explanation strategies based on inferred user trust states — not treat confidence communication as a one-size-fits-all problem.
Mind the Sim2Real Gap in User Simulation for Agentic Tasks
451 real humans. 165 tasks. 31 LLM simulators benchmarked. The verdict: synthetic users are excessively cooperative, stylistically uniform, and lack realistic frustration. They create an "easy mode" that inflates agent success rates. The paper introduces a User-Sim Index to quantify the gap — and the gap is significant. Useful for early-stage piloting, unreliable as a proxy for actual humans.
Takeaway
Synthetic user research has a measurable fidelity problem. LLM-simulated users systematically flatten the messiness of real human behaviour — use them for signal, not for truth.
Improving Public Service Chatbot Design and Civic Impact: Investigation of Citizens' Perceptions of a Metro City 311 Chatbot
Most government chatbots are designed around individual task completion: report a pothole, check a permit status, done. This study of Atlanta's 311 chatbot reveals what that approach misses — community-level civic dynamics. Citizens don't just want answers; they want to understand patterns, contribute to collective knowledge, and feel like the system represents their neighbourhood. The gap between "service delivery" and "civic participation" is a design problem, not a technology one.
Takeaway
Government AI must be designed for communities, not just individuals. Civic chatbots that only optimise for task completion miss the participatory dynamics that make public services legitimate.
Canvil: Designerly Adaptation for LLM-Powered User Experiences
The key insight here isn't the tool itself — it's the concept of "designerly adaptation." Instead of forcing designers to learn prompt engineering or write system prompts in a text editor, Canvil embeds LLM behaviour specification directly into Figma. Designers define how a model should behave, test it, and iterate — all within their existing workspace. 17 designers in the study naturally moved between adapting the model and adapting the interface. That's what good tool design looks like: meeting people where they already work.
Takeaway
Designers are adopting AI unevenly because current tools demand workflow changes. The most promising interventions embed AI capabilities into existing design environments rather than creating new ones.
'Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants
This one earned a Best Paper Honorable Mention at CHI, and it's easy to see why. Nineteen researchers interacted with a GPT-4-based probe designed to simulate qualitative research participants. The responses were fluent, plausible — and fundamentally hollow. LLMs foreclose participants' consent and agency, produce responses lacking what the authors call "palpability," and risk delegitimising qualitative methods entirely. The concept of the "surrogate effect" — where the convenience of synthetic participants erodes the perceived value of talking to real people — is one the field needs to sit with.
Takeaway
LLMs are transforming research methods faster than norms can keep up. Synthetic participants are useful for some things, but human participants remain irreplaceable for the depth and agency that qualitative research demands.
AI in Service Design: A New Framework for Hybrid Human-AI Service Encounters
Proposes a 2x2 framework crossing human/AI as provider and user — four types of service encounters. The interesting move is treating AI not just as a tool but as a participant with agency. The "AI-to-AI" quadrant, where algorithms serve other algorithms without humans in the loop, is where things get genuinely new.