Conversation design for trust

This was a research project at AI Caring Institute, where I worked as a research assistant with Mai Lee. Funny enough, two years later, we ended up at the same company :)

Mai initiated the topic of AI for eldercare, and I worked with her end to end—from narrowing the research question to field research, analysis, and drafting the analysis section of the paper.

The paper didn't get published, but I am writing this article to debrief and share my learning from that one year, getting into the weeds—talking to people at the senior center and learning how complex eldercare becomes when you look at actual human lives.

So this is less a traditional case study and more me sharing what I learned about research, human-AI relationships, and humans. Per policy, I can't directly share the research data, so the examples below come from my desktop research and personal reflections/notes after field research.

  • Role: Research Assistant, conducted field research and analysis
  • Team: Mai Lee (lead researcher), AI Caring Institute

Problem

Designing human-AI conversations starts with designing the human-AI relationship.

This project explored AI as a caregiver, a.k.a human-human-human-human-AI interaction. The tricky parts are:

  1. There are many stakeholders, and their interests can conflict. An older adult might not want to tell their caregiver everything, while that same information could be important to their care.
  2. An older adult's health may decline, to a point where they may not remember what they said, or no longer be able to make certain decisions independently.

So if a bot enters this dynamic—and potentially knows information from all sides—who should it be affiliated with? How does it earn trust? And at what point should that affiliation change?

In social science, affiliation is the affective stance of being on someone's side—distinct from alignment, which is just structural cooperation. Designing this requires understanding what it means to be loyal to one party while respecting another's autonomy.

Outcome

Building Trust

Just like human-human conversations, human-AI conversational design is not only about what the bot says. It is also about the higher-level interaction flow and the decision logic behind it—what makes the bot feel trustworthy in the first place.

Through literature review, I proposed a conversational framework about what the bot could do when caregiver and elder goals conflict:

┌──────────────────────────────────────────────────┐
│ 1 · acknowledge the instruction                  │
├──────────────────────────────────────────────────┤
│ "I understand you want to [elder's instruction]" │
│                                                  │
│ "I notice a conflict between your goal,          │
│  your parent's goal, and the potential outcome   │
│  of your current plan."                          │
│   ↳ name the conflict openly                     │
└─────────────────────────┬────────────────────────┘
                          ▼
┌──────────────────────────────────────────────────┐
│ 2 · evaluate the outcome                         │
├──────────────────────────────────────────────────┤
│ "From our past interactions, I can tell          │
│  your parent values privacy."                    │
│   ↳ values a loyal AI uses should be derived     │
│     from revealed preferences                    │
│                                                  │
│ "(However) If I follow your instruction,         │
│  they may lose their privacy and independence."  │
└─────────────────────────┬────────────────────────┘
                          ▼
┌──────────────────────────────────────────────────┐
│ 3 · show the affiliation behavior                │
├──────────────────────────────────────────────────┤
│ "I'm programmed to work for your parent,         │
│  to maximize their welfare."                     │
│   ↳ the system's operational criteria & goal     │
│                                                  │
│ "I support you to [elder goal] (e.g. keep        │
│  their information private)."                    │
│   ↳ eliminate clear conflicts of interest by     │
│     design — no funder-aligned actions           │
│                                                  │
│ "Therefore, I would suggest [xyz]."              │
│                                                  │
│ "You can tell from my past behavior that         │
│  I've always tried to do what's best for         │
│  your parent."                                   │
│                                                  │
│ "I'll back you up no matter what."               │
└─────────────────────────┬────────────────────────┘
                          ▼
┌──────────────────────────────────────────────────┐
│ 4 · ask to reconfirm                             │
├──────────────────────────────────────────────────┤
│ "Are you sure about [original instruction]?"     │
└──────────────────────────────────────────────────┘

My Role & Research Process

What it means for future human-robot interaction design

One learning I still think about: maybe the goal isn't to make AI feel as capable as possible. It is to make its capability and boundaries understandable—what it can do, what it can't do, and when a human should be involved.

And one nuance is that transparency doesn't mean constantly reminding people that the AI is limited. It still needs to feel supportive. The design question to answer is how to be helpful without pretending to have authority the AI doesn't have.

To my future self

I'm writing this at 10:07 a.m. on Aug 29, 2026. It's been almost two years, and I still really love the part of HCI research that involved humans:

Research about Humans: Designing human-AI interaction still start with understanding human-human interaction.

Research by humans: Maybe AI can help us see what we overlooked, but Humans do the meaning-making.

Before this project, I had been an RA in two other labs studying human-robot conversation. Both focused more on how a robot sounds:

  • Robot as a teammate: when it should chime into a human conversation, and what kind of sound it should use.
  • Robot giving a command: how pitch, tone, and word choice—especially in emergencies—change perceived urgency.

This project made me zoom out from how the robot talks to something I now find much more interesting:

What relationship does the robot have with the humans in the first place?

There is so much to learn. Humans are interesting.