Q1: How has your position changed in the last 15 years?
Celene: It's interesting. It definitely has changed.
There was only a couple of companies that were doing [chatbots back then]. Users didn't have a lot of expectations. Now you have to design for users' expectations, and they have grown. They're expecting it to be personal [and] integrated. They're expecting the bot to know everything. So it makes the designer's life much harder because you have to think about all of those expectations and try to meet them, and we have the technology to do it now.
It's getting there, but I feel like users' demands are growing at an exponential rate — faster than we can sometimes keep up with the technology. I've heard people comment saying they haven't really found a helpful chatbot. As designers, we have to be much more on the ball and forward-facing in what we know and how we can design for [chatbots].
Q2: Bots vs. Voice — what's the difference?
Celene: Voice is restrictive.
There's a concept called voice-first design, and I feel like that is important to think about just like we did in saying mobile-first for digital. We have to think of voice-first — especially because now users are not only expecting bots to be personal, but to jump channels.
If I call the IVR and then chat online, I want you to know where I came from. Designing for voice-first gives you the ability to bring in those pieces — the visual applications and things like that.
There's not so much a difference, I would say. I mean — there are differences, but think of it like this: voice is the basic, pared-down version, and then the digital stuff can be applied on top of it. You can add a lot of the digital components as you go, which allows you to keep the same flow of the two channels generally [while] having the benefits of both.
Q3: Digital vs. Voice — which is better?
Celene: In my experience, there is no winner. You have to think about things in multimodal applications. There really is power to voice because you can do things in the car. You can do things without looking at them. There's [also] power to digital because you don't need to rely on the processing power of your brain while listening using voice.
You can't really pick a side. You have to use things blended together on multiple devices, multiple screens, multiple channels. That's what I've learned in my 15 years.
Q4: Tell me the first thing you do when tasked with creating a new conversational experience.
Celene: Somebody taught me this once, and I loved the idea.
There's this idea of data, principle, and precedent. Three things. My personal preference is always to go with data first. If we have the data, that's what I want to start digging into because it's going to give me the most applicable design possible.
If we don't have data — which sometimes happens — then we have to go into principle [where we say to ourselves] we know this is what works. This has been tested. And so let's design based on that and later validate with data.
If you don't have [principle], then you can look at precedent or what other people are doing which is not great. Data really is my first collection step because it's going to help me make the best bot possible that's going to reflect the actual deployment that we're in.
Because we have [clients that use] our company that already have chat and voice deployments (with us), we can take actual user utterances from those to collect data. But sometimes we don't have that. Sometimes we have clients that are starting off new that don't have our chat underneath the data. Then you have to look at other things.
We've looked at search logs — [which] still tells you the intent. They still tell you what people are asking about. They just don't know how they're asking it. We've looked at transcripts from all different kinds of sources. We've looked at web analytics data [to show us] where people are going. [For example], going from page one to an escalation page or a contact us page tells me that's where they're getting stuck. So that's where we need to put the bot.
The best data is actual user conversations because I know how they're going to talk, how they're going to ask, and what the intents are. But in lieu of that, there's a lot of other ways you can get data.
Q5: What approach do you take when designing a voice or chatbot persona?
Celene: It's really interesting. You have to experiment to a degree. You have to understand the client's persona. You have to understand where they're coming from, [and] how the rest of [the company's] stuff is written too. You have to match their brand.
But then you can have a little bit of fun with it. We talked a bit about persona, and naming it, and how that can affect the conversation. I really like trying AB tests at the beginning — one welcome message versus another welcome message and then rate the quality of the conversations. That's always a good approach.
It does have to match the tone. If you're making a healthcare bot, it needs to be empathetic and sincere, and trustworthy. If you're making a banking bot, now you're into being a certain way — very matter of fact and maybe even curt sometimes [since] people want their money questions answered.
So we've just learned that through our data collection, you have to try and see how your users do it. You need to try different things and see how users fit with it within the context of the brand.
Q6: What's the hardest part of your workflow currently? How do you overcome it?
Celene: There's always the challenge of [...] the art of compromise. That's the hardest because, as a designer, you want something very passionately. You know it's the best thing. You know it's the best experience. And either your tech team will say, 'nope you can't do it' or the client will say, 'nope you're not getting that API' for whatever reason. So you have to learn how to say, okay, here's what I want to achieve. I know how to best get there. I know that there are limitations on how to get there. So how can I overcome them?
And that's the hardest part of it [along] with the art of negotiation. It might just be the conversations you have to have too. So it's the hardest part, but what I find is the most core to the role [of a] a designer.
I was talking to Cathy Pearl about this too, and she kind of confirmed this [when I] asked her what she would look for when hiring a senior conversation designer.
[The art of compromise] — that's really a skillset you need to have. It's not the design part. It's not teaching how to write prompts effectively. I mean, you have to know that — but it's not the hardest. The hardest thing is getting over that hurdle of figuring out [how to handle] if you can't do something [when] you want to do it.
Q7: How do you assess success or user frustration when looking at the data?
Celene: That's a big thing. I was just on a call this morning about it.
It really depends on what the client's KPI is. So, for example, if their KPI is resolution rate, then all of my metrics go around that. 'How many conversations are we automating and not escalating?'
Maybe it's about satisfaction. Maybe I just want to make sure that users are happy, and I don't care if they escalate or not. Then you're into measuring CSAT.
We have an airline that has crew deployment — and it's interesting what's important for them. For them, it's really about affordability and making sure that they're not wasting five hours on the job talking to an IVR and actually doing their jobs. Everybody has a different KPI. Based on that, you have to form your metrics around the core KPI and how the bots are helping to drive that.
Q8: Talk about the importance of optimization in chatbots and voice.
Celene: This applies to both digital and voice. When you go live with a chatbot, you can't leave it be. That's because — and I use this analogy a lot — when you use these natural language applications, you can think of it like training a human.
You start with a baby. It's new. You have to teach it a lot. You have to handhold a lot and give it a lot of TLC. Eventually, they can start learning a little bit by themselves. You can then hands-off a little, [and] get into more of a monthly or bi-monthly optimization cycle.
If you don't pay attention to it, it tends to turn into an angsty teenager that doesn't listen to you and won't perform and behave like it should. People [then] ignore it and bias against it. So [optimizing for chatbots/voice] is very similar to how we teach humans.
Q9: What does your team look like? What roles are most important when it comes to the success of a project?
Celene: It's funny because it was just me on the original chatbot team/client services team that we had. Before we had professional services, I was the trainer. I was the project manager. I was everything to get it up and running. But then [after] we built out the team that we have today.
We found a lot of iterations of what teams look like. It's tough to say what works best. It kind of depends on the environment of the company.
I would say generally, there are two models that I've used — and both have been good. The one model is you have one person that does everything. They're familiar with project management. They're familiar with design. They're familiar with the models underneath [and] data analysis. So that's one role. And then we always have like a tech role as well. And that tech person does like tier two support or tier two integrations. We call [that person] a design analyst or a tech analyst.
So that's one model we had, which worked well for a while, but it doesn't scale to bigger things. It works really well from a client touchpoint perspective, but to really do things well — to design well, to do the models well — you really need to specialize.
So as [our] company [grew] and you have more demand and maybe higher enterprise clients, that's where we decided to split it, where all of those roles that I mentioned become individual roles. So you have core designers. You have people like data scientists who are doing more of the data and analysis of the utterances. You actually have data analysts that are taking all of the reports and putting them in dashboards. And then you still have the tech [role] separate [as well].
Both models work. It just depends on what stage the company is at. You have to see what works for your clients, what works internally for you, and what you can hire for.
Q10: What roles do you think best transition into conversation designers?
Celene: Right now, we have someone on our team that was a video game developer. We've had people that have been actual digital UX designers. We've had people that were content writers. We've had developers even.
Back in the day, when we were hiring and bringing people in because there was no such thing as a conversation designer, we had people from everywhere. We gave people quizzes and tests to see how they could put together the logic. I always use this example, but [we hired] a tool and die maker for automotive, and he was a really good designer/analyst.
So a lot of times, they can come from all walks of life. You just have to search for how their brain thinks and how they can structure things. With more automation and more bots being created, we're going to need people to power them — and like I said — optimize them.
Q11: Why does empathy matter when designing for chatbots and voice?
Celene: In both cases, you're not going to have a bot that seems intelligent or human if you can't be empathetic to the situation. It's very interesting how you balance that empathy in a bot because bots can't genuinely feel. [However] in order to have a good conversation with the person you're addressing and dealing with their problem, you need to be empathetic.
If the bot can't be empathetic you're not going to get very far with the conversation and it's going to end up escalating to a person anyway.
Interested in joining our next live webinar? Sign up for upcoming events here.
.svg)