“A modern IVR experience made sure that customers across any conversational channel would be interacting with one The Home Depot - not 7 different versions of our brand”
- Molly Dickinson, Senior User Experience Designer at The Home Depot
Yet, the testing phase of the build was proving to be tedious.
Conducting a la carte user tests was not only a massive waste of time and resources - it could never satisfy the data points needed to call testing complete or high-quality.
Finding the right solution
With the goal of scaling guided and unguided user tests of their smart IVR system, the conversation design team used Voiceflow in order to automate wide-scale and repeatable IVR tests.
Working with three groups of 50 users each, the CxD team was able to send high-fidelity test experiences using shareable prototype links to each test group and track how each user interacted with the IVR.
“We wanted to understand the customer journey during a call, this type of testing is essential for distilling if an experience helps to adequately solve a customer problem.”
With this new data, the team could find answers to key questions, including:
- Is there a specific flow that causes confusion for a majority of users?
- What percentage of users required a live agent's help?
- What is the average call time?
- Are there any missing intents that callers repeatedly ask for?
The first week of deploying repeatable IVR tests
The team was able to scale user testing from 12 to 300. With no drop-off in test quality, this sharp increase in user input and data ensured the conversation design team could proportionately improve the IVR caller experience.
Better yet, when compared to the previous one-off method of user testing, the team at The Home Depot has reduced time spent by 50% with the automated and scaled testing environment.
Instead of the team’s time being directed towards sending out individual user tests, they can now focus on analyzing user feedback and designing a better IVR experience.
Looking ahead
The innovative team plans to scale this type of unmoderated user testing.
The valuable data can be seamlessly gathered and collected using a single source of truth for the team, which will allow them to make informed and real-time improvements to the dynamic IVR experience across in-store and call-center touchpoints.
As all user testing information is logged for review and analysis, Molly and the rest of the team can collaborate in-app on a variety of levels, including intent confidence score, sentiment, any missing utterances/intents, and more - all with the goal fully understanding and improving the IVR experience before hand-off to the development team.
This smart user testing gives Molly and the team confidence in their design decisions.
“From design to hand-off, Voiceflow gives our team a voice on every channel and its results.”
.svg)