In-Person Qualitative User Research
User Research – September 2019
There had never been any in-person qualitative design research carried out by the design team at Aspiration. I wanted to change that. For this first session, we wanted to learn about our customers’ relationship with banks and observe how people experienced our sign-up funnel.
Contribution: I led the first ever in-person user testing / qualitative research at Aspiration. I recruited and prepped the testers. I prepared the test plan and script. I moderated the test. And I synthesized the data with the help of Randy Jones.
We had prototypes of our funnel on usertesting.com
. But it was clear that users were not experiencing the real thing. In a prototype, users don’t actually have to link an external bank account or transfer their own money. So we were missing out on observing real reactions to these crucial tasks.
Confluence overview of the research project
Here are the messages I sent out to potential test candidates:
Would you be interested in participating in a design research session for Aspiration?"
If they answer [Yes],
"You haven’t signed up yet, correct?
Our main goal is to observe how people experience our sign-up flow. We will have you go through our sign-up flow in person. We’ll also ask you some questions. How does that sound?
It will take about 30 minutes – 1 hour. We’ll provide a $75 Amazon gift card for your time! It should be pretty fun, actually. : )"
Here are our awesome testers
We began by explaining to the testers that they were actually signing up for a real bank account whilst being recorded. We made it clear that if at any point they wanted to stop or not be recorded, that was totally fine.
I asked them a series of questions to understand which of our personas they resembled most. I covered everything from their views on social responsibility, their relationship with money, and their experiences with banks.
List of persona questions
Sign-up usability test structure
The testers' hands and phones were recorded with a GoPro. And their facial reactions were recorded and live-streamed with Google Meet to the whole UX team. Randy Jones and Michael Traquair took notes on the other side of the room.
Face recording + camera setup
I asked them to go to aspiration.com
and sign-up. Throughout the process, if they asked any questions, I would refrain from answering and turn a question back on them. For Example: Tester, “What’s Plaid?” Me, “What do you think Plaid is?” This allowed us to learn more about their thoughts and perceptions. Other times, I would remind them to do what they would do if they were signing up at home.
After the test, we spent a few days making sense of all the information we gathered. We sorted the post-it notes to identify common trends. We had discussions about the participants' answers and questions. And we thought about the largest problems, brainstormed potential solutions, and created recommendations.
Card Sorting the notes from the test session
We learned a lot observing people experiencing our funnel. The three largest problems occurred at email verification, link bank account, and opening deposit.
Problem 1: Email verification is slow and it often ends up in spam. All of the users said they preferred SMS verification.
Recommendation: Provide SMS verification as an alternative to email. It’s faster. It normally works. And there’s no spam folder for the verification code to get lost in. This will likely increase throughput and decrease errors/burden on customer.
Problem 2: Users are caught off guard in the Link Bank account and deposit steps. It was very difficult for them to remember their external bank account credentials.
Recommendation: First, show a short list of the next steps so they can mentally prepare for gathering their credentials and depositing money. Second, if they get multiple errors trying to sign in, let them skip until later. Third, If they skip to the dashboard, give them an option to set up direct deposit from their employer. This would be an alternative to funding their account with a bank transfer.
Problem 3: Opening Deposit is a large hurdle for people who just want to make sure that Aspiration is a legitimate financial institution.
Recommendation: Provide a way for skeptical users to skip this process and see what the dashboard and app are like. The key issue here is trust. Perhaps showing them that the app is real would be an extra nudge to deposit some money. We could even
One observation that really shocked us was their experiences with Big Banks. The four who banked with big banks had really positive experiences. Most of them described their first time opening a bank account as really exciting and fun. They spoke of family tradition, “All my family banks there. They always have.” They talked about the relationships they have with certain tellers. The mentioned how good they feel knowing the bank will be there tomorrow or in five years.
At Aspiration, it’s easy to view big banks with contempt. But listening to customers showed us that we have a lot of work to do if we want to compete. We need a flawless and delightful sign-up and on-boarding experience. We need reliable customer support. We need clear and transparent marketing that informs customers about our products.
Another thing we learned was about our own research process. Asking the participants what they expected to happen next too often, messed up the test. When their expectations were not met, they became confused. When we asked them what they expected a second time, they became even more confused and slightly annoyed that their expectations weren’t met again. Contrast this with normal human interaction that is more open about what comes next. People don’t usually consciously predict what type of screen will come next. Rather they respond to the new information as it is presented to them.
The next steps in the process included creating a deck which outlines the key findings, notable video clips, and other observations from our research. As the wider team ingests the data, they can start to prioritize the work required to solve the problems we uncovered. Those ideas would be built and tested or measured quantitatively.
Thank you Chris, Randy, and Michael for all of your help with this project.