When talking with potential customers about Design Sprints I always spend some time explaining how to properly conduct the interviews on Day 5. I still encounter quite a few entrepreneurs and proprietors who are not familiar with this style of user research and confuse it with focus groups or usability studies. Without proper context on how to prepare and conduct these interviews, the outcomes are flawed and not reliable. In an effort to help you dive into your first interviews, I’ve compiled a list of common mistakes and pitfalls.
Poor Recruiting
Recruiting 5 users in 2–3 days is often a challenge. First, be super careful that you are talking to the right people; take the time to prepare a proper screener. Ask yourself what fundamental research questions you need to answer, then work backwards to determine who you need to engage. Otherwise, you risk tainting the experiment by talking to people, then challenging the recruit. Be explicit up front.
Consider the following:
- Are you looking for a new or existing user?
- Are they people who fit your Sprint target?
- Whom should we exclude?
Ask yourself what fundamental research questions you need to answer, then work backwards to determine who you need to engage.
Insufficient Interview Preparation
Build an interview plan. Start with a reminders section of important logistical things that are often easy to forget when in the heat of the moment, i.e. clean up desktop, turn off notifications, turn on caffeine app, etc.
The most neglected part of an interview is the ice-breaker. The next section of your plan should include your chit chat and warm-up questions. Think of your interviews as a flow or story arc. How will you transition and slowly guide the conversation into the topic that is at hand, without them realizing it? It is often helpful to review the recruiting screener answers for specific details about each user. Using these user specific details can help you build rapport and put them at ease.
After the warm-up questions, include a section per prototype/site you are testing. These should include your Sprint questions and any additional questions you now have, after building the prototype. Remember to challenge your assumptions.
Not all the links in the prototype will work. Spend time to familiarize yourself with the prototype so that you can assist this user when they get confused or lost. This will ensure that the interview keeps moving fluidly. Sometimes I will mock interview one of the other Sprint participants or my wife so that the first interview of the day runs without a hitch.
If you are conducting an in person interview, remember the breath mints!
Not building Rapport
Building rapport is important. When your user arrives for the session, pay close attention to your tone and body language. Smile and do whatever you can to make sure they are comfortable and feel taken care of. It is tempting to jump straight into hard questions, but if you do, you’ll come across as a lawyer and your user won’t likely open up to you. If you are conducting an in-person interview, remember the breath mints!
Pitching Your Ideas
Founders are habituated to pitching their ideas. They are pitching to convince investors to fund them, employees to join them, other companies to partner with them, and customers to buy their product. This makes it really hard for founders to switch to a listening mode, as they are in a different mode. They are pushing, not pulling and aren’t truly open to the candidates feedback. It is critical that the interviewer not present or show things, but instead simply setup the environment and begin to observe and listen.
Taking Your Own Notes
When taking your own notes, you are distracted and will miss much of the nuance. With a dedicated notetaker, you can concentrate on subtle cues from the user and dig deeper to extract more insights from them. Likewise, your notetaker will take better notes, as they can concentrate on simply capturing all the details, without having to concern themselves with the conversation.
Don’t reject feedback simply because you don’t hear what you want to hear.
Confirmation Bias
Most people are afraid of being wrong. We are taught from a young age to always know the correct answer. Because it is uncomfortable to be wrong, our brains skillfully steer us away from discovering we are wrong. As a result, it is tempting to discount or explain away evidence that doesn’t match our assumptions. Don’t reject feedback simply because you don’t hear what you want to hear.
A simple way to avoid confirmation bias is to craft your interview questions such that they attempt to disprove your assumptions. Ask yourself, what are you afraid of hearing? Keeping referring back to Sprint questions. If you truly got pessimistic, then those questions should guide you away from confirmation bias.
Stakeholders Not Watching in Real-Time
When stakeholders don’t watch the interviews in real-time, it is impossible for them to empathize with the user. If they rely on a summary from the team or are simply reviewing a previously recorded session, they will inevitably rely on their prior assumptions and will be skeptical of anything that doesn’t align with their current opinions.
Rambunctious Observation Room
Interviews are often exciting, as they can lead to both cheerful and heartbroken moments. Similar to a sporting match, it is easy to fall into a mode of cheering and booing. When this happens it is extremely difficult for anyone in the room to take objective notes. These moments are usually followed by more nuanced reactions or comments which are usually lost in the chaos. If your team, like many, are susceptible to this behavior, consider having everyone use individual headphones. This reduces the temptation to speak out and when someone does, they are less distracting to the others in the room.
Jumping to Conclusions
It is important to remain objective throughout the entire day. Don’t extract meaning from what one individual user tells you. Instead, wait for the patterns that emerge after interviewing all 5 of the users. Also, remember that at the end of the day, you are susceptible to recency bias. Your memories of the last interview are more vivid than the first. Provide ample time to digest all of the interviews and adequately synthesize the learnings. I advise my clients to meet the following week to discuss their insights and key takeaways.
