User interviews are dead: why you should use Shadowing

I’m guilty of the clickbait title, but don’t stop here—because the content is worth your time! In this post, I will explain why user interviews aren’t always enough, especially when you’re working in B2B, and how the Shadowing method can really make a difference in your research.

Starting from the basis: what’s Shadowing?

Shadowing is a behavioural user research technique that involves directly observing users in their natural environment while they are performing their daily tasks.

The researcher silently follows and watches the user, taking notes on their behaviour, challenges, and workflow without intervening.

User interviews, instead, are an attitudinal technique based on what people say about themselves. As every researcher knows, what people do and what they say they do are often quite different, both because human memory is imperfect and because people suffer from social desirability biases.

What are the advantages of Shadowing?

As said before, human memory isn’t perfect, and people are influenced by many biases.

Shadowing puts people in their environment performing common actions, which helps them to act as they always have and remember things.

But be careful, people know that they are observed, so they still tend to modify their behaviour (the Hawthorne effect). Observations made during Shadowing are usually more truthful than self-reported observations, but you should still use your judgment and be conscious of human biases.

Other advantages of Shadowing are:

• You can see the spaces and how the users interact with them. This allows you to collect inestimable insights for your design phase.
• You will be conscious of the time needed for actions.
• You can map actors and touchpoints that the users interact with.
• You can understand the technical level of your users.
• You can observe when they are more frustrated.
• Some users, especially those who are very technical, aren’t good at explaining their work. Observing it is easier.
• You built a common knowledge that can help you communicate with your users and stakeholders in the future.

Plus, if you are designing software for an instrument or a machine, observing how it works and how users interact with it is essential. Demos are done in a protected environment and show only the happy path. But you aren’t designing for the happy path, you are designing for your users’ daily experience.

Remote Shadowing

As said before, seeing spaces is an essential part of Shadowing. Therefore, in-person sessions are preferable.

Going in person is not always possible, for example, due to budget limits or for users spread around the globe. But if you are inquiring about a standard desk job, you can still have a remote Shadowing session.

Ask your participants to share their desktops and show you how they work. This way, you can still see how they do their job and the tools they are using.

It is handy when planning to redesign a company tool or process. Often, you discover many shortcuts and misuses of software that users do to bypass software limitations or malfunctions. These are all inestimable insights for the software redesign. You can also have a comprehensive overview of other tools that can integrate into your future solutions.

Three real-world cases

Now we'll walk through three different use cases where we applied the Shadowing method to gain deeper insights and deliver more effective solutions. We were able to uncover hidden challenges and fine-tune our approach to meet their needs more accurately.

Safeguarding power users’ skills

One of our clients asked us to create a custom application to support one of their internal processes. The main tasks were performed by employees using Excel, leading to a lack of control for the management and difficulties in identifying the source of truth for the data shared among stakeholders involved. Additionally, each department had its own approach, making it impossible to aggregate data and complicating the movement of personnel between departments.

From previous inquiries, we knew that users weren’t happy to leave Excel, but we didn’t know if it was only resistance to change. Thanks to Shadowing sessions, we observed how they used Excel quickly and how many features they used to accelerate their workflow.

We realized that a custom tool couldn’t provide the same efficiency for power users keen on their tool. For this reason, we decided not to substitute Excel but to work with them to reach a common way to manage the process with a single Excel template.

Later on, we built an assistant software with the goal of meeting the management’s needs:

• Check the Excel quality and consistency between different departments;
• Managing the Excel creation and approval process;
• Merge of different Excel files to obtain aggregated data;
• Automation of tedious tasks that can be done more easily by the software.

Understanding a complex tools map

We needed to create a complete map of our client's tools with the goal of reducing their number and streamlining the process. They had dozens of different tools used by various people.

Firstly, we organized workshop sessions with the employees to map all the tools. Unfortunately, for each employee we talked with, new tools came up, other tools were considered dismissed, and some tools were mentioned, but none knew what they were used for. Even putting all of them in the same room, they couldn’t produce a complete map of their toolkit.

So, we decided to conduct Shadowing sessions to understand their work better.

Finally, all those names on a whiteboard started to mean something. While performing their tasks, employees remembered all the tools they used (because they needed them to complete the tasks!). Plus, we discovered that to perform some steps, they asked for help from other employees who, until that moment, weren’t involved in the research, nor were they even mentioned.

After talking with the new actors, we finally completed our map with the remaining tools and understood what they are used for.

Interacting with an instrument in the laboratory spaces

During a project where we designed the control software of an instrument to perform cell therapy experiments, we conducted Shadowing sessions in research laboratories.

The real power of Shadowing is expressed when your users work in a place totally different from an office. This experience allowed us to collect a lot of invaluable insights that guided our design phase:

• We observed how biologists used a color code to improve their efficiency;
• We discovered that some tasks should be executed far from the instrument, so the biologists should print the needed instructions;
• We noticed that the parts that should be loaded in the instrument were really similar and easy to load in the wrong way;
• We discovered that the biologists didn’t like the way they were supposed to write the experiments for the instrument, so they previously wrote everything using different tools and then translated it into instrument instructions.

The last finding, in particular, was our compass throughout the project when we tried to recreate the biologists’ preferred experience and automatically translate their user-friendly instructions into the format needed by the instrument.

Want to know more about this use case? Read the case study.

5 signs you need to watch instead of ask

As I said before, the title is clickbait: user interviews aren’t dead, they are the best way to understand what your users feel and think. All user research methodologies have pros and cons, and it is important to choose the right one in each context.

So, when should you do a Shadowing?

1. When you need to map a complex workflow that is difficult to remember and describe;
2. When you are working on a product that is used in a strict relationship with spaces and/or other tools.
3. When your users are expert professionals who have behaviour and skills so automatic they don't consciously recognize them;
4. When your interviewed users aren’t able to describe their process or give you uncoherent descriptions;
5. When there is a strong risk of bias in your users, for example, if they are your client's employees, and they should talk about their work.

However, watching people is not enough to really understand what they’re feeling and thinking, so you can always reserve some time at the end of a Shadowing session to interview your participants. This allows you to ask questions about what you observed and permits them to add information. You should always remember to listen to your users after observing them!

To conclude

Shadowing, when possible, is always our preferred method of investigating and understanding a new domain. Every time we conduct a Shadowing session, we find insights that are difficult to obtain from a user interview. We challenge you to organize a Shadowing session and to take note of all the insights you wouldn’t have collected with a regular user interview. This will let you feel the power of this technique.