17 September 2020
4 mins read
Image of the back of a man talking to another smiling man on a laptop screen. An open book sits to the right of the laptop. Photo credit.
The impact of COVID19 on the working practices of organisations across the globe has been immense, and Difrent is no different (see what we did there?), especially when it comes to User Research.
We’ve entered a time in which conducting purely remote User Research is now paramount to achieve our goals and, even as the country is starting to open back up, this will very much be our new normal for a while.
In light of this, three of our User Researchers (Merissa Brown, Rob Edwards, and Steven Palmer) have put together some of their thoughts on remote user research following their recent experiences. Have a gander.
Somebody’s watching me
Ever act completely differently when you’re being watched? You’re not alone!
This is known as the Halo (or Hawthorne) effect and it’s something a lot of social researchers have to be wary of when speaking to participants.
The effect can be worse when a participant is in a strange lab setting sitting with a researcher they’ve never met before, using tools that aren’t their own.
In lockdown though, all of our participants are usually sitting in their kitchens or living rooms, on their own laptops, drinking a cuppa, even holding their kids.
There’s also a significant distance between them and us, making them more comfortable and more likely to act as they normally would.
They’re also using our prototypes within their own contexts which adds even more validity to our tests.
“Can you see my screen?” and “Can you hear me?” are phrases that are all too common in any online meeting.
In remotely moderated usability testing sessions, not being able to see the participant’s shared screen or hear their thoughts can stop us from gaining key insights.
I was recently taking a participant through the NHS Jobs website we were working on, only for the call to disconnect a total of 3 times before we completely gave up.
Connection issues can also cause people to speak over one another on a call, making focus groups a risky situation.
There’s no real way around this (other than face-to-face) but I find that loosely preparing for an unmoderated user session is helpful. For the user I mentioned previously, I sent them the prototype with a list of detailed instructions and questions. They then emailed me the feedback I needed.
When performing user research, the general aim is to be representative of all users. Whilst these days I find most users have access to a form of connected device they know how to use, and an internet connection, there are still some who rely on public services, such as libraries or community organisations, to access the internet; both in terms of hardware and on-hand expertise.
In a COVID-19 world, I’m not able to perform any kind of face-to-face sessions, and this does mean I’m potentially missing out on some groups of participants. This is something I need to take into account when analysing any gathered data, and when planning future sprints. I’ll need to ensure there is a focus on assisted digital and those users with access needs as soon as face-to-face methods, and workspaces can be safely adapted to include vitally important social distancing measures.
While speaking to users who diversify in their technological abilities where availability is an issue, doing our sessions purely remotely has really helped with accessing users within a larger radius.
Even this week, while sitting in my South London office/bedroom I’ve spoken to users in Newcastle, Plymouth, Exeter, and Liverpool!
If these were face-to-face sessions, the time I would’ve spent on travel would render this range of weekly sessions almost improbable.
Being very personable and really enjoying meeting users face-to-face has really made the lockdown and subsequent restrictions hard for me.
You can get a lot of information from a user’s body language and tone of voice and this can be lost over a video call. Sometimes the cameras are even turned off to save bandwidth.
As a workaround, I have found I need to be a lot more specific in my questions, even to the point of asking the same question from several different angles instead of being able to pick up on a body language.
The COVID-19 world has definitely taught me to be flexible when it comes to the tools used for remote sessions, and always have backups.
Some participants will be using work-issued devices, which will have the same (or sometimes greater) restrictions on them as office devices.
I currently have Zoom, Teams, Chime, Meet and Whereby installed; all essentially do the same thing, but some participants have not been able to use some tools over others.
While remote research has its limitations and it cannot completely replace face-to-face, we cannot deny that it has consistently allowed us to generate valuable outcomes during this global crisis.
While in-person user research one day will become a reality for us again, the lockdown really reinforced the benefits of remote research to us and, when exploring our business practices in the future, we will be considering everything we’ve achieved in the past 6 months.
We’d love to hear from others who’ve been conducting remote user research. Share your experiences with us on Twitter.
For more information about our user research services, get in touch with our team.
Written by Merissa Brown, Rob Edwards, and Steven Palmer – User Researchers @ Difrent
Receive the latest and greatest insights on digital transformation and service delivery, fresh from our newsletter.