As designers we have strong intuitions and it gets us far, but nonetheless involving users in making products and services takes it to another level. It is interesting to see how they interact with your offering.
How do you do this while physical(social) distancing?
Recently, I had a chance to conduct a Usability Study for one of our clients. Usually, we would invite the participants to our usability lab, but this was a special one, as for the first time we were doing it remotely. Both has their own pros and cons:
- Habitat: In a remote setup the user happens to stay in their natural habitat, which makes them feel more comfortable. On the other hand, a controlled environment, might effect the way they react and hence, the outcomes of the tests.
- Geography as demographics: This parameter would get conveniently eliminated while choosing the participants. Now we can even go international.
- Availability of participants: Now, being at home people are more accommodative of these kind of requests and are willing to participate in the hope of a new normal.
- Note Taking: Multiple participants can view the test at the same time and bring along different perspectives. Now we have to rely on the user’s setup. Hence, the recordings are centralized, collected in one place and pre-digitized instead of fragmented notes from different sources. We used lookback to facilitate our study, which also helps you in time-stamping the observations. Amazing feature, makes it so easy to go back to the videos and also measure the duration of a task.
- Distractions: While the setup is at home, be it participant or the moderator, the whistle of the cooker from the kitchen or kids playing and talking becomes our icebreaking conversation. While you are in a lab you can request the participant to mute their phones but in these set ups when the world is trying to reach you digitally, you cannot really have a say. We had situations where the participant put us on hold and received the call, declined the call and sent a quick message, as they were sharing their screens, we could also read their notifications and messages.
- Consent: This step of explicitly asking for permission becomes optional as the user will now have to give relevant app permissions so that moderator could see their screen and be on a video call with them.
The current restrictions have caused major disruption to most businesses but it nowhere means that we cannot continue to improve the experience we create for our products and services. Modern tools mean we can still do most design activities, from workshops to collaborative prototype building from dispersed locations.
The Preparation
- Build a prototype:
We have already defined the journeys for the user and stitched them into a prototype. Any tool that you might be using would have some limitation. Understand them and build accordingly. - Determine the tasks:
Create a dump of unknowns. Identify the purpose of meeting them. Run a mock conversation in your head and define the tasks. Run it past the team to iron the kinks out. Be organic, be transparent and be interested. We don’t really need to script it all. - Define the UX Metrics:
Define when would you call it a success or failure. Be open to feedback. This is likely to be more subjective and dependent on the research team’s intelligence. Observing gaps in usability is definitely a thing, but apart from that one can also measure duration of tasks and compare it to an expected time frame. No. of steps, Starting Point, Validation on Categorization, Navigation, etc. could be other parameters to observe. - Schedule the conversation:
Schedule a date and time for the users. Make sure you have time for setting up and a little cushion for unpredicted situations too. Never end up doing it all alone. Ask people to join in, observe and take notes. This would give you room to keep yourself engaged in the conversation only. Ensure there is enough time gap between two slots (around 30mins) to catch up on the thoughts from the earlier one while they are still fresh. Also, it becomes too tiring to keep talking without breaks. You could motivate your team members to conduct and can keep taking turns. Keep yourself and encourage the participants to stay hydrated too. - Tools required:
An ideal session should be done in pairs, where one researcher does the talking with the user, while the other takes notes of all the observations made. Recording the interview always comes handy. Pick a tool which helps you add timestamps while recording. So you can go back & listen to on any missed pointers. Conducting mock sessions is a good way to get to know the set up well.
The Setup
We used a tricky set up though.
- The moderator used lookback dashboard on web and the participant used the lookback participate app on their phones.
- We called the participant through lookback which enables screen share and camera.
- The moderator was also connected to the stakeholders through Microsoft Teams and was also sharing the lookback dashboard so they can see what is happening.
- As the call progresses between moderator and participant through lookback, the stakeholders connected through MS Teams barely are able to listen to the participant. Now to solve this, we connected a speaker to moderator’s laptop, so the conversation is amplified and stakeholders can also passively listen to the same.
- The moderator can have his team as observers on lookback dashboard and they need to drop off from MS Teams while this call is happening. This is to save their eardrums from the echo that is caused because of connecting to two calls at the same time.
- While all this, the research team was using lookback to record the conversation, take notes, timestamps and chat. The stakeholders also had their share of recording through MS Teams. All of us had our perspectives, we shared thoughts and brought our app to life.
The Execution
- Set the context right
- Clearly state the expectations
- Have handy instructions for them
- Have a list of tasks with you
- Ask them to Think Out Loud
- Observe and Empathize
- Always have backup plans ready
A few failure lessons for caution
- Prototype not running on low end devices
- Different Screen Sizes might make your prototype non usable
- Night Mode On/ Off might affect the prototype
- Camera/ Mic/ Speaker Not Working
- Internet Connectivity Issue
- App Permissions Declined
- Multiple App Usage at the same time
- Wearing a mask might cover the expressions
- Voice Amplification or echoes while using multiple devices to moderate and collaborate at the same time.
Hope this helps you in conducting yours. Good Luck!!
