After running many regional UX Research projects, we have synthesised our main learnings and current process to show you how we work and help you run your research projects when you don’t speak the same language as your target group.
At Objective Experience, we work with companies that want to grow their knowledge across regional markets. Many of the markets in which we have worked such as Indonesia, Thailand, China or Mexico do not speak English natively. This might complicate the data collection and the analysis as usually the stakeholders and the team members do not speak the same language as the participants and do not have a ground knowledge of the local market.
For this kind of research, we usually work with a local team to do the recruitment of participants and a simultaneous translator (source language to English). At Objective Experience, we are lucky enough that our team can speak many languages and we can usually do the moderation. Otherwise, you will have to hire a local moderator. We advise not to moderate in English and use a translator to communicate with your participant because participants will get tired faster, you will collect poorer data and there is a risk that some things might get lost in translation.
Quick tips when working with local moderators:
- Share the study objectives and the moderation guide with your moderator. Alignment is key to ensure that the right data will be collected. Furthermore, your moderator can offer their local knowledge to improve it.
- Talk to your moderato about the technical set-up beforehand. Explain all the different tools, how are we using them together and the most frequent problems and solutions.
- Schedule a dry-run with a real participant and simultaneous translator. Use this dry-run to improve your moderation guide, your moderator’s style and to ensure that the translator understands the interview content.
Remote testing is already challenging and adding more parties involved in the project will also add complexity if we are not preparing it well. In this article we explain how we usually conduct regular remote testing. With this article, we hope to give you some tips on how to conduct remote testing when your participants speak a foreign language.
The testing process
In the following section, we will explain what to do before, during, and after the testing to ensure quality data, the session’s smoothness and stakeholder satisfaction. With this set-up, there will be two virtual rooms; the participant room and the stakeholder room. In the participant room (we use Lookback), the participant will interact with the moderator. The translator will also be there observing and listening.
Simultaneously, in the stakeholder room (we use Zoom), your translator will be translating the session and sharing the video so stakeholders can observe and listen in English. The notetaker will also join this room to take notes, record the session and be the communication bridge between the moderator and the stakeholders.
Before the testing days
- Aside from preparing the testing guide, you will also need to prepare the following:
- For participants: provide instructions on how to join the platform for testing. We use Lookback but there are many other available testing platforms out there. Provide a document with a step by step guide on how to join the testing session. If you can, provide a video that illustrates all the steps in their own language.
- For the translator: provide instructions on how to join the participant room (Lookback) and the Stakeholder room (Zoom).
- If you have to work with a local moderator, train them to use the testing platform and to familiarise with the testing material and moderation style
- If you are working with the local recruitment team, be sure that they know how to use the platform and what to remind the participants.
- Technical test with participants. Involve your local recruitment team to make sure that all participants meet the minimum requirements to have a smooth session where they can communicate effectively with the moderator. The technical test is like a short mockup user session that will try to recreate the same conditions in which the real user session will happen. The objective is to let the participants familiarise themselves with the testing platform and to verify that the conditions are optimal, otherwise your local recruitment team has to find a replacement. Things to check during the technical test:
- Participants comprehension in using the platform and the nature of the testing
- Internet speed and stability. Try to avoid phone data if you are testing a phone prototype.
- Audio quality and conducive environment. We ask participants to use a headphone headset and be in a quiet space to avoid distractions.
- Troubleshooting any software problems.
During the testing days
- Even though we have tried to minimise the possibility of participant drop-offs during the testing session, it is still possible that you’ll need a replacement. When this happens it is better to have some buffer time in between sessions so you can accommodate a possible replacement. Otherwise, you might end up having to extend the testing days or being exhausted after running too many sessions in a day.
- Give feedback to your moderator and translator after the first sessions. Ensure that the moderation is following the moderation guide and that the translation is accurate.
- Keep an open communication channel.Create a channel where the moderator, the notetaker, the recruiter and the translator can communicate easily. This channel will help troubleshoot technical issues and will ensure everyone is up to date with new development.
After the testing
- End of the day debriefs.Discuss the testing session with your clients or stakeholder, you might feel fine with the sessions but your client might feel different. Use these debriefs to seek alignment and bring clarity when necessary.
- Notetaker – moderator sync up.If the notetaker senses that some parts of the session were lost in translation or local knowledge is needed to understand the notes, coordinate with the moderator to fill in the knowledge gaps.
These are some of our tips, we hope you find them handy to help you feel confident running your multilingual remote usability testing sessions. Please, share with us your experiences and any tips you want to share.