In part 2 of our interview with User Research Analyst Janine from the UR Lab in Düsseldorf we take a look at the typical day of a tester and how the test results are summarized and presented.
You can find the first part here.
How does a day/session for a tester look like?
All tests and studies vary in duration and focus, but the most common one is a whole day playtest.
We welcome our testers in the morning. However, before we explain the details of the test, they have to sign an NDA (non-disclosure agreement), which is mandatory to take part in a test. By signing the NDA, the testers commit themselves to not revealing anything about the test and its content.
Afterwards, we explain the details of the test and take the testers to our lab. Sometimes there isn’t much introduction needed, and the players simply start playing the game. In other cases, some more explanation about the test is required. We always make sure that our guests understand that we are not testing their gaming experience or knowledge; rather, we are looking for issues in our game. So we always encourage them to be honest and to talk to us if they are unsure about something.
There are usually two observers present during a test. Their task is to watch the players and to look for any issues they encounter.
During full-day playtests, we normally order food and drinks for lunch. This way we are able to take a break and to get to know our testers a little better.
When the second half of the test is done, we often conduct 1-on-1 interviews with each tester, where we ask questions related to the experience they had with the game.
When the day is over, we usually hand out some small gifts to thank our testers for their participation.
The atmosphere during our tests is generally relaxed and we are always happy to see some new faces!
Are there different kinds of tests? Regarding time played, content, focus, …
Yes, no user test is like the other! We always adapt each test session to the needs of the dev team and what they want to find out.
The duration of a test is dependent on the amount of content the game already has. The focus always differs according to the current state of the project and what the team is still unsure about.
We have several kinds of studies. Typically, we use common playtests, where testers come to our facility to play. Additionally, we also conduct diary studies, during which participants are able to play the game from home over a longer period (usually 2 weeks).
There are other special cases as well. For example, if a project is still in a very early state, we often start testing when there is no game yet. In these instances, we use prototypes or card sorting. Furthermore, we sometimes invite testers for a focus group to discuss various topics like artworks or settings.
How are you summarizing the test results?
Within Ubisoft we have to stick to three specific reporting documents: a high level summary, a detailed appendix, and a presentation from one of the assigned user research analysts. The presentation allows for a detailed explanation and discussion, directly with the development team, about the findings.
This way we can deliver different report formats, each one tailored to specific job families.
The reporting documents are standardised to make sure that dev teams have comparable results, even if they conduct tests with different UR Labs.
How close is your contact with the respective dev team?
Usually we adapt our behaviour to each team’s needs. Each team has a different working philosophy and we always work and change our behaviour according to that.
We are always available to our dev teams in case of occurring questions or if they would like to schedule another study. Our teams know that early testing saves both time and money. Therefore, after a test, we always encourage the dev teams to do follow-up tests. Frequently teams will send us updated game builds and ask for additional feedback. Opinions of players are very important to them, which is why they contact us regularly.
We also received two additional questions by our German community member IppoSenshu which we forwarded to Janine:
How many UR Labs do you have in total? And when did they start the first playtests?
You can sign up at 16 different Ubisoft UR Labs worldwide, distributed over 10 countries. But we also have some smaller labs which are used differently.
The UR Lab in Düsseldorf was founded about 10 years ago.