Skip to content

Tag: Diary

Player Feedback – We hear you

Over the course of the past months we shared blog posts about different mechanics, buildings and other parts of the game. You, on the other hand, shared your feedback in the comments, on Facebook, Twitter or other channels.

Additionally, with the help of the User Research Lab, we received feedback from different play-tests where you experienced the game.

The constant flow of feedback is very important to us to improve certain features, the user interface or other aspects of the game.

In this blog post we want to share some of the learnings and changes we applied after going through the feedback.

Every game starts with your colony ships and your settlers unloading materials to bring to the shore line. However, the player doesn’t know how many carriers, workers, builders or resources are left on this ship.

Now the ship is clickable and provides you with an overview of its cargo.

In order to make it clearer to the player we improved the visibility of the gather-able resources when placing the appropriate building.

The Settlers will have more buildings than any other The Settlers game before. Therefore, it is important to us that you not only can read the building when it is placed and constructed but also differentiate the buildings easily in construction menu.

The mini map is crucial part in a build-up strategy game. Displaying all the correct information at the correct time could be challenging. While our first iteration of the mini map was not giving enough information, the team came a long way to improve the mini map, changing the visualization, adding various filters and more. (Please note: The minimap is still not final)

Those are just a few bigger examples of many adjustments we implemented based on your feedback to offer you the best The Settlers experience.

Thanks again for all your comments and valuable feedback. More play-tests will come in the near future. Keep your eyes open and your ears peeled.

As always, feel free to express your opinion in the comments below.

8 Comments

[Diary Studies] UR Lab Interview – Part #2

In part 2 of our interview with User Research Analyst Janine from the UR Lab in Düsseldorf we take a look at the typical day of a tester and how the test results are summarized and presented.

You can find the first part here.

How does a day/session for a tester look like?

All tests and studies vary in duration and focus, but the most common one is a whole day playtest.

We welcome our testers in the morning. However, before we explain the details of the test, they have to sign an NDA (non-disclosure agreement), which is mandatory to take part in a test. By signing the NDA, the testers commit themselves to not revealing anything about the test and its content.

Afterwards, we explain the details of the test and take the testers to our lab. Sometimes there isn’t much introduction needed, and the players simply start playing the game. In other cases, some more explanation about the test is required. We always make sure that our guests understand that we are not testing their gaming experience or knowledge; rather, we are looking for issues in our game. So we always encourage them to be honest and to talk to us if they are unsure about something.

There are usually two observers present during a test. Their task is to watch the players and to look for any issues they encounter.

During full-day playtests, we normally order food and drinks for lunch. This way we are able to take a break and to get to know our testers a little better.

When the second half of the test is done, we often conduct 1-on-1 interviews with each tester,  where we ask questions related to the experience they had with the game.

When the day is over, we usually hand out some small gifts to thank our testers for their participation.

The atmosphere during our tests is generally relaxed and we are always happy to see some new faces!

Are there different kinds of tests? Regarding time played, content, focus, …

Yes, no user test is like the other! We always adapt each test session to the needs of the dev team and what they want to find out.

The duration of a test is dependent on the amount of content the game already has. The focus always differs according to the current state of the project and what the team is still unsure about.

We have several kinds of studies. Typically, we use common playtests, where testers come to our facility to play. Additionally, we also conduct diary studies, during which participants are able to play the game from home over a longer period (usually 2 weeks).

There are other special cases as well. For example, if a project is still in a very early state, we often start testing when there is no game yet. In these instances, we use prototypes or card sorting. Furthermore, we sometimes invite testers for a focus group to discuss various topics like artworks or settings.

How are you summarizing the test results?

Within Ubisoft we have to stick to three specific reporting documents: a high level summary, a detailed appendix, and a presentation from one of the assigned user research analysts. The presentation allows for a detailed explanation and discussion, directly with the development team, about the findings.

This way we can deliver different report formats, each one tailored to specific job families.

The reporting documents are standardised to make sure that dev teams have comparable results, even if they conduct tests with different UR Labs.

How close is your contact with the respective dev team?

Usually we adapt our behaviour to each team’s needs. Each team has a different working philosophy and we always work and change our behaviour according to that.

We are always available  to our dev teams in case of occurring questions or if they would like to schedule another study. Our teams know that early testing saves both time and money. Therefore, after a test, we always encourage the dev teams to do follow-up tests. Frequently teams will send us updated game builds and ask for additional feedback. Opinions of players are very important to them, which is why they contact us regularly.

We also received two additional questions by our German community member IppoSenshu which we forwarded to Janine:

How many UR Labs do you have in total? And when did they start the first playtests?

You can sign up at 16 different Ubisoft UR Labs worldwide, distributed over 10 countries. But we also have some smaller labs which are used differently.
The UR Lab in Düsseldorf was founded about 10 years ago.

Leave a Comment

[Diary Studies] UR Lab Interview – Part #1

With all the buzz regarding the Diary Study going on, we thought it would be a great opportunity to simply get some more information for you about: What is a user research lab? How does it work? What kind of playtests are organized and how is all the gathered data used?

So, we sat down with User Research Analyst Janine of the UR Lab in Düsseldorf to answer some of these questions for you.

What is the UR Lab?

The UR Lab frequently conducts playtests and studies for our projects and is often the first intersection point of players and games. We combine high-technology with common sense in a way that helps dev teams to improve the usability and user experience of their games during all stages of production.

Where are you located?

We are located in the Ubisoft Düsseldorf studio where we offer our dev teams a modern test setup (eye trackers, current gen PCs, etc.) that is built according to current scientific findings. We do our best to welcome our testers and to make them feel comfortable in our lab.

Are there several UR Labs? If yes, do they have different tasks/foci?

Yes, there are several Ubisoft UR Labs worldwide. These teams focus on improving the usability and user experience of our games. Each lab has varying expertise, often depending on the projects they usually work on. Labs stay in close contact with each other to benefit from the others’ study findings and methodologies, and to spread their professional competence.

Do these research studies only happen pre-launch or also post-launch?

We do both. User research and usability testing should start as early as possible to find issues during early stages of development. This also helps dev teams get to know their target group even better while developing the game.

At the same time, user research does not stop after a game is released, as there is always room for improvement. For instance, we also jump in when DLC or other new content, is created to guarantee the same quality as the main game.

Are you also analysing user data from live games or only hosting playtests directly in-house?

The UR Lab also encompasses Game Analytics and we work closely with our data analysts. They analyse data from our live games and support the development teams with their findings. Usually they are responsible for quantitative data, while we provide teams with qualitative insights. Combined, we are able to support our dev teams with the extensive data they currently need to make decisions.

How can I take part in a user research study? Are you only inviting people from the area?

If you are interested in being invited, you need to register at https://playtest.ubisoft.com/bluebyte/enUS . There you can pick the UR Lab that is nearest to you. Keep in mind that you must be able to show up at the studio in person, to participate in a test.

When a user test is scheduled, you will get a preselection mail from us. This will redirect you to a survey that you need to complete, in order to apply for the study. In the survey, we ask you to answer some questions about your gaming habits like how much you play or what your favourite games are.

Following the survey, we will analyse your answers and pick the candidates that best match the target group. Sometimes we are looking for people who spent a lot of time with a specific game, and sometimes we are looking for the opposite. So no worries if you are not a hardcore gamer!

As soon as you’ve been selected, you will get a final invitation mail from us, and we will be happy to welcome you to our studio!

We do not only invite people from the area either. We invite those, who have applied and are willing to come to us. Sometimes we get testers who have travelled for hours to visit us, which is always an honour!

This was the first part of the interview. Stay tuned for next week when we talk about a typical day of a tester and how the results are summarized.

Leave a Comment

[Diary Studies] FAQ – Dates and language requirements

After we announced the Diary Study last week, we not only received very positive feedback from all of you, but also some additional questions.

Not all of them were covered by our initial FAQ, so, to answer these questions and provide you with some additional details, we have set up another Q&A list for you.

If there are any more questions, feel free to leave them in the comments below this article.

I have not yet received any reply to my application. How do I know I was picked?

Due to the number of applicants we’re not able to reply to all individual e-mails. Everyone chosen for this playtest will instead receive a separate invitation e-mail. If you have not received an invited by 25. November, we have not picked you for this playtest.

Will there be additional playtests in the future?

Yes. We plan to have additional playtests with different scopes in the future.

Do I need a permanent internet connection to participate in the diary study?

In order to download and play the game via Uplay PC you will need a constant internet connection.

Can you share the exact date with me so I can plan accordingly?

At the moment the diary study is scheduled to take place from 2. December until 8. December. While the minimum playtime per day is 1 hour, you can freely decide at what time of the day you want to play during.

Can you share some details about the test itself?

While we already had multiple tests via our Ubisoft Research Lab, we also wanted to involve our most loyal fans from our community platform “The Settlers Alliance”. Our goal is to learn more about the game’s initial onboarding experience – how it feels in the beginning – your opinion on the user interface and your interaction with the game world.

Can you share any details about system requirements?

While we cannot share any specific details about the system requirements at this point, we recommend using Windows 10 with the latest updates installed. The game is not compatible with MacOS or Linux systems.

Until what date can I apply for the diary study?

You have until 24. November 11:59pm UTC+1 to apply for the diary study. After this date we will not take any additional applications in consideration.

In which language do I have to write my daily feedback reports?

While the version of the game for the playtest is in English only, you can write your daily reports (diary) in either English or German. If you cannot provide your feedback in either of these two languages, we’re unfortunately unable to consider you for this specific playtest.

8 Comments

[Diary Studies] We want you! Apply now

We want YOU! It is time for our first “Diary Study” for The Settlers.

With our community platform “The Settlers Alliance” we continuously informed you about the development about the game, different game mechanics, introduced various buildings and asked for your opinion.

Now it is time to take it a step further and give the game in to your hands.

The first “Diary Study” is currently scheduled for the end of November, but enough with the introduction. Time for some details.

Diary Studies are a great way for us to get detailed feedback from a core groups of testers.

The first test will run for a period of 7 days and we are currently looking for “The Settlers” fans who are willing to play at least one hour a day, over the duration of the playtest, while providing daily reports and feedback.

This minimum playtime is a requirement to ensure that we receive valuable data about the current state of the game and certain features. You are not limited to one-hour playtime and will be able to test the game as long as you like over the duration of the test. However, the version for the first test will be limited.

How do I apply for the “The Settlers” Diary Study?

For our first diary study we will choose players using an application process. Please keep in mind that the application process is not a competition about who played the most, as we will always look for a variety of dedicated The Settlers players but also newcomers to the game and /or genre.

Requirements:

You must be 18 years old, able to play at least one hour a day plus additional time for reporting for the 7 days duration of the first test. Furthermore, we will only invite players registered on the “The Settlers Alliance” website. We expect that you are a constructive member of the The Settlers Alliance, as we do not tolerate any form of toxic or abusive behavior in our community.

To apply for one of the seats in an upcoming focus playtest, you have to write us an application, following the format below, to the following mail: E-Mail removed – application phase over.

E-Mail must contain (please copy the format below):

Title: Diary Study #1 – Your The Settlers Alliance Name

General information

Nationality / Age / Gender

Your The Settlers Alliance Name:

Your display name from your Uplay account you used to register on Thesettlers-alliance.com
(Simply post a comment under the news to figure out your User Name)

Your current hardware:

CPU, RAM, Videocard, Operating System
DXDiag attached to the email would be optimal

Gametime (approximate playtime):

The Settlers Online playtime in hours:
The Settlers 1 playtime in hours:
The Settlers 2 playtime in hours:
The Settlers 3 playtime in hours:
The Settlers 4 playtime in hours:
The Settlers 5 playtime in hours:
The Settlers 6 playtime in hours:
The Settlers 7 playtime in hours:

What is your favorite The Settlers title and why:

Please describe your “The Settlers” playstyle:

For example: Building the perfect economy, watching everything coming together (slow build up), military focus, etc.

What is your most important feature/aspect about The Settlers:

Tell us your three favorite game genres:

Action; strategy; city building; RPG; adventure; etc.;

What other games do you currently or frequently play?

Game A; Game B;

What does focus playtest mean?

Diary studies are in-depth online play sessions from your own PC where you test & play the development build. Focus test sessions will run for at least a week and it is required that you play at least one hour each day in order to be able to provide us detailed feedback.

I can manage one hour a day for one or two weeks easily, but can I play longer?

While 1 hour is the minimum, you will be able to test the game for as long as you like during the test. Please be aware that the version for the first test will be limited.

What if I get invited but won’t have time during that playtest?

If you won’t be able to join that playtest due to specific circumstances, please let us know. This will give the chance for other testers to join the test.

What kind of feature will I be able to test?

During a playtest, we will test certain features of a current development build of the game. While we might test the whole game loop during certain tests, we will additionally test specific features and game content.

Are the playtests under NDA (Non-Disclosure Agreement)?

As you can imagine, you will see many unreleased details in order to help us improve the game. As the content you will experience is not in a state representative of what the final game will be, we don’t want that any content from the focus playtests is getting shared with the public.

With the NDA, you acknowledge that you won’t share any details about the game or other internal information with the public.

Is there an age requirement?

While we know that many fans across all age groups enjoy The Settlers, you need to be at least 18 + for all playtest sessions under NDA, which includes all current focus playtests.

41 Comments