Latest Blogs

EM Open House: Playtest Lab

Posted by Valerie Bourdeau

Judging from the crowds at our open house events, you guys are dying to get a peek behind the scenes and see how your favourite games are made. You know us, we can't deny our fans anything. For those who can't make it to our studio in person, we have prepared little photo tours of the most secret recesses of Eidos-Montreal.

First, we're stopping by the playtest lab to chat with Alexandre Dessaint, Research and Playtest Manager, who graciously agreed to answer our questions.

EM: When did you start the Playtest department?

Alex: The department was created in April 2010, the lab itself opened in June 2010.

Let's talk interior design. What were your main priorities when building the lab?

We sought to create the most comfortable environment possible to put gamers at ease when they come for a playtest session. We wanted it to feel a little like a living room. That's why we built the lounge area with sofas, games and refreshments. We also put nice comfy chairs and quality screens in the playtest room.

Playtest lounge

The Playtest Lounge


Playtest cubicle

A Playtest cubicle

As for the layout, it was really important to create a certain distance between testers and observers. We're there to take notes and help testers if needed, but we want to avoid getting in their bubble and interfering with their gameplay experience.

Playtest room

The Playtest Lab

What feature of the lab are you most proud of?

My favorite feature of the playtest lab is the audio system, which enables the developers standing outside the lab to listen in during sessions. There is nothing better for developers than being able to hear gamers' spontaneous reactions, be they positive or negative.

Headphones

Fancy wireless headphones

Did you have to abandon some features? Do you feel the lab is lacking something?

I think it'd be great to be able to record video and capture screens during sessions for later viewing. An image is worth a thousand words, especially when it illustrates problems with game rules or navigation.

Unfortunately, that's very costly. Also, when I really thought about it, nobody here has time to go through hours of footage to find one little useful clip, like a needle in a haystack.

Playtest observation room

From the outside looking in

My team and I are always present throughout playtest sessions to take notes and identify issues, and the development teams spend a lot of time watching the players in real time.

With the benefit of experience, what improvements would you make to the lab?

I would improve:

* The audio system, to make it even cooler.
* Video recording, as mentioned, maybe on a few screens only.
* It would be nice to be able to record testers' faces as they're playing. Body language and facial expressions are very telling, and it's important for us to understand what players feel at key moments of our games.

Sound console

The audio console

Speaking of the playtesters, how are they selected?

We recruit playtesters using an online form on the Eidos-Montreal site.

All we ask is that gamers answer a few questions to help us understand their preferences and experience. The whole application process takes 5 minutes max!

What qualities are you looking for?

It really depends on the objectives and the type of game we're testing. All kinds of gamers, hardcore or casual, men or women of all ages, are invited to apply.

Are some types of gamers harder to find?

Right now, we have no problem recruiting, but when I look at our database it's clear that we have more boys than girls who have shown an interest in playtests. Adult women are few and far between.

What do you expect from playtesters? Do they have to have a technical background?

Sometimes, people sign up thinking they are applying for a Quality Assurance position. Even when they get here, some people think they need to give us feedback on technical issues or bugs they encounter in the game.

That's not it at all! We want participants to play the game and give us subjective feedback about what they liked and disliked, what they had trouble with, what they found less intuitive, what confused them, etc. The idea is to improve the overall game experience.

Walk us through a playtest session from the tester's point of view.

It's pretty simple:

1. We brief the playtesters on the ground rules and our expectations for the test.
2. The testers play the game. They are encouraged to speak up whenever they see something they like or dislike and to ask questions whenever something isn't clear.
3. At the end of the session, participants and observers chat, either individually or as a group, and share their experience and opinions. We may ask for written feedback using questionnaires, or test the players on certain elements of the game to confirm that they were well understood.

Observers and testers

Playtesters observed

Playtest sessions are always conducted under observation. What is the role of the observers?

For the playtest team, observation is largely passive. We take a lot of notes, but we try our best not to bother the players. Testers are encouraged to interact with us whenever they feel like making a comment or asking a question, but otherwise we feel it's important to give them space to get immersed in the game. Think about it: when they're playing at home, they don't have some dude poking them every five minutes to ask whether they like the game. We don't want to break their concentration.

What do you do with your notes once the testers are gone? Who sees them, what kind of report do you get from the data?

We always have a debrief meeting for the playtest team after the session to discuss important issues. It is important for us to distinguish between:

* Major and minor problems;
* Individual issues (reported by one individual) and generalized issues (reported by all the participants).

Then, all our notes are transcribed, compiled and analyzed to build a report. Playtesters are never mentioned by name; all the data we gather remains confidential.

Reports are used by project managers (producers, associate producers, directors, etc.), art directors, developers, level designers, and sometimes programmers and writers. Even the General Manager reads playtest reports! It's awesome to see how much the development team respects and values feedback from actual gamers.

What's your biggest challenge as Playtest Manager?

I'd say objectivity. I always have to put my own opinions aside and that's really hard, since I'm a gamer myself. I'm there to serve as a conduit for the players' opinions, not my own.

It's also a challenge to sort through the mountain of suggestions and comments we receive to determine which are useful and which aren't that relevant.

Do you have a fun playtest anecdote to share?

In our very first playtest session, we had a major problem with controls: like in many first-person games, you had to hold L3 and push the joystick up to run. But L3 was also used for crouching, which caused serious headaches.

Imagine the players' frustration when they tried to run away from trouble and instead crouched meekly and got their butt kicked. They were tearing their hair out! The dev team jumped on that immediately and remapped running to L1, which fixed the problem once and for all. Crisis averted. 

Alex in action

Alex in action

Thanks Alex for showing off your corner of the studio, and thanks Antoine for the pictures. 

If you live in or around Montreal and would like to take part in future playtest sessions, sign up now!

Comments

Please login to make a comment

Archive

-->