Playtesting: Running a Test
Preparing for your playtest
You've committed to playtesting your game, so now it's time to get ready! There's a number of ways you can set yourself up for success before the playtesting even begins.
Your ultimate purpose for playtesting is to gain actionable feedback from your players that you can use to improve your game, but you'll need to narrow this broader purpose down in order to capture the feedback you want. Consider more specifically what you'd like to learn from your playtest.
A few examples of specific questions are:
Do players understand how the controls work?
Do players easily grasp the mechanics without a lot of instruction?
Is the objective of the game unclear or do players wander around looking for what to do next?
Can players navigate the inventory or menus in the game?
Are players getting lost on the map?
The more specific your playtesting goals, the easier it will be to get feedback you can actually use. Broad questions like "Is my game fun?" are important to ask, but harder to measure in a playtest because fun is an intangible concept. Instead, you'll begin to assess if the game is fun through these smaller, more specific questions and test goals in addition to observing your players' body language as they play. It's not fun for a player to be frustrated or lost in a game, for example, so addressing these issues will serve to make it more likely that your game is fun.
A quick word on prototypes
There are two main types of prototypes you can use in a playtest - a coded prototype and a paper prototype. You are likely already familiar with a coded prototype, but a paper prototype is a great tool to test ideas out even earlier than coding them and with less time investment up front.
Paper prototypes are made out of low-fidelity materials such as paper, cardboard, sticky notes and markers to create a version of your game mechanics operated by you as a game master for your players. Paper prototypes are a great option either very early in your development cycle or very early in the development of a new feature. They take a very short time to set up and can help you test a rough draft version of a mechanic before you even have to jump to coding.
When you are looking to begin really refining small detail aspects of your game, a coded prototype is the best option. You'll be able to tweak things slightly enough to gain feedback from players. Details like walking speed, firing speed, controls, layout of the heads up display and clarity of characters and items on screen are all great candidates for testing with a coded prototype.
How many players should you recruit?
There's no hard and fast rule about how many players you should test with, but a good starting place is 5-10 players. If you start on the lower end of this range, you can usually start to see themes throughout the playtest sessions that will give you a clear answer. If the results of the playtest are mixed, then you can always recruit more to see if a theme emerges.
Do a dry run before you test with actual players
The importance of doing a dry run cannot be overstated when it comes to playtesting. A dry run is when you run through your playtest yourself, similar to a dress rehearsal for a play. Doing a dry run of your test lets you feel out any issues with your script, the test flow or your equipment before you're in a test with an actual player. You don't want to waste a player's time or lose out on valuable feedback because you encounter an easily fixable bug in your prototype or a question that doesn't make sense. If your first dry run goes poorly, don't be afraid to do a second or third. It's better to catch any mistakes or issues before you are with a player in a live test session.
Running the Playtest
Set the Scene
The first step to running a successful playtest comes before the player even touches your game. It's crucial to set the scene for them. Explain to your playtester why their feedback is valuable and that they are not the one being tested. Emphasize that the more honest the feedback, the more your game can improve.
You'll want to remind players to think out loud as they're going through the game to explain their thought process to you, and that you're available for hints and help if they get stuck. Lastly, set the expectations for players about how long the playtest will take, and stick to that time frame.
Recording feedback
Ideally you will have two people on the playtest - yourself and a notetaker. It's very difficult to observe body language, non visual cues and remember what players have said about your game while also moderating the playtest. A notetaker can capture any observations or quotes while you are focused on asking the questions.
It's also good practice to record the session, but make sure you ask players for consent first. You can set the expectations with them about why you are recording and who will have access to the recording to ensure they are comfortable with it before proceeding. If you're playtesting over a video conferencing platform like Zoom or Google Meet, it's easy to record sessions through the software. Then, after all of your sessions are completed, you can go back and re-watch all of them with a more critical eye.
During the Playtest
Players generally won't tell you what they're thinking about the game and if it's fun, they're going to show you through their behavior, in-game actions and body language. Observe non-verbal cues to determine if they're interested, frustrated, bored or confused. Watch for if they wander around too much because a section of the game is unclear or keep hitting the wrong button because the controls don't meet their expectations.
When you're watching the playtest, whether live or recorded, you want to look for themes rather than one-off pieces of feedback. If most of your players get stuck on the same section or struggle with the same mechanic, then you know it's something you'll need to update or fix.
Once players have completed the allotted time or section of your game. you'll want to have some questions prepared to close out the playtest. Good questions are open-ended and not leading. You don't want to accidentally influence the feedback that you're getting from players. Listen closely to what players are saying instead of thinking about what's next on your test script or you may miss out on opportunities to ask follow up questions based on what they said.
Let's say you want to find out if the players understood a new quest you just added to your game. Here's how you might approach asking about this quest:
What was the most difficult part about the quest?
What was your favorite part of the quest?
What was the most confusing part of the quest?
How might you change the quest?
Any final thoughts?
Don't forget to ask follow up questions to each of these to get even more information out of the player about why they answered the way they did. Give them time to think through their responses before you follow up.
Finally and most importantly, say thank you for the feedback and do not be defensive. Your players spend their precious time and hopefully eventually their money on your game and they're giving you their objective opinion to help improve the game. Graciously accept all of the feedback you are given by playtesters even if you don't agree with it.
Now it's your turn
Playtesting takes a bit of practice to get right, but you'll be comfortable with it in no time. It can be hard to hear honest feedback on a project that you've poured your passion into, but it will only serve to make your game better. Hopefully this quick two part series is a helpful way to get you started!
