Moar Playtesting Lessons

 Moar Playtesting Lessons

Fight the Grind; Distance is a Good Thing!

I have enough experience with writing and directing to know that sometimes, distance from a project is a good thing. I think "The Grind" is a destructive and self-sabotaging thing in the vast majority of cases. When you lift weights, you need to take time away to let your body heal so your workouts can continue to be productive. When you cook meat, sometimes, you need to let it "rest," or you can ruin the flavor. Despite the nonsensical "wake up and grind" attitude America tries to sell, there are all sorts of instances in which grinding away at a problem or task creates even more problems or tasks.

"Don't fall for it. She lives two trailers down."

In the wake of Stonemaier Design Day and quite a lot of feedback/data from our very first five "official" playtests of our prototype, I took time off from actively thinking about it. I'd jot down ideas if they came to me, but otherwise, I steered clear. I didn't intend to be away from it for two months, but that's just kinda how it turned out. And it's not like Chad and I are on a deadline, so why not? Take the time, come back fresh. I knew I'd be visiting some very good buddies in NJ in the middle of December and that we had planned for them to play-test the prototype. I had every intention of making some very specific changes before then based on what Chad and I talked about after Design Day. I didn't make those changes, which made me a bit wary of actually running the play-test in NJ. Honestly, it felt a bit dumb to bother since I already *knew* what I needed to change in this version. But had these friends lived closer, they'd have been the first ones to play-test our prototype. Additionally, I wasn't sure when I'd see them again in order to play. So I ran the play-test anyway, even if a bit hesitantly.

Turns out that distance really afforded me some excellent perspective and put me in a place to receive the feedback and data without feeling as though I'd just heard all of it. Plus, after this last play-test, a couple of the bigger changes I was confident we needed suddenly seemed far less necessary. Not that I've thrown those changes out and won't be making them period full stop, but rather I'm happy to make a lot of other changes first and play-test some more before revisiting some of these other ideas. The creative process is rarely linear and discovery takes time. Afford yourself the space and time needed in order to gain the perspective that will help your process.

Don't Interrupt, but Keep It Focused!

I'm pretty sure I've written about this before: listening to your play-testers is key. That's not to say you should implement every single bit of advice or feedback they give you, but you should be listening to all of it. And not with the intention to defend your choices or design. Too frequently, we listen to folks so we can prepare some sort of rebuttal. It's not always adversarial; I might just be waiting for an opportune moment to say, "Oh man, yes, me too!" and then go on to share an anecdote about an experience I had that I think relates to what the other person said. I'm not saying that's never appropriate, but I am saying that we're not really listening if all we're doing is preparing to respond.

It can be overwhelming to receive feedback (especially from multiple play-testers) after playing a game. Especially if the feedback is not necessarily delivered in a thoughtful manner, if you're a novice game designer (like me!), or you're very attached to the project and take any of that feedback personally.

"Man, I am really close on this one!"

But all of this to say: don't interrupt play-testers when they're offering feedback. One of the ground rules I set for feedback after the play-test ended was this: "I will be asking questions as well as clarifying questions, but I will not say anything else during this session unless it's to correct a mistaken understanding of the rules/gameplay or if you ask me to explain some of my thinking." And you know what? There was only one instance in which the play-testers asked me about my intention. For every other piece of feedback they had, my intentions were all but irrelevant to the feedback they were offering. In other words, my feedback was much less useful to them than their feedback was to me! By choosing not to interrupt, I got a literal TON of useful data and insight from my play-testers because they could share everything without worrying about how I was taking it. Plus, not every single thing they shared turned into a full-blown discussion. The very few times I interrupted were when the notes being offered had to do with production. Each time that was the case, my question was: "did this impact the play-test for you?" If the answer was affirmative, then I listened so I could improve the prototype in order to ensure that future play-tests were not slowed down by any confusion I could easily avoid by said improvements. If the answer was negative (footprint of the game, heft of polyomino pieces), then I said they could note it on their feedback form, but I wasn't looking for that feedback. Why? Because Chad and I plan on pitching this game to companies rather than produce it ourselves. So production insight (style of artwork, quality of components, etc.) just isn't helpful feedback in this context.

This isn't always the easiest advice to follow, but I promise that it creates a much more fruitful play-testing environment.

Don't Accept Feedback During the Game

I truly can't stress this one enough. I know it can be frustrating for play-testers when they can't share things as they think of them. I also know it's sometimes a pain for players to take all of these notes while playing because it's not normally a thing we do unless we regularly play-test games for other people. I cannot begin to guess how many times a play-tester began offering feedback during the game, only for me to gently interrupt them and ask them to write it down, have them sigh, and then write it down. It's frustrating all around. But I tell you what: it's huge. For a couple of reasons.

1. It's hard to run a play-test when you're not only answering questions about the rules and gameplay, but also having to take notes literally every time a player wants to share something. Especially if those thoughts need to be clarified so you can understand exactly what they mean, since you're not living in their head. If you're taking notes as players are dictating them, you can't really take notes on your own observations because there's no bandwidth for it. I can't really observe how the players are responding to the game in the midst of playing it if I'm too busy taking dictation from three or four different people.
2. Sometimes, feedback changes from the time it is conceived to the end of the game. Perhaps the thing resolved itself! Maybe the thing they really liked turned into something they despised. The feedback simply isn't complete until the play-test is completely ended, so receiving it in an incomplete form just isn't that useful. It's like if someone asks you to read one and a half chapters of a 35 chapter novel they're writing: problems that crop up after a chapter and a half will likely be resolved over the course of the other 33.5 chapters. So that feedback simply isn't relevant anymore. Additionally, the angsty kind of attitude a character displays may not bother you over the course of that first 1.5 chapters, but if it continues for the remainder of the novel, by the end, you may have every inclination to set the book on fire. The same is true of play-testing board games.
3. It slows down the play-test. This literally messes with everything else in your play-test: it increases the game length, it colors the observations people have of the game, it might change the tactics the players were planning on using...seriously, when a game I planned on finishing in 1.5 hours takes 2.25 hours instead, that colors how I feel about the game. And rarely in a positive way. Plus, if I don't agree with the feedback or it's simply turning into a dissertation, I get bored. I want to do something else. I won't be paying attention. The data you get from players after a play-test like this won't really accurately reflect the game in its current state if you spend the entire test taking feedback and slowing the whole thing down. Even the easiest metric (game length) is suspect. I won't go so far as to call it a wasted play-test, but it's not going to yield nearly as much as each play-test should.
4. This is what the feedback forms are for! It's so much more helpful when a player's feedback collected during the course of the game is all in one place: on their feedback form. Also, I don't collect play-testers' names on their sheets, but since we haven't play-tested that many times, I know who has written on which sheet. But regardless of any of that, if I collect feedback during the game and make the notes in my own notebook, I can't necessarily connect the feedback dots, so to speak. What I mean is this: one play-tester said he wouldn't have played our game if he'd had a better handle on the central mechanic because he doesn't like dice games. Totally valid! But it does color the other feedback he shared after a single play-test. Being able to group the feedback he offered with that knowledge helps determine whether or not some of the feedback is worth implementing! After all, if he would never play this game anyway, how many of his proposed changes are things that we even want to change? If a player doesn't like pick-up and deliver archetypes, then it's possible that implementing the changes they suggested make our pick-up and deliver game something else. Which...you know, isn't necessarily what we were trying to do. So having a player's feedback altogether on their sheet helps offer perspective than should inform how and why we implement feedback. Additionally, if three players all agree something isn't working, I might only make a note of it once. If I see the same feedback spread across several feedback sheets independently of one another, that feedback may hold more weight as I'm debating changes to make.
5. And finally, what will you do with this feedback that you have received during the game instead of after the game? What changes will you implement after collecting some of this data an hour earlier than if you'd just waited until the end of the play-test? The answer in 99% of situations: absolutely nothing. The play-test is already underway, so unless you're calling it early to start the whole thing over (which may totally be the right call!), there's just not much to gain from getting the feedback an hour earlier than you would have gotten it anyway.

I doubt any of this will come as a surprise to seasoned game designers. I'm sure even plenty of neophyte designers are saying:

And hey, I get it. This stuff really feels obvious and in a vacuum, I'm sure these conclusions are easy enough to reach. But in practice, things are a little different. Maintaining perspective and being patient are tougher when you're personally or emotionally invested as opposed to being discussed hypothetically. And honestly, every single one of us is worse at actively listening than we realize. We can always afford to improve that particular skill. I'm pretty good at maintaining perspective and actively listening, and I still need to work on both while running play-tests for games, so I thought these were things worth mentioning. Honestly, if these thoughts help smooth over even a couple of play-tests, just that little bit will likely help you in collecting more productive feedback than you were collecting before, I promise!