I may not be the best game designer in the world, but if there’s one thing I do know, it’s playtesting. I’ve been a playtester for a variety of games from RPGs to party games to board games to light card games to heavy war games. I’ve been chief of product development for a startup card game publisher, and a lead playtester (and copied on ALL playtest reports) for Marvel Heroic Roleplaying. A good set of playtesters can make your good idea great, or kill your bad idea before you invest too much time and effort.
With the open playtest of the new iteration of D&D coming tomorrow, I wanted to offer some of my advice on playtesting and giving feedback. Wizards of the Coast will provide plenty of instructions on what they do and don’t want to see, so obviously that could easily supersede anything I say here. These are some general guidelines to keep in mind for D&D, so hopefully you find these tidbits helpful while playing the game and collecting your feedback.
Respect Their Playtest Decisions
The designers at WotC have decided that the first thing we’re going to see is going to include pre-generated characters, and not have character creation rules initially. I understand not being happy with this decision, however, it’s not like they’re going to suddenly decide that there will never be character creation rules. So when submitting your feedback, you don’t need to tell them “I wish I could see the character creation rules.” As professional game designers, they’ve decided (after many meetings, I’m sure) on this method of rolling rules out, so try and respect that. Keep your responses to what you were provided, not complaining that you don’t have what’s already been promised.
Work From the Big Picture Down
There’s plenty of time in the development cycle of the game. The wording of a feat or the exact text of a spell are important but shouldn’t dominate your feedback. After all, if the entire magic system changes, that’s a lot of wasted effort in critiquing 1st level spells.
That’s not to say you shouldn’t note these things down. However, the big picture is the most important: how the game feels overall (in this case, does it feel like D&D?), was combat satisfying, and so on. This part will probably be represented most in surveys. Also important are core mechanics: rolling to hit, how skills work, hit points, etc. Then the lowest amount of your attention should be given to the small details that could easily change later.
Be Charitable With the Format
If the playtest files are anything like the previous versions, it’s going to look like a Word document. No art, no fancy layout, only minimal attention to organization. Again, this is something that we know the final product will have. So picking apart the exact placement of a sidebar or location of a rule isn’t going to be worth a lot.
If there’s something that you find confusing because of placement, it’s OK to note it. Just don’t focus on it. Be charitable with the format the content is taking right now, and do your best to be charitable about these kinds of issues in favor of focusing on the actual rules and gameplay.
Don’t Kiss Up…
Even if you love everything about the game, you don’t need to gush to the game designers about how great they are. It’s simply not helpful to the game.
…But Be Sure to Praise What You Like
That said, if there are specific rules or aspects that you like, be sure to note them. All your feedback doesn’t have to be about what doesn’t work: noting what does work and what you really enjoy can be just as important (if not more so) than what doesn’t work.
Offer Suggestions, But Let Them Design the Game
We’re all game designers to some extent. We really want to be helpful too, by offering alternatives. This is OK in controlled pieces. Just don’t hang too much on each of these. Remember, you’re likely responding to only portions of the game, while they’re both looking at the game as a whole, but also comparing your feedback to everyone else’s, and trying to find mutual solutions to as many issues as possible. Thus, your suggestion on how to fix a rule might be brilliant, it just doesn’t necessarily fix everything that needs to be fixed at once. Let them do the heavy lifting as far as the actual changes, and instead, focus on communicating what you feel works and what doesn’t work.
Note Your Dealbreakers, But Try and Put It In Context
You may decide that any game with bards is not a game you want to play. You may even refuse to play any game that uses a d12. I can’t tell you how many things I’ve now seen on forums of taking the form “if D&D has _______, I know it’s not the game for me.”
Certainly note this in your feedback. But be aware that the vast majority of players aren’t going to share this view, and the game designers have to cover all of those, not just yours. So try to take those kinds of rules in context. If that’s the only thing that doesn’t work for you, do your best to put it aside. Give the best feedback as you can, taking a larger view of it all… even if you can’t stand bards.
Playtest What You’re Given, Not What You Fear
Finally, remember that what you’re seeing is only an early portion of the rules. It’s an important portion of the rules, probably the most critical parts, but there’s plenty more development and expansion to come. Thus, if you try a rule, and it works out or it doesn’t, report on that. In a lot of cases, you can extrapolate some worst case scenario of how a rule couldn’t work based on what comes later. Resist this urge. Assume that whatever comes later will be properly balanced, come out at the right rate, and be explained perfectly. You can’t predict what form these rules will ultimately take, nor does it help the designers for you to try and out-guess them. Playtest what you have in front of you, not what might come later. If this open playtest goes well, maybe you’ll be there to playtest everything else too.
One final note: I compiled this list of advice based specifically on playtesting this version of D&D. Every game with potential is different in some way than what’s come before. That means there’s not one-size-fits-all advice for whatever game you might be playtesting. Hopefully, however, this will help you when filling out your surveys when playtesting the new version of D&D, to help make the best game possible.
Great well thought through and sensible advice, to make sure we keep our own feelings in check and offer what the game needs by way of feedback. This advice goes a long way toward helping making the playtest achieve its purpose, turning D&Dnext into the game we all want to play. Thanks for sharing.
An additional consideration is how a group should approach playtesting. It is very valuable to capture initial reactions when you first look at the material, but it is best to keep that separate from the actual playtesting you do. Here is what I recommend:
1. Read the material. If something causes a strong initial reaction, capture that and flag it as an initial reaction. Mentally, shed that reaction (even if positive, but especially if negative).
Example: A game says that every time you roll a crit you roll 7 dice and then roll and subtract 4. It sounds atrocious to you, so you note it as an initial reaction. You don’t let it color your experience.
Looks like I got cut off by mistake. Here is the continuation:
2. Play the game. Play it without negative/positive bias from those initial reactions. Do your best to cleanly assess the game. Don’t let those reactions color and prejudge your assessment.
Example: In play, we find that the crit rule worked ok. Surprisingly, it was exciting. Maybe the number of dice could be cut back if we found that the rule took up too much time. (Or, maybe we find that the extra time heightens the experience).
3. Go back and assess the initial reactions. Was that initial reaction an issue? Could it be? Make a clean assessment based on the actual play.
Example: On the whole, we find the issue really scared us initially, but was really minor once we played.
4. Decide whether to report the initial reaction. The initial reaction can sometimes be worth mentioning.
Example: If most of the players had a strong initial reaction, we might report that. “Rule xyz caused a very strong reaction from most of our group. The wording made it seem like it would detract from play. In actual play, however, it was not an issue.”
I mention this because without that approach it is very easy for the group to color their game with any strong initial reactions. This is especially true if the game uses a personal pet peeve. As Dave noted, we want to keep in mind that the game has to serve many players with many different takes on the game. One gamer’s pet peeve is another’s favorite aspect.
alpha, i really dont think some of that is very good advice, it sounds like you just want to give it a free pass no matter what it looks like. ‘you hated it at first glance, oh well maybe you should sit down and explore your feelings’. first impressions and reactions are important, and if you still hate it after a game, you dont need to meditate on it until you convince yourself to like it
i dont think playtesters should be ‘charitable’, they should be honest. if you hate it, say you hate it. you dont have to psychoanalyze why you feel that way, god knows the average person playing the game wont. strong intitial reactions are crucial and priceless feedback to any business.
i also dont think you need to have other players in the back of your mind, or that you should try to let assumptions about other gamers color your response. provide your own personal opinion, and dont worry about what someone else likes
That’s not entirely correct. You’re a playtester for the GAME, not market research for the FINISHED PRODUCT. As I said, you do want to note where you run into issues, especially with first impressions, but it shouldn’t dominate your feedback, since that is not useful feedback.
I agree with froth. Constructive negative feedback is just as important, if not moreso, than positive feedback. First impressions are important too, as people in the wild are going to have them after the actual product is released as well.
EDIT: I agree with Dave too! It should be a part of your feedback for sure. 🙂
The key word is constructive. See my point above about dealbreakers.
Froth, “hate” is a word I would almost never use in a playtest. When you playtest enough you see that there is just a huge diversity in what can please a gamer. Strong language like “hate” is emotional rather than constructive. Consider the Warhammer minis player who also likes D&D and the FIASCO/Vampire/LARP player that wants to dabble with D&D. Ideally, the game should appeal to both (as it has to some extent with every edition). It isn’t fair for me as a playtester to label something with “hate” because of where I fall on that spectrum (or any other).
It is incredibly important to point out where rules do not work well, where game play suffers, where clarity is lacking, etc. But this should be done in a constructive manner. Here are two examples:
1) Healing surges were the best thing introduced to 4E. We all agree you were idiots to remove them and hated our game because of this.
2) Character fragility was an issue. In our playtest we found one or more PCs were often low on health and our healer’s spells had run out, causing the entire group to return to town. Players often wished they could keep going. Several commented they missed self-healing in the style of 4E surges.
Beyond being rude, the first one just doesn’t tell us anything. It isn’t constructive. It doesn’t explain why healing surges were desired, and specifically why in this playtest they were missed. The second might be faulted at some because it is prescriptive, but it isn’t heavily so and it does focus on identifying the issues. It trusts the designers to resolve the situation based on what they know (which is nearly always greater than what the playtester has seen).
Does that make sense?
Here’s another quick example of playtesting. I sent my second LFR adventure, Scout’s Honor , out for playtesting to 3 groups. Here is what they said about one encounter:
1) Total cakewalk. Our group was done in 15 minutes. Consider making the boss more resilient, maybe some terrain to punish ranged types.
2) Frustrating TPK. Our group had several conditions on them at nearly all times. We could never get into position and the monsters danced around us, killing us. Way too much domination.
3) Awesome! Our group had a lot of fun. This was our favorite encounter of the adventure, though it ran a little long.
I don’t know if you can put yourself in my shoes. I had worked my tail off on this adventure and lost a ton of sleep on it. I was exhausted. And I get back those three conflicting results!
Thankfully, I knew the groups. I went back and asked questions. Here is what I discovered:
1) This group won initiative and used ranged strikers to eliminate the key monster before it could act. The encounter was too easy when that happened. I changed the terrain to force ranged PCs to move closer, and I further protected my key monster for the opening round.
2) Group two had only 4 players. I had a trap that dominated, and when it hit (it wasn’t very accurate) it was devastating for a party of 3. Add in that the monsters had additional control and the combat was very frustrating. The solution was to remove the trap for a party of 4, as well as a few other minor adjustments.
3) Here I found a party of 5 with normal initiative and good party balance. I looked further and found issues that would affect other parties and removed those. I re-worked the monster and PC placements to make the encounter end sooner.
As playtesters we can feel “our results” are king. It can be surprising how what we see is not at all what another group sees. Understanding that can help us be constructive with both our praise and our criticism. It can also help us realize that our playtesting will be more useful when we provide the play information that tells the designer why we had a particular result.
Clean and simple advise.
Thank you.
x+1) Playtest with your group and come to your own conclusions without reading about other people’s ideas and experiences. “Me too” responses don’t help much.
I see Alphastream, being omnipresent in good discussions, already hammered on the points I was mostly going to make, but one thing I do think he actually missed is understanding your own explanations and suggestions.
It is very important for game developers, as evidenced by an entire team devoted to this concept during the playtesting of Borderlands (a video game folks may have heard of due to its “SO MANY GUNS!” factor), for you to note for them what you feel is going on in addition to what you feel should be changed to improve or fix that.
For example, one playtesting group for Borderlands recommended that a location be a lot smaller because it took too long to cross. Indeed, in modern day public transportation we see this sort of factor all the time (improve the speed of the rail line, engineers, the passengers are getting bored!), but that’s not actually good feedback as it turns out (or, rather, it’s only half good feedback). The good part of the feedback is “this location seems to take too long to cross”, that’s about as objective as playtesting feedback can get in a useful sense, but the subjective feedback was “you can fix it by making it smaller”. Instead the team decided it seemed too big because there were too few enemies and just added some of those, giving the player something to occupy them as they crossed the location. Similarly, rather than spending a 100,000 dollar budget improving the speed of a rail line, you could instead just install wifi for about a thousand and the people on board the train will be occupied and will not notice how long it takes to get from A to B nearly so much anymore.
Therefore, the MOST important part of our feedback is the ISSUE. “D20 seems too random”, “I feel too fragile as a fighter”, or “wizards seem too strong” are all good issues (potentially). What is good to see, but a lot less useful, as a designer is “you should fix this by using 3d6 instead”, or “fighters should get more money to buy better starting armor with”, or “wizards need fewer spells”. In conjunction it can be helpful for steering the changes made (if any) when presented alongside the issue, but it is not as important as the issue, and beyond a shadow of a doubt if it is presented in PLACE of the issue it is very nearly useless.
@Froth I can’t say I disagree more. First impressions can be knee jerk reactions. There have been times I have played a game and saw something I’d thought I’d hate and ended up really enjoying the rule once I saw it in play. I suspect feedback that says “We played this and as a result our feedback is….” will carry more weight than “I read through the rules and I think this is wrong because…”
Thanks, HydraDM! As my wife and Shrek both point out, it’s getting me to shut up that’s the trick! 😉
The Borderland tale is a great example, and I agree. I like the tale because it tells us a number of things. We want to provide good feedback so the designers can figure out what really triggered the reaction. And, we want to have some patience and objectivity to understand that what we envision as a “fix” may not be what is used. I can’t count how many times what I suggested for a playtest was not used, and yet the final version was awesome. And, I can’t count how many times a playtester gave me a suggestion I didn’t use, but which was useful in helping me assess things critically. In many cases the final choice was influenced heavily by the feedback, and would not have been possible without it… even if the final choice was not what the playtester recommended. That’s why when we playtest we need to check our ego and selfishness – we aren’t playtesting for us. We are playtesting for everyone.
Your points about making the issue clear is good. It can take practice to find that clarity though. And that’s why design teams often ask us not to try to solve the issue but rather to explain the particulars that make the issue occur. This is why _how_ you write your playtest report matters. It isn’t about harnessing rage, because that doesn’t help a designer understand the issue. Bold, underline, none of that helps. What most helps is analyzing what took place and explaining how the issue is triggered.
Shawn and Darryl make good points about first impressions and shutting off outside influences. I read a few D&D Next accounts where the person said they didn’t like what they were reading, then went on to run a game that they and the players really loved. I’ll say on my end that I didn’t expect D&D Next to be the most fun I had DMing at D&DXP, but it was. It really was. And that’s with all the flaws that have been fixed since that time. Similarly, I’ve seen tons of organized play games where the DM was negative. You sit down at the table and the DM says, “sorry, guys, this module is really lame,” and it just colors the experience. On the other side, I’ve had great DMs run a fantastic table and only at the end say, “you know, I was worried about this adventure. I thought it was going to be really bad, but we had a blast, didn’t we?”
4E itself suffered from that. It was a game that even for fans took several tries. I was part of the playtest (and can’t talk about that), then the D&DXP preview, and I really thought it had severe issues. A friend ran it for me and it was a bit better. I made and ran an adventure and had a lot of fun. Before I knew it, 4E became my favorite edition.
It is for those reasons that some companies go with closed playtests. They want to prevent contamination and group think, which can really be self-fulfilling. The playtest for the 13th Age RPG asked us not to speak to other playtest groups. Even after we were allowed to talk about our 13th Age playtest, the publisher asked us to try not to read other reports. All of this is to prevent that group-think bias. If you attend a session of Encounters or go to PAX, you can see the lack of that effect; new and casual gamers don’t care about edition wars – they are busy having fun.
As someone who playtests a lot, I hope that enough of the community can keep Dave’s post in mind. This is a really special opportunity to be part of the process. Give the game time. Give it a fair try and then a few more. This playtest is the opportunity to help the game be as good as it can be.
@Daryll
I never suggested not playing it! Of course you should play it. I just think this is a little Jungian for my tastes:
“Go back and assess the initial reactions. Was that initial reaction an issue? Could it be? ”
“Decide whether to report the initial reaction”
I’m afraid I have to agree and disagree with everyone here. (Call me a wishy-washy facilitator.) I’m a programmer by trade, and I write custom applications all the time. As such, I know a little about design testing and specifically, what’s helpful and what’s not.
First, you need to document your initial reactions to the material. Why? Because many people don’t use the “just shut up and try it” mentality. Thus, if a concept isn’t well-received upon first blush, there’s a solid chance that people won’t try it at ALL. Also, communication through a written medium is a tricky horse to ride. What makes perfect sense to me won’t make sense to my mother. So just because you were turned off by the way it was presented doesn’t mean someone else won’t get it immediately. WotC needs to know that you didn’t like it initially because it helps them to work on presenting the information in a way that makes it more palatable to everyone on the initial read.
Remember that D&D Next is supposed to be a “modular” game. So if your initial read of a feature or rule makes you want to vomit, there’s a really good chance that you won’t use it. If WotC sees a lot of people having poor initial reactions to a feature, then there’s not a good chance that it won’t get used by folks, even if it’s super-uber-cool in execution. What good is a feature if no one uses it?
Secondly, just because you didn’t like the way it read or was presented initially doesn’t mean you should just write it off. You’re a TESTER, so TEST it. As Alpha pointed out, the actual execution may be far better than your initial reaction. If that’s the case, you need to let WotC know about that too. “Hey, we initially were repulsed by the idea, but upon further review, it actually works really well.” That tells WotC that they need to re-examine the PRESENTATION of the concept, not necessarily work on the concept itself.
Thirdly, lather, rinse, repeat. Don’t just give things a single glance, or a single try and move on. Use the mechanic early and often. See if it makes more sense or works better the more you use it. If it doesn’t improve, then it’s likely a mechanical flaw. If your acceptance of the feature improves with use, it’s a presentation issue. Both bits of feedback are useful to the designer.
Lastly, is my favorite bit of advice: TRY AND BREAK IT. Nothing is more frustrating to me as a coder than to have someone test my app exactly as designed. Of course it’s going to work if you use it precisely how I tell you to use it; I WROTE IT! I KNOW it works that way. I want you to use it YOUR way and see if anything breaks; mostly because not everyone thinks the way I do (thank GOD). Who knows? Maybe you can come up with a cleaner, more logical way of executing or presenting the material.
From the D&D Next perspective, play the beta as they intended and see how well it runs. Once you’re comfortable with the rules as written, start thinking outside the box. Go out of your way to use rules in unexpected ways or combinations. Have the wizard engage the ogre in melee combat. Cast a Light spell on the nose-ring on its face. Have the fighter spend an encounter just pushing/pulling people around. See how long the rogue can stay hidden. Arm every character with daggers and see how long combat takes.
If everyone played Mario exactly as expected, we’d all be shocked when the game crashes because you jumped off the left side of the screen instead of going right. Though you may think something is crazy or that no one would ever do something, believe me, SOMEone out there WILL. As a designer, you need to know whether your rules will adequately cover most situations, or only target certain, specific ones.
I’ll end with a final bit of advice, which echoes what the others said here: don’t just say “what”, say “why”. Telling WotC that “The new initiative format sucks” is akin to my mother calling me and saying, “My computer’s broken.” Designers need to know specifics. What specifically didn’t you like about the feature? How were you using it when you found you didn’t like it? Was there a way you used it that you DID like it? How do you think it could be improved? It’s always better to give more feedback than less.
My $1.25 on the topic. 🙂
One thing I know from my limited playtesting experience (I just playtested Torchlight 2, ARPG videogame, if you didn’t know) is that the publisher had public feedback thru their forums. Lots of comments were ‘useless’ lots were ‘nitpicky’ and lots were ‘love/hate’ emotional.
This is all fine, but I realized something. I’m *just* a playtester. Big deal. Don’t get a swelled head because you’re playtesting something. You are not a paid game developer. Your opinion doesn’t really matter all that much. be helpful when asked (via surveys, etc.) and discuss with fellow gamers, but don’t get all arrogant. JUST HAVE FUN!
One important guideline that I learned when I worked in technical support (and that I continue to follow to this day as a software quality engineer) is to be as specific and verbose as you need to be, no more and no less. What I mean by that is that IMO this type of feedback:
“Rule X stinks.”
is less useful than:
“My character, Reginald Fuzzybottom, had a bow that does Y. When I used that bow in the second encounter, … [long character story removed] … So that’s why I dislike rule X.”
which is less useful than:
“The way we interpreted rule X is that it does foo. When I used a character with a bow that did Y, the interaction between Y and foo was that every monster died with the first shot. That made the second encounter trivial. I think that rule X should do bar instead.”
The first feedback doesn’t give any explanation, so there’s nothing that the designer can DO about it. It’s not actionable. It’s too cold.
The second feedback may hide the actual problem/concern in a long story and it can be difficult to tease apart the narrative piece and the actual feedback. The designer may be able to fix the problem, but only after spending ten minutes figuring out what the problem actually IS. It’s too hot.
The third feedback gets to the point quickly, explains the situation, and gives the developers something _specific_ to think about. There are several actions the designed could take; if the interpretation of rule X is wrong, they could clarify the rule. If the interaction is too strong, they can fix it. If the monsters are too vulnerable to the interaction, they can change the monsters. This feedback (again, just IMO) is just right.
As a professional software tester, I think a lot of the guidelines cross over. At this stage in a software project, the fact that the company’s logo is the wrong shade of blue is, while important, not entirely relevant. The important things are major crashes in the system, miscalculations, unreachable controls, etc. I think the point about the “Big Picture” is the one that really strikes home for me here.
Nice article, and good advice for how we can help make D&D be a better game and make contributions. Thanks.
Personally, what I’ve found most useful in playtest reports:
1. What you liked, and why.
2. What you didn’t like, and why.
3. What was missing, and how you discovered it was missing.
4. Three things you loved, three things you felt could be improved.
Season with specific points where play may have been either too bogged down or too quick.
-Ben.