Alternative titles for last week’s poll: “Blood, Sweat, and Tiers”, “Tiers of a Clown”, and “Tiers for Fiers.” 91% of you have played lowish level D&D recently, 27% in the mid to high range, and only 3% dare to go EPIC. Some interesting stories in the comments too! Whatever levels you’re playing (or not playing with something with levels at all), I hope you’re having fun with it!
Earlier, there was some controversy over a statement criticizing a review of an adventure module where the reviewer did not play through the module before reviewing it, but (as we here at Critical-Hits have done many times before) instead wrote a review based on reading it through with a critical eye. The manner in which the argument was presented lead something to be desired and caused some considerable debate, especially amongst those of us who do regular reviews on adventure modules and other products.
I would say there are two main schools of thought on the issue (which can extend to game rules as well), with plenty of gradation between them:
- Adventures/Games need to be played to be properly reviewed. Many adventures and games play better than they read, so the only way to really give them a shot is to play them. These kinds of reviews represent everything about the product, from beginning presentation to execution.
- Adventures/Games can be reviewed without running them. These kinds of reviews can tell you how the product is put together and constructed, and aren’t influenced by external factors (like the skill of the GM running the adventure or the group playing the game with.) Additionally, more reviews can be produced since it takes time to actually play the game/adventure in question.
Both clearly have their supporters, since this kind of debate rages often. Plus, people argue about reviews and reviewers in general all the time without even getting into the exact style. Sounds like the perfect time for a poll!
[poll id=”140″]
Your answers may guide how we do reviews. If absolutely no one wants a review based on a reading, we may stop (even though they’ll take longer to come out), and if no one cares about how a game plays and just wants first impressions, that helps us too.
I got the first vote!
I don’t care about either extreme – I generally don’t find play reports of a game to be helpful, because what one person found fun about playing the game with his or her group (“killing those kobolds was totally sweet!”) I wouldn’t necessarily care about. What I look for in a review is talking about the book as a book. Talking about the text, the encounters, the storyline, the presentation, the product quality. The game aspect is so varied that I don’t find it useful to read about how it went for you.
But that’s just me. So I write the review that I find palatable – a review that isn’t about and doesn’t bother with a play report, and just talks about the book. There’s also the factor of time constraints to take into account. I find it less useful to spare the time to play a module (which can take ages with your group – and playing it by yourself for a review removes whatever utility an actual playtest has, at least from my perspective).
I don’t care about the two schools of thought or whatever. Both have their merits. Me, personally, I don’t care for the playtest reports.
.-= Wyatt´s last blog ..Creative Skill Use In 4e =-.
I’m with you in the region of reviewing modules that I don’t run…who has the time to run every module. Now and then I’ll get to run something, but not often.
I think someone with a critical eye and experience can pretty well get the jist of a story, can understand if something is broken, and if a module is well constructed. So long as the reviewer isn’t claiming they ran it when they didn’t and are up front about the situation then I have no problem here.
That said, I have seen many cases where the product publisher probably shouldn’t complain about a module not being read. I often find problems, mistakes, and down right bad module design when running them that I may have easily passed over upon a critical reading. I can point to this in several occasions.
.-= Jeff Greiner´s last blog ..The Tome Ep 115: Halfling Cleric and Pacing =-.
I voted for “The reviewer is usually fine with just reviewing the product from a read-through, but a play might help”
In reviewing an adventure by reading it through you can pretty much see how its going to play out, but you can’t think of everything the pc’s are going to do.
So playing through with a group might change your opinion on the adventure to some small degree.
.-= Aaron´s last blog ..Dungeons and Dragons 4th Edition =-.
I think reviews just with a read-through are quite useful, though much more useful after actual play.
Also, if opinion about something changes post-read-through review after actually playing a game, it is very useful to republish/edit the review.
.-= deads´s last blog ..DMG2 Artifact on Facebook; Paragon Tier of Assassin on DDI =-.
I think I agree 100% with Goodman Games’ response to the situation.
They never said the review was bad or unnecessary. But many minor issues that may come out in the critical reading of an adventure won’t affect the actual play at the table.
Note that this mainly applies to adventures, as other sourcebooks are usually designed to be used primarily away from the table.
I think that the review in question was overly nitpicky, as most of the “problems” I read were minor details such as how long a cab ride would be, but was still a valid review.
So, I guess to answer your question:
Read-through reviews are great for new supplements, and work well in general. For adventures in particular, I’d prefer reviews (not play logs, as some above are saying) after actually playing/running them, but this, of course, is rarely possible.
For new games (different systems, or whatnot), after-play reviews are pretty much necessary. Just think of all the 3e players and their sans-playing 4e reviews when the pdfs got leaked, or the many reviews of d20 Modern that decried it for crazy things (that worked great in practice) like the Wealth system. This also extends to card/board games, as a review of Say Anything or Dominion based solely on their rules pamphlets would be… unhelpful.
.-= Graham´s last blog ..Damn you, Dave! You and your… logic… =-.
I picked “Other,” the last option.
I think the reviewer just needs to make it very clear whether or not they have played the game, or have just done a read-through. That way we, as readers, can set our expectations for the review and account for that bias. Usually it is clear from the text, and a good reviewer will say something like, “Maybe this works really well in play…” or “It was really fun, but it helped that we had an awesome GM.”
I think the ideal, as other people have said, is a hybrid model of sorts. First, a critical review of the module as it is written. This would give me a sense of how good the product is in and of itself, and what I’d get for the money I spend on it. Second, a playtest report, where the elements discussed in the first part are tested in play. If nothing else, this would give me a sense of how relevant the factors discussed in the first part of the review are to the actual play experience. Both elements are important to me when determining whether I’m going to pay attention to or ultimately buy an adventure.
I think every time a publisher chooses to go out and “discuss” the reviews of his products, he has already lost. Even if he is right and the review is flawed.
.-= Jack99´s last blog ..No more Ultrashort Reviews =-.
I chose “Other” too.
What should be important is that the reviewer discloses whether it’s an actual-play review or not. Then it’s up to the reader to decide how much weight to give to it.
.-= greywulf´s last blog ..With the wind in her sails =-.
I snapped to the judgement that Goodman was right at first. I’m thinking that a lot of the 4e material can’t really be judged until we see it running at the table. How many of the original Monster Manual reviews talked about the problems with solos and minions? Would this really have changed anything? For some of these sourcebooks, we might not really know how good they are until we’ve had it for a while. For adventures, we might not know how good it is until we run it but how much of that is the material an how much of it is the dm? These things are so subjective anyway.
The value of reviews is another question. I read the Critical Hits review of the DMG2 but there was no doubt that I’m going to buy it anyway. Unless the thing was an absolute mess, I doubt any review would sway me. I feel the same way for most of the 4e books. I usually know whether I will buy it or not based on the title alone. So much of this game isn’t based in the books anyway but on how we use them. Unlike movies or video games, I don’t just consume these books, I use them to create my own games.
Anyone can review anything they want. This is the net. We all have a voice now. As a reader, I will pay more attention to the authors I trust and ignore those I don’t. Playtest reports are more useful to me than readthroughs but I can’t expect this alll the time.
.-= Mike´s last blog ..Three Acting Tips for D&D =-.
Considering I’ve reviewed modules here without running them first, I think we know where I stand on the subject. The facts of the matter to me are that mostly DMs/GMs buy modules, and we then have to read the module, so if it doesn’t read well then you’ve just bought something that you don’t even want to run. For me this discussion only effects modules that are poorly written which somehow then play out very well, which I view as being a small subset of a larger whole.
I will just add that the whole “controversy” and discussion came about from Unclebear because Goodman tweeted at the same time as he was responding to the review calling the reviewer an idiot and referencing movies being reviewed based on a script. Aside from a lot of problems I have with comparing movies to published adventures, I think a lot of this is just a reaction to the personal connection that Twitter allows many of us to have. Goodman seems to have handled his responses online decently, if not a bit confrontational, but his tweets on the subject revealed what we can assume are his personal feelings on the subject that are a little bit concerning.
@Bartoneus –
Actually, you may have hit it with your comment in the Giants thread.
Use the video game review terms!
Did you play it? No? Then call it a Preview.
Yes? Then it’s a Review.
(It’s probably also been out for a while if you’ve had time to play it, while the non-played reviews tend to be at or before release.)
.-= Graham´s last blog ..CriticalAnkleBites and ChattyDM pretend to be journalists =-.
@Graham: That’s not a bad idea, though I called my E1: Death’s Reach article a preview because I posted it several weeks before the module was released. 🙂
I think Goodman had a decent point, but of course made it in an inflammatory manner which is usually the best way to generate discussion. 😉
On the practical side, of course you can’t play through everything before reviewing it (anyone managed to play all PHB1 classes and races from level 1 to 30 yet and can finally give us a review?). Plus there is plenty you can tell from just reading through a product. In fact, most of the useful information I get from a product is a run down of the what’s included since I don’t get much chance to go to my FLGS and flip through the book physically myself.
However, that being said, playing with something usually reveals even more information about something that might not be obvious to a mortal human just reading something through. So, when feasible, I find reviews based on actual play to be very valuable (and certainly not just play reports, there’s a world of difference). I’ve been gaming for decades and I’m still surprised at the difference in how something reads and how it plays at times.
As long as reviews are clear when they have actually played through a product, I’m fine with both kinds and realize that for purely practical reasons, there will be far more “I just read it” reviews than not.
First off, full disclosure: I’m a frequent freelancing contributor to Goodman Games, and have had many conversations with Joseph but do not presume in any way to speak for him.
That said, I voted for “The reviewer should usually have played the product in question, but some exceptions are OK.” As an editor and blogger myself, I know that Time is a harsh beeeyotch, and sometimes things must be rushed either for the sake of deadlines or for the sake of being timely. But to have the default position that an RPG adventure can be thoroughly vetted and reviewed without playing it is akin to analyzing a recipe for a new dessert without finding out if it tastes yummy. A review that includes some degree of playtesting, even if it’s just sections of the module, is likely to produce a more proper appraisal of the product’s intent.
At the very least, then — as Will and others have said — the reviewer should make it very clear upfront that the review does not include playtesting.
Would I have used the same words on Twitter as Joseph? Probably not. But then, as the other Ken above implied, would we be having this valuable discussion if he hadn’t? 😉
.-= Ken of Ghastria´s last blog .."Solomon Kane" movie looks NICE! =-.
As long as you’re clear about how you formed your opinion, either style is fine. I put more weight on playtested modules– but I’m very happy to hear “this is logically organized and worth investigating” without a long wait.
I tend to think that the more experienced a player/GM the reviewer is, the more readily they can probably evaluate a product without running it through its paces in play. Regardless of the experience of the reviewer, though, a review should definitely always be prefaced with what kind of going-over the product has been given. I.e. if the reviewer played it through, say so, if he only read it carefully, that needs to be made clear as well. That way if someone doesn’t feel the review has credibility because of the treatment, it can be disregarded.
My counter to Goodman’s argument is this: Let’s Read the Dungeon Crawl Classics.
Counterpoint 1: These reviews wouldn’t exist at all if the reviewer had to play through each DCC beforehand. It is not easy to find reviews or ratings of every DCC on the Internet.
Counterpoint 2: These reviews are the epitome of worthwhile. They have enough info to aid your buying decision regardless of your play style, and they’re written with enough style to be worth reading whether you buy or not.
This series alone is enough for me to say that reviewing an adventure without playing it does not make you an idiot.
Thanks for all the interesting responses so far (and for not calling us idiots.)
I wonder if the word “Review” is a bit too charged, where we might be better off breaking it down into:
Preview: Impressions of a product that is forth coming, even if we don’t have the product in hand.
First Impressions: We have the product in hand and are looking at it and reacting to it.
Review: We’ve done everything with it and here’s what we thought.
Regardless of the term we use in the title, I’m definitely going to make an effort to put in the post what the review is based on. Something I should have been doing anyway but got sloppy with.