• Critical-Hits Studios
    • Criminals Card Game
    • Sentinel Comics: the Roleplaying Game
  • Downloads & Tools
    • Critical Hits Fantasy Name Generator
    • Drinking D&D 2010
    • Drinking D&D 2011
    • Fiasco Playset: “Alma Monster”
    • MODOK’s 11 for Marvel Heroic Roleplaying
    • Refuge In Audacity RPG
    • Strange New Worlds RPG
  • Guides
    • Gamma World
    • Guide to 4e Accessories
    • Guide to Gaming DVDs
    • Skill Challenges
  • RSS Feed
  • Facebook
  • Twitter

Critical Hits

Everything tabletop gaming since 2005

  • News
  • Reviews
  • Columns
    • Dire Flailings
    • Dungeonomics
    • Musings of the Chatty DM
    • Pain of Publication
    • The Architect DM
  • Podcasts
    • Critical Hits Podcast
    • Dungeon Master Guys Podcast
  • Roleplaying Games
  • Tabletop Games
  • Game Hacks & Content
  • Video Games

Inq. of the Week: On Reviewing

September 14, 2009 by Dave

roger-ebert-thumbs-up-2Alternative titles for last week’s poll: “Blood, Sweat, and Tiers”, “Tiers of a Clown”, and “Tiers for Fiers.” 91% of you have played lowish level D&D recently, 27% in the mid to high range, and only 3% dare to go EPIC. Some interesting stories in the comments too! Whatever levels you’re playing (or not playing with something with levels at all), I hope you’re having fun with it!

Earlier, there was some controversy over a statement criticizing a review of an adventure module where the reviewer did not play through the module before reviewing it, but (as we here at Critical-Hits have done many times before) instead wrote a review based on reading it through with a critical eye. The manner in which the argument was presented lead something to be desired and caused some considerable debate, especially amongst those of us who do regular reviews on adventure modules and other products.

I would say there are two main schools of thought on the issue (which can extend to game rules as well), with plenty of gradation between them:

  • Adventures/Games need to be played to be properly reviewed. Many adventures and games play better than they read, so the only way to really give them a shot is to play them. These kinds of reviews represent everything about the product, from beginning presentation to execution.
  • Adventures/Games can be reviewed without running them. These kinds of reviews can tell you how the product is put together and constructed, and aren’t influenced by external factors (like the skill of the GM running the adventure or the group playing the game with.) Additionally, more reviews can be produced since it takes time to actually play the game/adventure in question.

Both clearly have their supporters, since this kind of debate rages often. Plus, people argue about reviews and reviewers in general all the time without even getting into the exact style. Sounds like the perfect time for a poll!

[poll id=”140″]

Your answers may guide how we do reviews. If absolutely no one wants a review based on a reading, we may stop (even though they’ll take longer to come out), and if no one cares about how a game plays and just wants first impressions, that helps us too.

Share This:

  • Tweet
  • Share on Tumblr
  • Email
  • Print

Filed Under: Board, Card, and Miniature Games, Inquisition of the Week, Roleplaying Games Tagged With: adventure modules, reviewing

About Dave

Dave "The Game" Chalker is the Editor-in-Chief and Co-Founder of Critical Hits. Since 2005, he has been bringing readers game news and advice, as well as editing nearly everything published here. He is the designer of the Origins Award-winning Get Bit!, a freelance designer and developer, son of a science fiction author, and a Master of Arts. He lives in MD with e and at least three dogs.

Comments

  1. Wyatt says

    September 14, 2009 at 8:07 pm

    I got the first vote!

    I don’t care about either extreme – I generally don’t find play reports of a game to be helpful, because what one person found fun about playing the game with his or her group (“killing those kobolds was totally sweet!”) I wouldn’t necessarily care about. What I look for in a review is talking about the book as a book. Talking about the text, the encounters, the storyline, the presentation, the product quality. The game aspect is so varied that I don’t find it useful to read about how it went for you.

    But that’s just me. So I write the review that I find palatable – a review that isn’t about and doesn’t bother with a play report, and just talks about the book. There’s also the factor of time constraints to take into account. I find it less useful to spare the time to play a module (which can take ages with your group – and playing it by yourself for a review removes whatever utility an actual playtest has, at least from my perspective).

    I don’t care about the two schools of thought or whatever. Both have their merits. Me, personally, I don’t care for the playtest reports.
    .-= Wyatt´s last blog ..Creative Skill Use In 4e =-.

  2. Jeff Greiner says

    September 14, 2009 at 8:09 pm

    I’m with you in the region of reviewing modules that I don’t run…who has the time to run every module. Now and then I’ll get to run something, but not often.

    I think someone with a critical eye and experience can pretty well get the jist of a story, can understand if something is broken, and if a module is well constructed. So long as the reviewer isn’t claiming they ran it when they didn’t and are up front about the situation then I have no problem here.

    That said, I have seen many cases where the product publisher probably shouldn’t complain about a module not being read. I often find problems, mistakes, and down right bad module design when running them that I may have easily passed over upon a critical reading. I can point to this in several occasions.
    .-= Jeff Greiner´s last blog ..The Tome Ep 115: Halfling Cleric and Pacing =-.

  3. Aaron says

    September 14, 2009 at 8:12 pm

    I voted for “The reviewer is usually fine with just reviewing the product from a read-through, but a play might help”

    In reviewing an adventure by reading it through you can pretty much see how its going to play out, but you can’t think of everything the pc’s are going to do.

    So playing through with a group might change your opinion on the adventure to some small degree.
    .-= Aaron´s last blog ..Dungeons and Dragons 4th Edition =-.

  4. deads says

    September 14, 2009 at 8:36 pm

    I think reviews just with a read-through are quite useful, though much more useful after actual play.

    Also, if opinion about something changes post-read-through review after actually playing a game, it is very useful to republish/edit the review.
    .-= deads´s last blog ..DMG2 Artifact on Facebook; Paragon Tier of Assassin on DDI =-.

  5. Graham says

    September 14, 2009 at 8:51 pm

    I think I agree 100% with Goodman Games’ response to the situation.

    They never said the review was bad or unnecessary. But many minor issues that may come out in the critical reading of an adventure won’t affect the actual play at the table.

    Note that this mainly applies to adventures, as other sourcebooks are usually designed to be used primarily away from the table.

    I think that the review in question was overly nitpicky, as most of the “problems” I read were minor details such as how long a cab ride would be, but was still a valid review.

    So, I guess to answer your question:

    Read-through reviews are great for new supplements, and work well in general. For adventures in particular, I’d prefer reviews (not play logs, as some above are saying) after actually playing/running them, but this, of course, is rarely possible.

    For new games (different systems, or whatnot), after-play reviews are pretty much necessary. Just think of all the 3e players and their sans-playing 4e reviews when the pdfs got leaked, or the many reviews of d20 Modern that decried it for crazy things (that worked great in practice) like the Wealth system. This also extends to card/board games, as a review of Say Anything or Dominion based solely on their rules pamphlets would be… unhelpful.
    .-= Graham´s last blog ..Damn you, Dave! You and your… logic… =-.

  6. Will says

    September 14, 2009 at 9:08 pm

    I picked “Other,” the last option.

    I think the reviewer just needs to make it very clear whether or not they have played the game, or have just done a read-through. That way we, as readers, can set our expectations for the review and account for that bias. Usually it is clear from the text, and a good reviewer will say something like, “Maybe this works really well in play…” or “It was really fun, but it helped that we had an awesome GM.”

  7. Aoi says

    September 15, 2009 at 12:42 am

    I think the ideal, as other people have said, is a hybrid model of sorts. First, a critical review of the module as it is written. This would give me a sense of how good the product is in and of itself, and what I’d get for the money I spend on it. Second, a playtest report, where the elements discussed in the first part are tested in play. If nothing else, this would give me a sense of how relevant the factors discussed in the first part of the review are to the actual play experience. Both elements are important to me when determining whether I’m going to pay attention to or ultimately buy an adventure.

  8. Jack99 says

    September 15, 2009 at 2:13 am

    I think every time a publisher chooses to go out and “discuss” the reviews of his products, he has already lost. Even if he is right and the review is flawed.
    .-= Jack99´s last blog ..No more Ultrashort Reviews =-.

  9. greywulf says

    September 15, 2009 at 6:00 am

    I chose “Other” too.

    What should be important is that the reviewer discloses whether it’s an actual-play review or not. Then it’s up to the reader to decide how much weight to give to it.
    .-= greywulf´s last blog ..With the wind in her sails =-.

  10. Mike says

    September 15, 2009 at 6:08 am

    I snapped to the judgement that Goodman was right at first. I’m thinking that a lot of the 4e material can’t really be judged until we see it running at the table. How many of the original Monster Manual reviews talked about the problems with solos and minions? Would this really have changed anything? For some of these sourcebooks, we might not really know how good they are until we’ve had it for a while. For adventures, we might not know how good it is until we run it but how much of that is the material an how much of it is the dm? These things are so subjective anyway.

    The value of reviews is another question. I read the Critical Hits review of the DMG2 but there was no doubt that I’m going to buy it anyway. Unless the thing was an absolute mess, I doubt any review would sway me. I feel the same way for most of the 4e books. I usually know whether I will buy it or not based on the title alone. So much of this game isn’t based in the books anyway but on how we use them. Unlike movies or video games, I don’t just consume these books, I use them to create my own games.

    Anyone can review anything they want. This is the net. We all have a voice now. As a reader, I will pay more attention to the authors I trust and ignore those I don’t. Playtest reports are more useful to me than readthroughs but I can’t expect this alll the time.
    .-= Mike´s last blog ..Three Acting Tips for D&D =-.

  11. Bartoneus says

    September 15, 2009 at 7:46 am

    Considering I’ve reviewed modules here without running them first, I think we know where I stand on the subject. The facts of the matter to me are that mostly DMs/GMs buy modules, and we then have to read the module, so if it doesn’t read well then you’ve just bought something that you don’t even want to run. For me this discussion only effects modules that are poorly written which somehow then play out very well, which I view as being a small subset of a larger whole.

    I will just add that the whole “controversy” and discussion came about from Unclebear because Goodman tweeted at the same time as he was responding to the review calling the reviewer an idiot and referencing movies being reviewed based on a script. Aside from a lot of problems I have with comparing movies to published adventures, I think a lot of this is just a reaction to the personal connection that Twitter allows many of us to have. Goodman seems to have handled his responses online decently, if not a bit confrontational, but his tweets on the subject revealed what we can assume are his personal feelings on the subject that are a little bit concerning.

  12. Graham says

    September 15, 2009 at 12:05 pm

    @Bartoneus –

    Actually, you may have hit it with your comment in the Giants thread.

    Use the video game review terms!

    Did you play it? No? Then call it a Preview.

    Yes? Then it’s a Review.

    (It’s probably also been out for a while if you’ve had time to play it, while the non-played reviews tend to be at or before release.)
    .-= Graham´s last blog ..CriticalAnkleBites and ChattyDM pretend to be journalists =-.

  13. Bartoneus says

    September 15, 2009 at 12:08 pm

    @Graham: That’s not a bad idea, though I called my E1: Death’s Reach article a preview because I posted it several weeks before the module was released. 🙂

  14. Ken says

    September 15, 2009 at 4:30 pm

    I think Goodman had a decent point, but of course made it in an inflammatory manner which is usually the best way to generate discussion. 😉

    On the practical side, of course you can’t play through everything before reviewing it (anyone managed to play all PHB1 classes and races from level 1 to 30 yet and can finally give us a review?). Plus there is plenty you can tell from just reading through a product. In fact, most of the useful information I get from a product is a run down of the what’s included since I don’t get much chance to go to my FLGS and flip through the book physically myself.

    However, that being said, playing with something usually reveals even more information about something that might not be obvious to a mortal human just reading something through. So, when feasible, I find reviews based on actual play to be very valuable (and certainly not just play reports, there’s a world of difference). I’ve been gaming for decades and I’m still surprised at the difference in how something reads and how it plays at times.

    As long as reviews are clear when they have actually played through a product, I’m fine with both kinds and realize that for purely practical reasons, there will be far more “I just read it” reviews than not.

  15. Ken of Ghastria says

    September 15, 2009 at 5:25 pm

    First off, full disclosure: I’m a frequent freelancing contributor to Goodman Games, and have had many conversations with Joseph but do not presume in any way to speak for him.

    That said, I voted for “The reviewer should usually have played the product in question, but some exceptions are OK.” As an editor and blogger myself, I know that Time is a harsh beeeyotch, and sometimes things must be rushed either for the sake of deadlines or for the sake of being timely. But to have the default position that an RPG adventure can be thoroughly vetted and reviewed without playing it is akin to analyzing a recipe for a new dessert without finding out if it tastes yummy. A review that includes some degree of playtesting, even if it’s just sections of the module, is likely to produce a more proper appraisal of the product’s intent.

    At the very least, then — as Will and others have said — the reviewer should make it very clear upfront that the review does not include playtesting.

    Would I have used the same words on Twitter as Joseph? Probably not. But then, as the other Ken above implied, would we be having this valuable discussion if he hadn’t? 😉
    .-= Ken of Ghastria´s last blog .."Solomon Kane" movie looks NICE! =-.

  16. ScottM says

    September 15, 2009 at 6:43 pm

    As long as you’re clear about how you formed your opinion, either style is fine. I put more weight on playtested modules– but I’m very happy to hear “this is logically organized and worth investigating” without a long wait.

  17. Taellosse says

    September 15, 2009 at 11:13 pm

    I tend to think that the more experienced a player/GM the reviewer is, the more readily they can probably evaluate a product without running it through its paces in play. Regardless of the experience of the reviewer, though, a review should definitely always be prefaced with what kind of going-over the product has been given. I.e. if the reviewer played it through, say so, if he only read it carefully, that needs to be made clear as well. That way if someone doesn’t feel the review has credibility because of the treatment, it can be disregarded.

  18. Noumenon says

    September 16, 2009 at 11:04 am

    My counter to Goodman’s argument is this: Let’s Read the Dungeon Crawl Classics.

    Counterpoint 1: These reviews wouldn’t exist at all if the reviewer had to play through each DCC beforehand. It is not easy to find reviews or ratings of every DCC on the Internet.

    Counterpoint 2: These reviews are the epitome of worthwhile. They have enough info to aid your buying decision regardless of your play style, and they’re written with enough style to be worth reading whether you buy or not.

    This series alone is enough for me to say that reviewing an adventure without playing it does not make you an idiot.

  19. The Game says

    September 16, 2009 at 12:01 pm

    Thanks for all the interesting responses so far (and for not calling us idiots.)

    I wonder if the word “Review” is a bit too charged, where we might be better off breaking it down into:

    Preview: Impressions of a product that is forth coming, even if we don’t have the product in hand.
    First Impressions: We have the product in hand and are looking at it and reacting to it.
    Review: We’ve done everything with it and here’s what we thought.

    Regardless of the term we use in the title, I’m definitely going to make an effort to put in the post what the review is based on. Something I should have been doing anyway but got sloppy with.

About the Author

  • Dave

    Dave "The Game" Chalker is the Editor-in-Chief and Co-Founder of Critical Hits. Since 2005, he has been bringing readers game news and advice, as well as editing nearly everything published here. He is the designer of the Origins Award-winning Get Bit!, a freelance designer and developer, son of a science fiction author, and a Master of Arts. He lives in MD with e and at least three dogs.

    Email: dave@critical-hits.com

    Follow me:

Subscribe

RSS Feed

Archives

CC License

All articles and comments posted posted on the site (but not the products for sale) are licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License. References to trademarks and copywritten material are included for review and commentary use only and are not intended as any kind of challenge.
Mastodon

Recent Comments

  • fogus: The best things and stuff of 2024 on Remembering the Master: An Inelegant Eulogy for Kory Heath
  • Routinely Itemised: RPGs #145 on Review: The Magus
  • The Chatty DM on Review: The Magus
  • Linnaeus on Review: The Magus
  • 13th Age: Indexing Truths — Critical Hits on The Horizon Conspiracy

Contact The Staff

Critical Hits staff can be reached via the contact information on their individual staff pages and in their articles. If you want to reach our senior staff, email staff @ critical-hits.com. We get sent a lot of email, so we can't promise we'll be able to respond to everything.

Recent Posts

  • Remembering the Master: An Inelegant Eulogy for Kory Heath
  • Review: The Magus
  • Hope in the Dark Heart of Evil is Not a Plan
  • Chatty on Games #1: Dorf Romantik
  • The Infinity Current: Adventure 0

Top Posts & Pages

  • Home
  • The 5x5 Method Compendium
  • Dungeons & Dragons "Monster Manual" Preview: The Bulette!
  • Critical Hits Fantasy Name Generator
  • On Mid-Medieval Economics, Murder Hoboing and 100gp
  • "The Eversink Post Office" - An Unofficial Supplement for Swords of the Serpentine
  • Finally a manual for the rest of them!
  • Dave Chalker AKA Dave The Game
  • How to Compare Birds to Fish
  • The Incense War: a Story of Price Discovery, Mayhem, and Lust

Copyright © 2025 · News Pro Theme on Genesis Framework · WordPress · Log in