Playtest Metrics

There doesn’t seem to be much advice — that’s discoverable advice from a few Google searches — on how to run a playtest of your shiny new RPG. As an outsider1 to this process, the prevailing attitudes seem to be

  • play it until it breaks, and
  • if you’re having fun, you’re not playtesting. Playtesting should feel like work, not fun.

The first is good advice but rather broad, and the second stems to the same school-of-hard-knocks mentality that pervades some professions — that you do not learn your job from a book, you learn from doing, being knocked back a few times, and getting stronger. And I’ve been there and done that with a lot of things, both work-wise and hobby-wise, so I’m sympathetic to this view.

Still, it doesn’t hurt to organise my thoughts — and in doing so, maybe I can avoid at least some iterative navel-gazing that arises from the “just see what works” approach. So this post is about me thinking about what I want from the game in a fairly high-level conceptual sense, and how to gauge the response of the players.

I’ve picked four (fairly obvious) axes for the performance metrics. These are

  1. Character
  2. World
  3. System
  4. Change

The axes are approximately in order of presentation — players will see character first, then world (at least, the bits they influence), then system and finally longer-term change.

To measure along these axes I’m going to ask different questions of the players, and try to get a sense of their satisfaction in the different areas. It’s not going to be easy and will probably be even harder if I try to turn those responses into measures on an objective scale. But I’m probably getting ahead of myself. Let’s just ask the questions and see what happens.

Character Questions

Is your character’s Origin (childhood, motivations, skills and experience) expressed?

Are the character’s Power Levels expressed? These include

  • Explicit powers (i.e. written down)
  • Implicit powers (i.e. inferred by writing, character, mannerism)

Is there anything which is implied about your character that should be explicit?

Is there a direct path from what the character can do as written, to what the character wants to achieve in the world?

Is the character adequately tied to the game in play?

World Questions

Do the players understand where the game is?

Do they get the Scope2 of play?

Is it clear to them what will happen if they go Outside the Boundary?2

System Questions

Do the players know what is a pass, and what is a fail?

Do they have a sense of relative ability and relative success?

Rate the system for

  • Seek time (that is how long it takes to read the dice)
  • Transparency of Results (how easy it is to translate the reading to a success or failure)
  • Malleability or Agency (how easy it is for the players to make tactical dice rolling decisions)

Change Questions

Do the players get a sense of change in the game world?

Do they feel able to affect the world and achieve change themselves? Perhaps not immediately, but could they make a change through executing a longer term plan?

———–

Cross posted to the UK RolePlayers Design Collective blog.

Footnotes

<

p>1. I say “outsider” in the terms of designing something experimental, then trying to turn it into something actually functional long-term rather than just mucking about for a session and discarding it. Done plenty of the latter.

  1. The terms Scope, Boundary and Outside are specific to my game, but I guess they could apply to any game.

Scope is the field of operations for the game to be played — for example the PC are occult investigators looking into a bizarre murder, or pirates after plunder, or modern magicians fleeing an oppressive regime.

Outside is the stuff outside the game “world”, which in my case is a city. It’s the place people don’t go, or there will be consequences. The Boundary is simply the line someone would cross to go to the Outside — it may be just a line in the dirt or it could be an obstacle.