Game Papers at SIGGRAPH 2011 (Part 1 of 2)

Following up on my general impressions of SIGGRAPH I wanted to write about a few of the games paper presentations I enjoyed seeing at the conference.  I paid most attention to the first four talks that were focused on analyzing player behaviour and experience.  This post contains summaries and thoughts on the first two of these papers, and a subsequent post will cover the second two.

Evaluating enjoyment within alternate reality games
[ACM Digital Library] [Direct PDF]

This work is all about formally figuring out what makes an alternate reality game enjoyable.  As explained in the paper:

In our work, we are interested in one specific sub-genre of pervasive gaming known as Alternate Reality Games (ARGs). Layering a fictional world over the real world, ARGs provide an interactive narrative experience played out in the physical world of the player. Although a sub-genre of pervasive games, the emphasis on the narrative experience brings with it even more unique and novel considerations not normally associated with game development.

In particular, narrative is given as a key element to ARGs, unlike in pervasive games in general.  In fact, ARGs are essentially positioned as reality-based interactive story-telling engines, though not in those words: "the game should allow for non linearity in the story structure and contain key user-decision points at which the player will make choices that affect their journey through the story."

The authors propose and test five key principles that they believe are important to player experience in ARGs:

1. The game needs a strong basic story, a principle generalized from traditional narrative research.
2. Game content should be split into modular pieces based on the fact that the story is spread out over the real world.  Players should be able to reassemble the pieces in whatever order they wish.
3. The story pieces should be meaningful in the sense that they are consistent and all play a role in the formation of the game.
4. Players must be able to interact with the system, and should feel that their interactions affect the game in a meaningful way.
5. Because the skill required by the game is set by the content alone, it is important to ensure that the content is accessible to a wide audience.

To test a set of eleven metrics that come from these principles, the authors designed two games with similar plots but different narrative structures: one was designed specifically to score well on the metric while the other was designed to score low.  Seven participants consistently ranked the "good" game better than the other one in a statistically significant way.

I hope to use some of the ideas in this paper to give me a starting point in thinking about metrics for narrative in educational games (either in general or for reality-based games).  This may become one facet of my thesis, which I'm thinking will investigate the role of narrative in educational games (i.e. is it useful just for engagement, or for actually facilitating learning, and how does it do this).

Visualizing and understanding players' behavior in video games: discovering patterns and supporting aggregation and comparison
[ACM Digital Library] [Direct PDF]

The motivation for this paper comes from wanting to evaluate how people behave in virtual environments using both quantitative and qualitative methods.  The tool presented makes use of telemetry to track the actions a player takes in a game (such as how often they jumped, how long they spent with a particular non-player character, and so on) for analysis later.

Some of the challenges in designing such a tool include the fact that there is a lot of data to choose from - often terabytes.  You need to have a deep understanding of the game you want to track and analyze.  You need a clear understanding of the questions that should be asked.  Information visualization is a field of its own, and there is the trade-off between flexibility and usability.  The focus of this project was to create a visualization tool that "allows analysts to make sense of telemetry data through visualization and comparison between different player types. By interacting with our system, analysts are able to visualize player actions by cluster or aggregated over multiple clusters."

Some of the interesting features include being able to:

• cluster data, which is important since gameplay usually does not follow a normal distribution
• filter data by specifying time windows
• filter out or superimpose colour-coded event categories
• provide detail information on demand
• visualize player progression over time
• build a story on the data and understand cause and effect

The presentation itself was a nice change of pace in that the speaker demonstrated her tool without using many slides.  I'm pretty picky when it comes to what makes a "good" presentation, so this was nice to see.

In terms of my own research thoughts: it seems like ideas from this paper and the previous might combine in an interesting way.   For example, could using this kind of visualization tool with a narrative-based game help find more useful metrics? How would it help in the educational context I'm interested in?