Taking Advantage of Emotion with Augmented Reality

I've been thinking about how augmented reality can take advantage of emotion to help people complete their tasks more efficiently. These musings were originally for a potential paper, but since this particular paper concept is going to be shelved for a while, I thought I'd share some ideas for applications I had.


Aster d'automne...!!! by Denis Collette...!!!

Many augmented reality researchers choose to define AR rather strictly, focusing on visual digital content registered to the real world in 3D.  This content could be shown with a heads-up display, a web cam as a magic mirror, or through a portable device as a magic lens.  Content can also be projected onto surfaces.

But by expanding our idea of what augmented reality is, we can be looser in thinking about the kinds of devices and technology that could be created to augment a user's perception of the world around her.  Some definitions of AR simply require that content aligns to the world in some meaningful way, but unlike the strong tie that visual augmentations often strive for, this could be as simple as location-based sounds and information.  Thinking this way takes away the distraction of not having good enough technology for the types of augmentations envisioned.

Emotion can be a “key source of motivation for driving thinking, learning, and problem solving” (Gee, Learning and Games, Ecology of Games). As Norman says in Emotional Design, “[w]hen you are in a state of positive affect, … neurotransmitters broaden the brain processing, the muscles can relax, and the brain attends to the opportunities provided by positive affect. … Positive affect arouses curiosity, engages creativity, and makes the brain into an effective learning organism.” The fact that AR allows us to provide content or information in the real world alongside real situations and tasks means we might be able to affect user’s emotions - and thus their activities - in their daily lives in a new way.

For example, consider someone learning to run. From experience, I can say that this can be a very unpleasant task (yet many of us still try because the rewards are great). What if we used AR to create an emotional sound scape that both distracts from the painful shin splints you might be developing, and takes you to various locations with a particular pace, effectively guiding you to run properly without even realizing it? Could a story be told that causes changes in the way you run, slowing you down as you look for the location of the next part, or run faster as the story intensifies?

Another type of system might use affect to keep your attention on a particular task (unlike the running example, which aimed to take your attention away from it). Warning lights in cars today might be loosely considered a form of augmented reality since they react automatically to the context of the car they are situated in. Could (or should) affect be used to build more effective warning mechanisms? How might we consider additional context beyond the car, such as weather conditions or other vehicles, and use affect to guide the driver’s attention or support his ability to problem solve very quickly in potentially dangerous situations? Could it help our driving calmness if pleasant imagery (or even sounds, for that matter) surrounded us in regular driving conditions? Could a visceral reaction to traditionally dangerous colours (possibly sharply pointing in the direction of trouble) alert us to danger sooner if there was a sudden change in the car’s surroundings?

A last example might take advantage of Norman’s reflective level of emotional design. Many have considered tourism systems that give historical or cultural information about one’s surroundings. Suppose we designed a system that took into account theory from affect to help users capture their memories in such a way that when they return to particular locations they are able to recall them vividly and fondly. In a sense, users would create a memory-scape set in real locations with sounds, images, and videos (or perhaps even haptic feedback, given the right device) that they can revisit anytime. The real trick is to not concentrate on the technology itself (which often happens in augmented reality research), but on the emotions, and how to help users capture their memories effectively.

There are several existing examples of augmented reality systems that read and react to a user's emotional state, so I've been concentrating here on systems that affect the user's emotions instead.  Have you seen any great examples of this? Could you see emotion being useful or enjoyable when used with augmented reality?