When is Augmented Reality the Right Choice?

Last fall, we published a paper about augmented reality, cognitive theories, and learning. I previously wrote about the design advantages of AR; these give insight into when AR is really the best choice for an interface.  The following is from our paper.  If you find it useful, be sure to check out the complete work on my website.

[Augmented reality's design] advantages can be used to decide whether augmented reality is a good interface for a particular type of learning scenario. The applications that most benefit from AR are generally those that make natural use of all four categories of advantages.

Virtual flexibility is inherently desirable in all digital tools and applications we design. Similarly, it is desirable to incorporate aspects of the invisible interface. We do not want our users to unnecessarily switch their attention between multiple artifacts, for example. We also often try to incorporate natural movements, direct manipulation, and gestures in our interfaces.

In contrast, not all applications need to make use of spatial awareness or even have an environment to align virtual objects in. For AR to be a good choice there must be a clear and meaningful relationship between virtual objects and the real world. This might mean that a virtual object is attached to an explicitly related object or location. For example, a digital label that describes the object must be spatially aligned to it to make sense. Or, the relationship may come from seamlessly integrating the virtual object into the environment. For instance, a virtual animal might be shown as though it were present in its real habitat, giving learners the opportunity to observe it. The cognitive theories above offer some suggestions as to where such relationships may be useful, such as when building new mental models or providing situated meaning to the virtual data. But if there is no good reason to associate the virtual objects with some aspect of reality, then AR is likely not best for the application.

Whether an application needs reality for free is also an important consideration. It is advantageous to make use of the real world when details found in reality are key to the application. Including reality as it is rather than building a virtual representation of it saves programming effort and reduces the risk that important details about content or behavior are left out. This can be critical when the application involves a task requiring specific behavior with real-world objects. When training for or performing surgery, for example, the exact dynamics, texture, and color of human tissue would be difficult to simulate, yet may be important to the surgeon.

Because virtual flexibility and the invisible interface represent goals we have for all applications, spatial awareness and reality for free offer the best insight into when to choose augmented reality over other options. If an application can't clearly take advantage of these, then there is likely a more suitable interface type, as is the case with abstract domains in which users do not interact with tangible, real-world objects or data. Instead, users work with virtual information (like data on a computer) or physical abstractions of reality (such as charts). When reality does not play a prominent role in the application, it is difficult to make a meaningful connection between virtual and real objects. For instance, some examples of AR artificially create a connection to reality by having users hold a specially designed card upon which a 3D model will be displayed. While the method of interaction allows an enactive approach to viewing the model, the same could be accomplished with fully digital interfaces that support natural gestures. It is not clear that augmented reality is well used for this kind of application.

On the other hand, applications designed to support learning tasks that are already centered on the real world can make good use of both reality for free and spatial awareness. Information or problem solving aids can be tied to the relevant aspects of the real environment, as for learning about car engine repair: virtual labels can identify components of the engine while visual instructions can guide the learner. It would not be as easy to do this task virtually given the physical changes made in the real world. The virtual objects have a clear connection to the engine parts in the real world. Many learning applications also benefit from these advantages when, for instance, real world context is important. For example, an application designed to teach photography could use augmentations to illustrate important concepts of composition, depth of field, and so on with visualizations overlaid on the actual scene being photographed. Though these concepts can be illustrated with photographs already taken, interacting with them in real time in the real world makes their context much clearer and helps build a much better mental model. For both these applications, AR is a strong choice.

Paper content copyright by AACE. Reprinted from the Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2012 with permission of AACE (http://www.aace.org).