Perspective and Spatial Relationships for Interface Design

Did you ever notice the moment we went from having a maximum of two monitors on our desktop side-by-side to having multiple displays, such as monitors and mobile devices, sitting on our desk in less static configurations? In some cases, we may additionally have the opportunity to interact with large wall displays and collaborative tabletop surfaces.  But these displays are usually pretty disconnected from each other, and even when they are used together, there's no easy way to configure them.

Photo from Dr. Nacenta's personal webpage

It turns out that “small changes in basic interaction issues can have effects on interface use,” and “perspective affects input and output.”  This was the main premise of HCI researcher Dr. Miguel A. Nacenta's talk held at Carleton yesterday.

Miguel has researched some interesting aspects of tabletop use in the past.  For instance, some of his experiments determined what kind of interactions results in the most conflicts (i.e. grabbing the same virtual item) between multiple users.  What happens at an input level really affects how users behave.

The main project he described stemmed from the fact that technologies really haven’t caught up with the view space that we have between multiple screens, smart phones, and so on.  For example, where you are sitting relative to screens not placed on the same plane determines which one appears to the left, and which to the right.  Even worse, if you use a cursor that can move between, say, a tabletop display and a regular monitor sitting on top of it, the jump between the physical boundaries will be very obvious.  The perspective of the cursor on the tabletop is very different from on the monitor.

Enter the "perspective cursor." Instead of moving in the separate coordinate spaces of the two planar surfaces, the cursor moves in a spherical coordinate space that can be reprojected appropriately to each display. This results in seamless movement between the displays.  If the user's head is additionally tracked, the perspective can be corrected based on where they are looking.

Photo from Dr. Nacenta's personal webpage

There are all kinds of interesting things you can do to correct display perspective once you start tracking users' viewpoints, and this tracking has never been easier with hardware like the Kinect.  Check out Miguel's research and publications to see some of the other cool projects he's been working on.