AR Tagging Given a Whole New Meaning

If you've read any of my previous posts (like this, this, or this), then you probably know that I can get a little giddy when it comes to augmented reality. Thanks to my friend Haz's shared items, I have found another very cool example of having fun with augmented reality.

From the project website:

Whether walls or trains — graffiti need to be sprayed on solid, “real” backgrounds. Doesn't it?

An answer to this is provided by the “Tagged in Motion” project, which builds a bridge between real graffiti art and its virtual depiction. The centre of attention is the graffiti artist DAIM, who co-created the nextwall. Equipped with the appropriate technology, DAIM sprays graffiti into empty space. In a large hall, three cameras using Motion Capturing record DAIM's position and the movements he executes with a virtual spray can. The assimilated data is shown to him in real time in a pair of video glasses — as free-floating 3D graffiti in space. In this way he can decide how and where to apply his strokes, and via a Bluetooth controller can also determine the colours, strength of brushstrokes and textures of his work.

This extended reality thus becomes a three-dimensional graffiti canvas, on which something completely new is created: street art of the next generation!

(Image of the augmented reality setup from the project site.)

The system works using AR marker tags to depict the artist's position and virtual spray can, captured by three cameras to ensure the markers are always in view somewhere. The motion path of the spray can marker is used to position the virtual paint in the world. The results are shown to the artist (and other spectators) using a slick head-mounted display that looks to be about the size of a pair of sunglasses. (That's why the artist's position must be recorded; the 3D paint has to be rendered appropriately onto the head mounted display, and so the wearer's face's position relative to where the paint was sprayed must be known.)

Check out this video to see it in action:



I've discussed augmented reality applications for gaming as well as for geographic information systems before, but never yet for art. In fact, I must admit I hadn't really imagined the obvious potential of this technological medium for the purpose of creating something that just plain looks good. Of course, a melding of digital painting, such as can be done with Corel Painter, with the real world, so you can paint directly onto your desk for example, would definitely be very cool!

And the best part? Creating your own augmented reality system isn't even all that difficult or expensive! I recommend the new book Augmented Reality: A Practical Guide to get yourself started. All you need is a camera (even a web cam will do!) and some printed markers to get yourself started.

Google Apps Team Edition

The following message is from Google, given to you on my behalf as Carleton's Google Campus Ambassador:

You may have already heard of Google Apps Education Edition, our free set of online tools for schools. But, you may not have heard yet that we just made it easier for students to get started using these services without any set-up or hassles for your IT administrator.

With the new Google Apps Team Edition, you and your classmates have a new way to share documents, spreadsheets, presentations and calendar information online. To get started, you just need to sign up with your school email address. After confirming that you belong to your school, it's easy to invite other people and start collaborating.

If you are currently a student who has ever had to work on a group project (and let's face it, who hasn't?), then this is definitely worth checking out. It could be the solution to all the hassle you've faced when trying to keep things straight between your team members, like schedules, project tasks, and so on. Alternatively, if you are part of a student group (like a club, society, or organization), these apps could help you keep all your important info organized in one place.

If you do sign up, share with me how you used the app and whether you found it useful!

My Favorite Interesting Stories

I have recently added an "Interesting Stories" feed to my blog, on the right hand side. The stories listed here are those that I come across in Google Reader and tag as relevant to the topics I write about on in my own posts. If you read Slashdot, you have probably come across most of the stories already, but there will be a few from different sources in the mix, too. The list of stories can be accessed via a web page, or a feed, in addition to the list on my blog.

Street Level Mapping in Real Time

If you haven't checked it out before, now is the time to marvel at how cool Google's Street View maps are. Somebody from Google takes lots of video images as he drives down most of the streets in a city like New York. The images are then augmented with lines and text to indicate what various roads are called and what compass direction they go. Finally, they become available in the neat little interactive interface seen below.

(Image is a screenshot from Google Maps)

Now, Google isn't the only one to come up this type of system. I mentioned in a post last month that the University of Ottawa worked on a similar system they called NAVIRE (Virtual Navigation in Image-Based Representations of Real World Environments). In the Google version, each area between the arrows you click to proceed is static. But with NAVIRE, you can look around this area with 360 degrees of rotation freedom before clicking on to the next area. (Edit: It turns out you can do this in Google too!)

(Image from the NAVIRE project page)

Both these systems use images that are captured, processed and stored in a database for access to users later on. I imagine a system that can take any video and augment it in real time. Then a user could walk around with his GPS-equipped PDA or smart phone, aim the built-in camera at an intersection he's unsure of, and immediately get his bearings based on the information displayed over the roads he's looking at. While he's at it, he might also like to see interesting buildings around him labeled.

Why would anybody want such a thing when GPS systems can already pinpoint your location on a map and orient the map the same way you are standing? Think of it this way: GPS systems usually have maps that are similar to those found in map books; that is, they are completely rendered from a bird's eye point of view. Some systems, particularly those for driving, are able to show an artificial oblique view that is rendered from a fixed viewpoint. My hypothesis is that the divide these interfaces create between the digital and real worlds makes them less natural than an augmented reality interface would be.

To look at it more deeply, consider what happens subconsciously when looking at a bird's eye view map. You must first ensure that you are oriented in some way that allows you to associate the lines representing the roads with the real-world counterparts. Then you essentially re-project the flat information from the map onto the 3D world, taking into account the translation between the distances on the map and those between the real roads. You also have to remember the names of the roads that you are transferring into the real world. If there are several you need to locate, you may need to look back and forth between the real world and the map until you get it all straight.

The augmented version would take care of all this for you. You would just aim your camera where you need information, and the information would be displayed for you right on top of the real world. No going back and forth, no remembering, no confusion. Sounds pretty good to me!

There has been some work related to this idea done in the past. For example, a Masters student from the University of Bremen focussed his thesis on a navigation system for pedestrians. He theorized that the visual clues given by a mixed reality system would make following a walking route much easier. However, instead of augmenting actual video in real time, his system used a database of prepared augmented photos. Arrows indicated where the user was to proceed next, and the photo changed with the GPS location.

(Image from Appendix A of pedestrian navigation thesis)

Meanwhile, the folks at Nokia are researching something closer to what I described above. Their MARA (Mobile Augmented Reality Applications) project has created a prototype application that displays information about "virtual objects" such as particular buildings and even people. You can see some demonstration videos on the project page.

(Image from MARA project page)

The Nokia research is definitely quite similar to what I envision. However, they don't augment roads, and I find the tagging a bit jarring visually, so there are definitely improvements to be made. Aside from this project, however, I have not seen anything closer to what I want to do.

I have decided to focus my research for my Masters, and possibly PhD if I continue, around this street view mobile augmented reality mapping application. It is a topic that contains some interesting computer vision and graphics research, and can easily be branched out to include topics like context aware computing, usability, distributed computing, and so on. Although the first step may be research that would not require the actual development of the whole system, I do hope to eventually develop a complete prototype. Watch this space for more to come as I head down what I expect will be an exciting and interesting path.

High Tech Job Workshop

It all started a bit over two years ago. I was thinking to myself, you know, I didn't really learn the skills I needed to get a job in high tech during those co-op preparation classes I had to take. They really focussed on interview questions and resume styles that suited the majority of students present, but those students were from arts, business, and other similar programs. It was high time somebody focussed on our needs as computer science students.

So I set out to design and deliver a workshop for my fellow students, intended especially to help new co-op students get their heads in the game as they started to apply for their first jobs. I would cover technical interviews, resumes, and co-op experiences such as living out of province. And I would showcase all of these things from the perspective of the students who had done it already.

(That's me at the very first event)

Thus was born my first annual high tech job workshop. It called "Co-op Tips and Advice" due to the co-op focus and inspiration. Around 20-25 students showed up to listen to me and three other co-op students talk about their experiences, as well as our program co-op coordinator's advice. After all that talking, they were happy to be fed with some free pizza and pop.

But it didn't end there. No, as was alluded to when describing the event as the "first annual," there were more editions to come. Last year was the second workshop, which was re-branded so as to attract and benefit a wider population. It was renamed the "High Tech Job Workshop" and focussed solely on technical interviews. By being more specific, even people who went to the first event could benefit from the second.


This time around, I developed my own content. I created a set of slides that included several activities for the participants to do. At first they were a bit shy about it and didn't really know what to make of it, but once they got into it, the evening was much more enjoyable and interactive. The silly-factor of some of the activities helped them remember the key points about succeeding in interviews. Our co-op coordinator also made his appearance to tell students what he felt managers were looking for during interviews with potential co-op students. Once again, the students were rewarded with free pizza.

So what about this year? Well, now that I am no longer an undergraduate and not part of CCSS (Carleton Computer Science Society), I'm not really in the best position to be providing these workshops. It was time to pass the torch, so to speak. So I encouraged the new CCSS executive to continue with what has been a really successful event. I offered my help to get them started, but they didn't need it. They were able to take what they learned in previous years and come up with a similar, yet unique format for the evening.


This year, the event featured a talk by the CEO of a successful local start-up, Protocode. The co-op coordinator did his thing as per usual, and several students once again spoke about their experiences. Best of all, I was able to obtain some sweet Google swag for those that RSVP'ed thanks to my contact for the Google Ambassador Program! There was awesome feedback about this year's event, and I am very happy to see that my tradition will be carried on, hopefully for years to come.

Harry Potter is Gender Neutral

I recently got a great deal on the latest Harry Potter game for the Wii: Harry Potter and the Order of the Phoenix. I suppose you could call me a medium Harry Potter fan: I read the first four books before they were available to my friends after my mom brought them home from England, and have made a point to catch each movie as it comes out. Having just watched Order of the Phoenix, I was anxious to get into this game and see what it had to offer.

As I delve into the early stages of play, I want to give you my impressions of the game from the perspective of a female backseat gamer, and tell you how I think EA has implemented much of the advice provided in the book I discussed here.

(Image found at EA Games Harry Potter site)

Right from the moment that I read the user manual (yup! I actually did!), I knew that this game would provide a really good alternate reward system and thus appeal to a wider market. Besides the usual game progression, there is a special room called the Room of Rewards. You can earn discovery points as you make your way through the enormous world that is Hogwarts by flicking your wand here, playing games there, repairing statues in between. Basically, it seems that anything you can interact with in the scenery that is not directly related to the game will give you these points. The more points you have, the more surprises you can unlock. I have yet to return to the room after collecting the first set of points needed for a reward, but I can tell you that the drive to search for these things is making the game immensely more enjoyable for me.

One aspect of gender neutral design that I didn't mention much in the post referred to earlier is one that should be pretty obvious: girls don't want to play games that feature either: a) hyper-sexualized women characters with no dimension, or b) story lines that focus solely on the "damsel in distress" concept. Neither of these concepts is present in Harry Potter. In fact, as many know, Hermione is integral to Harry's success and as much a hero as any of the others. While it seems natural that the player has no choice in avatar and must play as Harry, the lack of a female choice is perhaps made up by Hermione's presence and help.

So far, the game has also done a really good job of mirroring the movie, keeping the important emotional and social aspects in the final storyline. As we have seen before, this provides the kind of stimulus that females react to more so than visual stimuli. At the beginning of the story, many a student at Hogwarts is none too impressed by Harry's supposed lie about Voldemort returning, for example, and taunt and tease him as he passes by in the hallways. I imagine that things will also become sticky between the trio of friends (Harry, Ron, and Hemione) as happens in the movie. Resolving tensions like these gets girls more involved in the story and therefore the game itself, giving them a reason to come back for more.

It is interesting to note that providing this storytelling aspect does not in any way detract from the way boys prefer to play games. For example, I'm sure there will be many a fight scene with awesome visuals and opportunities for direct conflict. If you play in easy mode, as I am, these duals will likely not be the epitome of accomplishment, but if you play in the more difficult modes, the satisfaction of beating the hardest enemies may be the only reward you need to come back for more. It goes to show that you don't have to neglect the male market to appeal to women: you can have your cake and eat it too!

These are but a few thoughts on the game as I just begin the journey. I am thrilled that I have finally found a game that I am able to play by myself and that I actually feel compelled to finish! I guess game designers really are starting to understand this whole "gender neutral" thing.

Learning OpenGL

I've neglected to learn OpenGL in the past, even though I've really wanted to. I guess when there are a million things you want to learn about, the things you need to immediately learn about take priority.

But now I have to learn it, thanks to a few school projects on the go that will require it, including (most likely) my upcoming thesis.

When I searched for OpenGL tutorials, the first hit was for Video Tutorials Rock. This website is maintained by an MIT computer science student, and was created, as far as I can tell, just for fun! He figured that video tutorials would be easier to learn from, and wasn't too satisfied with the other tutorials online anyway. I really like these tutorials -- check them out!

I hope to also have a look through the course material for a fourth year graphics course at Carleton that I was never able to take due to scheduling issues. Hopefully I can catch up with some of the basic theory.

Once I'm done with these resources, I think I will check out the Simple DirectMedia Layer (SDL) libraries, as they seem to be more standard than, say, GLUT.

If you have any advice as I embark on this graphics journey, please do leave a comment. And with that, I return to learning about animation using OpenGL...