Thursday, November 22, 2007

Fun With Accelerometers

Accelerometers: Teeny little chips that can be found in more places than you probably imagined. And trying to actually write code that makes use of them? You either love 'em or you hate 'em... or maybe a bit of both.

As many of you know, the Wii remote has an accelerometer built in as one of several sensors used to enhance game play. Apple's iPhone uses an accelerometer to sense how the device is tilted; that's how it's able to change the orientation of its screen based on how you are holding it. Robots can use them to keep their balance, and Nike's shoes can count your steps with them. And that's just the beginning - there are tons of applications for this simple little device!

I recently had the 'pleasure' (if you can call it that) of working on a course assignment using a Kionix accelerometer encased in a little joystick-like cover. I'm not sure what model it was; it may have even been a demo or evaluation version our teacher was able to obtain. Anyway, this is where the love-hate thing comes in. The ideas behind what you could do with these devices are really cool, but the reality kind of hurt. It's really hard to make them work!

Our assignment had us first code up some basic calculations you can do with an accelerometer. Tilt wasn't so bad to figure out. In fact, you can determine pitch and roll, rotation around the x and y axes respectively, and tilt, the angle the z-axis makes with the vertical. What you cannot compute, however, is yaw - that's why many inertial systems also use gyroscopes.


The second set of calculations to implement used integration equations to find location based on the rate of change of velocity (i.e., acceleration). We tried out several methods of doing this: Euler, Verlet, and Beeman equations. Through an experiment of our own design (I used the known properties of free fall in what I thought was a clever way), we comparatively evaluated the accuracy of these. Incidentally, I found that the Verlet equation (the velocity version for those keeping track) performed the best of all of them.

At this point I could tell that there were going to be some real limitations when trying to use the accelerometer in any practical application. The tilt calculations actually worked fairly well when measuring the orientation of the device when it wasn't moving. The two limitations of this are not being able to figure out yaw (as mentioned), and the fact that you can't tell the different between acceleration due to gravity, which is what allows you to compute pitch and roll, and actual movement in some direction.

The former is an annoyance only in the sense that in many of the demo applications I played with, yaw was the most natural direction to move the accelerometer to accomplish what I wanted! When I wanted to move a character in a game, for example, I always tried to use the yaw rotation for left and right movement.

The second limitation of distinguishing between movement and gravity can't be that bad if all these phone companies are able to implement their screen orientations functionalities so well. Still, I imagine things gets hairier when you want to use the same accelerometer to measure actual movement in addition to tilt orientation. Again, this is probably why so many systems use gyroscopes too.

Ok, so now onto the fun stuff. The second half of the assignment involved applying these calculations in fun and useful ways (can you sense the joy?). In the first question, we were to implement a simple gesture recognition system. In the second, we would use the accelerometer in some application of our choosing (or of our own creation), and think about how this would affect the users of the application.

I can tell you right now that although position information seems the most natural choice for most applications, the calculations alone are not good enough! Besides the annoyances of inaccuracies, consider this: If I start moving my accelerometer with increasing speed, the acceleration being read is a number strictly above zero. So far so good. Now I stop moving the device. Maybe there is some deceleration for a short while, but eventually I should get a reading of zero. Fine, makes sense. Except for one thing. Because you are relying on acceleration alone, you don't get any information that would tell you your velocity is actually zero now, too. No, a zero acceleration simply means a constant velocity, so unless your reading came at exactly that point in time that your velocity was also being calculated as zero (which is unlikely as it turns out), your velocity will keep on pumping out changes in position. Sigh.

So tilt orientation it is.

For the gesture recognition, I took inspiration from these mouse gestures that can be used in a web browser. My up/down movement came from changes in pitch, and the left/right from changes in roll. My very simple system just looked at the last gesture made after a predefined dead time, and tried to make some sense of it. I was able to use the system with better than 85% accuracy, and my husband was able to succeed at performing 50% of the dictated gestures (which actually isn't bad if you consider that he had only a minute of training first).

I implemented my own application for the last question rather than seek to integrate the accelerometer into something that already existed. I created a little matte chooser program because I used to work at Ross Video, and they have a matte chooser interface for their live video production switchers. I thought it would be interesting to see how the same type of interface would work when controlled with tilt instead of knobs. Basically, you get a colour wheel that you can control by tilting the accelerometer in various ways. This worked reasonably, but I think it could be a lot better with some more fine tuning.

Looking back, the assignment was kind of fun, despite the numerous problems I ran into (not all mentioned here). I don't know how well I did yet because I only just handed in the completed assignment.

There are many potential uses for the accelerometer even in just the software I use and write. But to be honest, I think I'll probably avoid the frustration in the future and just stick with my usual mouse or game pad, and wait for the industry to iron out the bugs for me!

28 comments:

Haz said...

- Wouldn't the yaw be best observed with a compass?

- If you look at the iPhones operation, I think it gets over the hiccups my waiting for stabilizing...if you shake it like crazy then return it to original orientation, it probably never changes the screen. But if it finds x seconds of movement silence, then it takes a reading and adjusts accordingly.

Gail said...

- Does a compass have to be "flat" to work properly? What about when it's upside down? Yaw has to work even if the device has the z-axis parallel to the ground or flipped completely over.

- Makes sense about the iPhone. I'm sure there's all kinds of fuzzy logic going on there. I also figured you could use the combined acceleration of the three axes and see if it equals the acceleration due to gravity. If it does, you can trust the orientation readings much more. Didn't try that out myself though.

Haz said...

- Don't see why a compass would need to be horizontal (sphere based ones like on sailing ships?). As long as you had a north reading, you could combine that with the other results to calculate the yaw, no?

- Did you get to keep the hardware?

Gail said...

- Mayhaps, mayhaps... seems like it would work...

- No, I had to return it yesterday :( I know it supposedly wasn't very expensive - maybe you could check out the website to see what they have to offer?

Jester said...

The device you used is being sold by a company called ActionXL. (actionxl.com)
Considering the work you did, I'd be happy to send you one along with some free games and demos we did for it. Let me know

Andrew said...

I'm also having fun trying to use an accelerometer in a computer science capacity but with the added pain of having a camera attached to it. From what i gathered reading your blog yaw is one reading i can't get which of course is the most important reading for my project. Is there any theory that implies it can be done?

Jester said...

You can't get yaw from an accelerometer. Epson makes an inexpensive gyro, which is about the same size as the accelerometer. Some of the PS3 controllers use 2 accelerometers to calculate yaw but you can't get it with just one.

Andrew said...

I'm actually using a wiimote at the moment. Might have a solution to the problem just not sure yet. Will look into using a gyro or hacking a ps3 controller. Could I combine two wiimotes.

Gail said...

Whoa! Jester, not sure how I missed your comment way back in December, but I guess I never got an email. I'd love to get an accelerometer and demos!

Andrew, speculation is (I'm sure somebody knows for sure) that the Wiimotes use the IR sensors to compute yaw. I once thought there was a gryo in it, but I'm not sure if that's true or not.

Andrew said...

Your right it does use IR which is a pain. I might be able to get around it though.

Haz said...

Only absolute positioning from the wii-motes is the sensor bar (2d). Rest is accelerometers.

I get to do a project this term creating an interface to view/manipulate molecular structures on a projection screen with 2 wii-motes...gonna be fun ;)

Jester said...

Gail - I just read from your profile that you are at Carleton. I guess that means you worked for Anthony Whitehead. One of his classes did some great demos, which used to be on Youtube. Were you on that team?

btw - If you send me your shipping info I can send one of the controllers out today. Or I can send it to the University. Let me know.

Jester said...

Andrew - The company that makes accelerometers has an application note on their site http://www.kionix.com that gives all the details on using 2 accelerometers to measure yaw. Check out AN19 on their Application Notes page.

Gail said...

"Only absolute positioning from the wii-motes is the sensor bar (2d)."

Yes, but I believe that information is used to determine how fast the remote is moving in its yaw direction. It seems reasonable.

"I just read from your profile that you are at Carleton. I guess that means you worked for Anthony Whitehead."

You got it. Who else would put us through such pain? :) I was not part of the group that presented the so called "Wii suit" with the accelerometers on it. Those were undergrad students I *think*, but certainly folks from engineering, not comp sci.

If you can send me an email (via my profile or portfolio) I can give you shipping info.

max123 said...

Excellent work done....
i am working on accelerometer based mouse on position sensing with double integration for getting actual x & y coordinates....
i am using 3-axis accelerometer from st electronics & successfully completed tilt sensing based mouse...
now facing problem with integration for dynamic & static acceleration seperation...
can you please explain me vernet integration bit detail which will be helpful to me so that not to use additional gyro...

Gail said...

Hey Max,

Unfortunately, my best advice to you is to simply NOT try and use a single accelerometer for absolute positioning in terms of x/y movement. :( This is because it's pretty much impossible to do well enough. One of the major reasons why is that once you stop moving the accelerometer, your acceleration may be zero, but that does not mean your equations will note a velocity of zero. Therefore your coordinates will keep changing even after you've stopped moving the device.

These people have tried to do something like you and haven't had much luck either.

Jester said...

Max - Double integration will contribute to some significant lapse rate. Once you add a gyro (if you do) the gyro error will compound exponentially. Absolute positioning appears to require an outside reference, like an IR source. I did find a cool link for a new way to do this. Check out joystiq

ao2 said...

Hi, fairly recent (and still affordable) joypads contain accelerometers _and_ gyroscopes. For instance you can get a Sony Sixaxis (the Playstation 3 controller), and read acceleration and "instant yaw" values over usb or even bluetooth. The newer Sixaxis Dualshock3 controller uses the very same Kionix ICs you're talking about.

There are efforts to make a freesoftware driver for it, and showing some sample applications (and code, yes, we want to steal your code :P) could be very useful to teach people how to use these data. So, please consider getting involved! :)

Bye, Antonio

Some links:
sixaxis data format
sixaxis driver

max said...

Thanks For Reply.
I was going thr' freescale an3397 application note in which they given entire algorithm to do it..But no practical result..

Jesse Curry said...

@Andrew

Not sure if you still keep up with the blog, but technically you can detect motion about the y-axis, the only problem is that you'll have to tilt the accelerometer forward, back, or to one side; this allows the y-axis of the accelerometer to actually be influenced by gravity.
I suppose that *really* moves you out of the realm of yaw, but it can be effective. Just be careful, if the orientation of your device brings any of the accelerometers to the point that they are perpendicular to gravity you'll lose the ability to sense tilt.

A good reference comes from freescale, the company that makes the accelerometers for the Wii-mote:

http://www.freescale.com/files/sensors/doc/app_note/AN3461.pdf

Kent said...

I am very excited about the accelerometer, specifically how it applies to character animation. The iPhone seems to be generating a lot of gaming interest in this area lately. Where do we stand with the accelerometer as it relates to 3D character animation? As a person with no computer science knowledge, can you describe, currently, what is capable? How long will it be before we can have a realistic pet in a glass box?

Gail said...

One way to think of it, Kent, is to imagine the Wii. While it uses an accelerometer, it must also use other means to get a full-fledged control system (including the infrared sensor). Most games I've played that used just the accelerometer were about simple tilting or shaking. So in that sense, what you are imagining isn't really possible without more sensors or input devices.

Kent said...

That makes sense. Is it possible to write code, let's say for the iPhone accelerometer, that reacts to time as well as movement? So, not only the direction of the tilt, but the length of time would be considered to determine the amount of movement. In return, inertia rules can be calculated. Does that make sense?

What are your thoughts about all this iPhone App excitement? Owners are so excited about the new Applications and use of the accelerometer. First the Wii, but now a common, mobile device. Are you interested in that stuff? Just curious...

Gail said...

From the post:

"I can tell you right now that although position information seems the most natural choice for most applications, the calculations alone are not good enough! Besides the annoyances of inaccuracies, consider this: If I start moving my accelerometer with increasing speed, the acceleration being read is a number strictly above zero. So far so good. Now I stop moving the device. Maybe there is some deceleration for a short while, but eventually I should get a reading of zero. Fine, makes sense. Except for one thing. Because you are relying on acceleration alone, you don't get any information that would tell you your velocity is actually zero now, too. No, a zero acceleration simply means a constant velocity, so unless your reading came at exactly that point in time that your velocity was also being calculated as zero (which is unlikely as it turns out), your velocity will keep on pumping out changes in position. Sigh."

That should answer your question. Perhaps people who are making apps for the iPhone will be/are able to overcome this. I don't know. I don't have one and won't be getting one. :)

Corry Fitz said...

Hi,

I am trying to apply accelerometer to paddling and measuring efficiency of the stroke. i see that I can buy an accelerometer on line, but what is the best interface to read and chart the data output from the accelerometer? is there software?

thanks for your help?

Joseph said...

Most accelerometer manufacturers provide demo and development boards for their accelerometers. The older boards used a serial interface, the newer boards use USB. The Kionix boards come with software that captures the accelerometer data and allows you to export it as a csv file. The same software also has an oscilloscope type display that shows you the data in real time. I believe the software is still written in Perl and uses OpenGL for the graphics display.

Phil Cheetham The 3D Guy said...

Interesting blog article, Gail. We have developed our own sensor that included accelerometer, magnetometer and gyro; we use it for golf swing analysis and training, and general human motion. It now does a good job of calculating orientation (see www.youtube.com/amm3dtv) but I want to calculate such sport parameters as "hang time", jumping time etc. Integrating the acceleration does seem problematic. Searching the web for answers.

Gail Carmichael said...

@Phil: Cool stuff. Hope you find your answers! I haven't had a chance to play with accelerometers too much since this class (which I took some time ago), but hope to soon (my PhD research could use the knowledge more than my Masters did).

Post a Comment

Comments are moderated - please be patient while I approve yours.