Does Your Reality Need Augmenting?

Reading the news about a University of Washington professor's experimental electronic contact lens, I wondered if my dream of the ultimate personal technology has finally moved from over the horizon to in sight. Here's what I want: To be able to walk into crowded cocktail party, and know exactly who I am looking at -- names, last time we met, and other pertinent information like names of spouses or kids. I want that information beamed into my field of vision, in text floating over their heads, like the health indicators over the bad guys in a computer game.

It'll be a while, though. The contact lens just has one blinky light in it so far. But you can now get what's called augmented reality apps for the most modern of mobile phones. New smartphones that have not just cameras and location sensors (global positioning satellite receivers) but also compasses and inclinometers, so they know what direction they're pointing and which way the device is being held, can run cool sci-fi apps that show data on top of the live video of what they are looking at.

Put another way: Augmented reality apps show, as their baseline image, a video of what the camera is pointing at, a redundant display of what you can see already. Then, on top of that, they overlay data -- graphics or text -- telling you about what the camera is seeing. Until smartphones are smart enough to recognize people by their faces, the cocktail party application isn't quite possible. Although it should be noted that Google and other companies do have face recognition functions in some of their photo album services, and a Swedish company called The Astonishing Tribe has a compelling demo of such a product. There's a video here.

But here's what you can do right now with a AR-capable phone (all these apps are available in the app stores for the indicated devices):

Get Yelp on the iPhone 3GS. A secret feature (shake the phone three times to activate it when Yelp is running) overlays restaurant names and user ratings over the video you're looking at. So you can see, in theory, a review score hovering over the image of the restaurant you're standing right outside of.

Get Layar for an Android Phone. Layar is a building block AR platform that can use datasets from other providers. There's Yelp, for example (although the pure Yelp iPhone app is a bit better). Layar can also show you Twitter posts from users nearby, Wikipedia entries based on what your camera is pointing at, houses for sale from Trulia, and other info. It's incredibly cool.

Lost in a strange city? If it's New York or Londan, try AcrossAir's subway locators that'll tell you just where the station you're looking for is.

Unfortunately, for these urban apps, sometimes the orientation sensors aren't as accurate as you'd like, and the data that you see is placed erroneously. I guess it's hard to get a reliable compass reading in a modern city setting. Also, the AR apps can't tell, yet, if what the camera is seeing is relevant to what they want to display. The Yelp app will often pop a review up over a building that's located between you and the building it means to display its info over.

The most interesting and useful augmented reality app I've tried so far (but keep in mind, I'm a geek) is Google Sky Map for the Android phone. When you're out in a field at night, far from lights and magnetic interference -- and assuming you still have a data connection -- your Android phone can tell you what stars or planets you're looking at. Better yet, ask it where a body like Mars is, and it'll tell you which way to point your phone until you're looking right at it. I'd like the app to have real-time data on man-made objects like the International Space Station, but it's still an amazing product. It makes me look like a genius to my son, and that's worth the price of admission (it's a free app, so it's an easy calculation).

If you want a phone capable of running AR apps, the choices today are be the iPhone 3GS (not the older 3G, it doesn't have a compass) or an Android phone like the MyTouch 3G.

And here's a tip: Until we're all set up with cocktail-party implants, if you meet me at a party, do me a favor and introduce yourself the old-fashioned way.

By Rafe Needleman