TECHNOLOGY FOR CHANGE

AR Gets Real, Real-Time, That Is

Salvatore Salamone | Date: 03-11-10 | Comments

A new technique for the real-time identification of objects in a video stream holds the promise to simplify the development of augmented reality (AR) applications.

Researchers at Oxford University have developed software making it possible to fuse real and 3-D computer-generated visuals in real-time. The software has great implications for AR (augmented reality) applications�it has the potential to greatly simplify AR development and allow the expansion of AR to new application areas.

Called PTAM (Parallel Tracking and Mapping), the software maps the images obtained using a video camera and turns the objects� surfaces into platforms for virtual objects without the need for pre-stored tags or prior knowledge of the objects� shapes and locations.

PTAM software in action (Source: University of Oxford).

PTAM potentially can help overcome limitations of current AR implementations. In particular, current apps often rely on one of two methods to identify objects.

In many AR applications created in controlled environments such as computer labs or movie studios, the size, shape and location of all objects are known in advance. A computer model of these objects is then used to support the final AR animation or video.

A second approach is being used in many mobile phone-based AR applications. Here, information such as the physical location and orientation of the phone is gathered by the AR application. It then uses this information in conjunction with an existing database to display the AR images and virtual objects on the phone�s viewing screen.

An example of this latter technique is the WIKITUDE World Browser for the iPhone 3GS. The application makes use of the iPhone�s digital compass, and overlays information about landmarks from sources such as Wikipedia, Qype and Wikitude.me.

In contrast to these two techniques, PTAM examines a video view and attempts to identify in real-time items such as a book on a desk or a building on a street. In particular, the software builds a detailed 3-D map of the objects in a video frame. It can recognize objects from clusters of features that form a �digital signature� of the items or location. As the map is built, the camera�s viewpoint and angle are calculated so that 3-D AR images can be projected into the video stream.

A demonstration of PTAM software running on an iPhone acquiring images and building 3-D models can be seen in a YouTube video here.

Within the last month, Oxford licensed the PTAM technology to AR company QderoPateo, which plans to integrate the software into mobile phone applications to provide novel advertising and other services.

AR for the masses

As venture capitalists fund AR startups and technologies such as PTAM make their way into new applications that will run on mobile phones, an unlikely source is bringing AR to a potentially much wider audience.

The United States Postal Service has developed an amazing AR application called the Virtual Box Simulator. The simulator is aimed at people who use Priority Mail to ship items. With Priority Mail and the USPS�s �If it fits, it ships� service, users pay a flat rate to send an item based on the size of the box used.

The simulator helps users determine which size USPS box to use. It requires a printer and a Webcam. To get started, you print an eagle icon image and then launch the Virtual Box Simulator on a Web-connected computer. You then place the eagle icon in front of a Webcam to trigger the AR application.

The application generates a transparent box, and by placing the object that needs to be shipped in the Webcam�s view, you can see if the item fits in that sized box. You can interact with the AR image, changing its transparency and orientation. If the item does not fit into the box you are sampling, you simply select another size box. A demo of the USPS Virtual Box Simulator can be seen here.