We are working with the University of Michigan Exhibit Museum of Natural History to design an augmented reality interface for an iPhone app. In a museum setting, users would view an exhibit through the iPhone camera, and the app would recognize it and overlay information about the exhibit and visualizations about its relation to other exhibits. This is a great opportunity to brainstorm creative ways to explain works of art or science without a tour guide and without the limitations of audio-only pre-recorded tours, and really define how ubiquitous computing can be designed to better inform us of our world.

What We’re Doing

We are designing an iPhone app that can be taken into museums to enhance the learning experience by providing contextual information.

Who We’re Working With

The app will be designed for use by nerdy, gadgety adults. We are collaborating with museum staff to gain context. So far we have interviewed museum-goers and staff at the UM Exhibit Museum of Natural History.

Observations and Interviews

To date, we have performed participatory observations of 4 potential users in a museum setting. Our subjects, two women and two men, were all educated adults from a variety of backgrounds: nanotechnology and art history PhD students, a non-profit database administrator with an art background, and a web app developer. Their interactions with the exhibits and each other have led us to focus on providing clearer context for exhibit contents, allowing users to filter what information they see, and on preserving the social nature of museum-going.

We have also interviewed Amy Harris, the current Director of the Exhibit Museum. Harris provided us with a sense of the mission of the museum as an informal science education institution, and is keenly interested in the museum playing a role in science literacy. She also gave us some background on how the museum currently disseminates information (through exhibits and events, largely) and discussed with us the weaknesses and strengths of the current model.

Our Focus

Context

Using an iPhone in a museum to gain context for exhibits

Using an iPhone in a museum to gain context for exhibits

Museum-goers with hand-held wireless devices are already using these devices to gain more information about museum exhibits they are looking at. One of our goals is to maximize context with supplemental information, whether it’s provided by the museum, Web content, or even other users.

Tradeoff: How much information can be shown at once will be limited by the size of the iPhone screen. Avoiding information overload will be a major hurdle.

Filtering

Small, hard-to-read signage segregated in one side of an exhibit

Small, hard-to-read signage segregated in one side of an exhibit

Information, with some exceptions, tends to be shown in a written format in museums. However, participatory observation, personal experience, and discussion with museum staff tell us that users don’t read. We have to find alternative ways to display exhibit information visually, and in spatial proximity, with overlay legends, outlines, timelines, and other visual cues. We want to give museum-goers as much control as possible over what aspects of an exhibit they want to learn more about using the device.

Tradeoff: Museum signage and labeling contains a large amount of information (that people aren’t reading). Abstraction may not be possible for some exhibits, and giving users complete control over the museum experience may leave out large amounts of information museums want to bring across to visitors.

Supporting Current Museum-Going Practices

Museum-going is a social activity. We performed participatory observation and interviews because it’s hard to not have a conversation in a museum space.

Tradeoff: People would be gathering around device instead of talking to each other. Additionally, the device will only be usable by a few people at once, and thus will not be helpful for large groups.

Our Design Pattern

Map-based information like this would be an excellent candidate for interactive display through our app

Map-based information like this would be an excellent candidate for interactive display through our app

We aim to design a display with flexible, user-controlled overlays that depends largely on spatial proximity to a given exhibit or group of exhibits.

We will be able to build on some previous work. For example, work has already been done to mock up a similar application for contextual Web search. Additionally, some contextual functionality we would wish to include is already available through an iPhone app called SnapTell, which provides product information based on a photo taken with the iPhone camera.

Future of Internet Search: Mobile version profiled on petitinvention

SnapTell profiled on Lifehacker

There is presently no platform for the iPhone to build what we are envisioning. However, we are following development of ARToolKit, an augmented reality platform, for the iPhone. ARToolKit appears to be a good candidate for developing our app at a later date.

A prototype of ARToolKit v4.4 is viewable on YouTube: