Mobile Mixed Reality Assistant

Augmented Reality (AR) systems offer a promising novel human-computer interaction metaphor by superimposing computer-generated images on the real world and thus combining the familiarity of the physical user environment with the flexibility of virtual content. Despite their large potential as easy-to-use and attractive user interfaces, the commercial exploitation and widespread use of AR systems have been hindered by the bulky and fragile form factor and high costs of previous hardware setups. Recent advancements in cell phone technology yielding high performance, robust and lightweight handheld devices have opened up new possibilities for the development of innovative mobile user interfaces. These devices enable AR systems to step out from instrumented research laboratories and become a real-life platform for large-scale commercial applications delivering rich multimedia content for a great number of users. This "Impulse" project proposes to design and develop a Mobile Mixed Reality Assistant, which is a consumer cell phone-based software platform offering services for mobile AR applications such as sensor support for pose tracking of real world objects, rendering virtual 3D models on top of the physical environment, and interaction with virtual application elements. The platform enables a simple and economic way to embed versatile digital information such as 3D models, videos, images, URLs, or sound sequences into the real world by using black-and- white paper markers as physical anchors in the virtual world. These markers merge advantageous features of barcode technology and fiducial marker-based pose tracking. Virtual content associated with the markers is retrieved and rendered by camera-equipped cell phones, turning them into mobile information browsers for a ubiquitous augmented information space containing real and virtual objects simultaneously. The project sets up a strategic partnership between the industrial partner Verdandi GmbH and the Graz University of Technology and leverages on TU Graz`s rich research background in AR user interfaces and handheld AR technology combined with Verdandi`s commercial experience in creating interactive Virtual and Mixed Reality installations. During the entire project a big emphasis is put on the project`s commercialization aspects including the development of a suitable business model and continuous evaluation of the system in various application domains to anticipate and match the needs of potential customers. The project will provide Verdandi with significant know-how and business potential in the quickly growing domain of mobile multimedia systems, while TU Graz gains precious practical experience and feedback from real-life applications and the evaluation of research results previously tested only in laboratory environments.