WIXLIB

JessicaSantana5on a computer, a strong CPU is also required. This presents a significant limitation formobile AR. However, manufacturers like Qualcomm are developing processors tospecifically meet the demands of augmented reality, so a strategic MAR project shouldkeep in mind that processor limitations will be overcome in the not-too-distant future.Mobile AR means that the user can quickly enter and exit an AR experience usingequipment already on their body. A key limitation of mobile AR, however, is the size ofthe device’s screen space. While smartphone devices boast a larger screen than mostmobile phones, the display still limits the amount of the user’s real view that can beaugmented. This means that the user must focus on either a small object or one distantenough to fit in the screen. Thus users cannot request information about a building thatthey are standing next to unless they are able to either focus on a distinguishable aspect ofthe building (like a business logo) or can query a geographic information system (GIS)database for information on the GPS coordinates of the building. In addition, if the targetobject is hidden or partially obscured from the user, the AR application may not correctlyrecognize it. One potential way to overcome the mobile device’s limited display is toproject the AR experience beyond the mobile device and onto the object itself. PattieMaes and Pranav Mistry of the MIT Media Lab are researching such projector-basedaugmentation.10 As smartphones gain projection capability, projector-based AR willbecome more feasible. Regardless of the display with which the AR user interacts, thepresentation will need to be managed in a way that avoids conflicting with the user’s realview. This means that the virtual layer does not hide or otherwise obscure reality. Insome cases, such as an automobile windshield, a mismanaged augmented reality view canbe lethal.For MAR applications that seek to provide the user with information based on thegeographic coordinates of the target location, GPS functionality is also required. In orderto improve emergency calling capabilities after the September 11, 2001 attacks on theNYC World Trade Center, the Federal Communications Commission required cell phonecarriers to be able to trace cell phone calls to within 100 meters or less.11 GPS-enabledphones, including the iPhone and Android phones, are now widely available.C. Determining LocationIn order for a device to present a user with information about a location, it must obtainthe coordinates of the device’s location and then query a database for the informationassociated with those coordinates from a geographic information system. Most smartphones come equipped with GPS capability, providing you with the phone’s coordinates.To identify the coordinates of an object in the distance, AR programs calculate thelocation of the object relative to your location using a method such as triangulation. Aftercollecting the coordinates of the target object, geographic information system (GIS) based AR requires the application to have a GIS database to query. The target locationmust have information recorded in the GIS that you are querying. Many GIS exist, /sixthsenseAdomatis, Doug. “Using the GPS for People Tracking” s/tracking.php.11

JessicaSantana6there is currently no single all-inclusive repository. This means connecting yourapplication as a developer to the best database for your context or building your own GISdatabase. As you can see, GIS-based AR faces significant, but not insurmountable,hurdles. Nevertheless, it is a convenient way to obtain information about a generallocation that has information recorded in a GIS database.D. Recognizing ObjectsThe second method for collecting information for an AR system is based on patternrecognition using computer vision. Computer vision is a rapidly growing field where animage is compared to similar images in a database until the matching image and itsassociated information are identified. While GIS-based AR is based on a key-valuedictionary of coordinates and information about a coordinate, pattern-based AR requires amore processor-consuming process of first detecting an object, comparing the object withobjects in the database, and then recognizing the object in the database.To detect an object in the camera viewfinder, the AR application must use a process likeedge detection to differentiate the object from the rest of the background. Once theapplication detects the object, it must compare it with the objects on its database,meaning that the object must be recorded on the database for recognition. Registrationerror rates are high, since the tools used to detect and recognize objects are still underdevelopment. In order to recognize a large set of objects, the application would needaccess to an all-inclusive database. Companies that already enjoy an extensive database(such as Google) hold a competitive advantage in delivering image recognition software.For real-time (using video rather than still images) AR, the program must constantly sendstill-image shots of the video to the database, requiring a robust CPU.Pattern-based AR uses two primary types of patterns to relay information about an object.The most widely used, because it is the simplest, is the fiduciary marker. A fiduciarymarker serves as a point of reference for an AR object. An AR object is measured andplaced in reference to the fiduciary marker according to preset requirements in theapplication. (Figure 1.5)Figure 1.5: Fiduciary Markers1212From http://words.transmote.com/wp/flarmanager/

JessicaSantana7AR can alternatively use marker-less pattern recognition. This means that the object itselfis the marker. Applications exist that recognize simple patterns like book covers orpaintings, but more complex patterns like moving objects are much more difficult.(Figure 1.6) Shadow interrupts the pattern recognition object, so lighting plays a key rolein recognition success. The University of California at Santa Barbara’s Four Eyes Labcreatively used fingertips as a way to track objects without markers. Handy AR, as it iscalled, identifies the user’s fingertips and tracks their location, effectively turning thefingertips into a moving pattern.13 (Figure 1.7) Facial recognition software is improvingrapidly, but privacy concerns limit its application. (Figure 1.8)Figure 1.6: Pattern Recognition14Figure 1.7: Handy AR, UC Santa Barbara Four Eyes Lab1314http://ilab.cs.ucsb.edu/From Google Goggles for Android, www.google.com/mobile/goggles

JessicaSantanaFigure 1.8: Facial Recognition Software 15As you can imagine, marker-less tracking is prone to many errors in both detecting andrecognizing an object. One way that researchers have overcome this uncertainty is tobuild systems that take registration error into account by changing the display based onthe quality of alignment between the real and the virtual object. Georgia Institute ofTechnology’s Enylton Coelho developed this research, along with the OSGAR toolkit tosupport the research.16Development Environments and Tools for ARAR development environments have traditionally required a thorough mastery ofsoftware development for utilization, excluding the casual hobbyist from tinkering. In1999, Hirokazu Kato and Mark Billinghurst built ARToolkit,17 an open sourceaugmented reality development environment still in use today and the foundation ofmany newer toolkits. The introduction of ARToolkit opened the floodgates for opensource developers to hack away and build independent AR applications.18Georgia Tech’s DART, the Designer’s Augmented Reality Toolkit, is meant to assist inthe design of AR experiences and applications.19 DART is a set of extensions to AdobeDirector, a multimedia application authoring tool that uses a movie metaphor to assistusers in building a project. Despite attempts like DART to bridge the gap between15L-1 Identity Solutions, www.l1id.comCoelho, Enylton Machado. “Spatially Adaptive Augmented Reality”. Dissertation. 28-Nov-2005.17See http://www.hitl.washington.edu/artoolkit/18A comprehensive list of mobile AR development environments is available lity-resources-for.html.19See http://www.cc.gatech.edu/projects/dart/168

JessicaSantana9development and design, much work remains to create a robust design environment foraugmented reality.One of the significant obstacles preventing AR from reaching its potential is the lack oflanguage standards to manage AR components and interactions. AR architects arecurrently forced to create many fundamental objects and methods instead of focusingtheir efforts on authoring an innovative, meaningful user experience. A standard, usableAR language would mean that developers could build new programs that target differentaspects of the AR experience, customers could analyze the user’s interaction with theseaspects, and users could establish meaning around each aspect of the experience. As PeterMorville asserts, “while the Web’s architecture rests on a solid foundation of code, itsusefulness depends on the slippery slope of semantics.”20 Several attempts atstandardization exist, but none are universally accepted yet. AR Markup Language(ARML), based on Google’s Keyhole Markup Language (KML) and managed byMobilizy and Neogence Enterprise, is the most widely known. 21 ARML seeks tostandardize a language around points of interest (POIs). In the image below, you canrecognize the AR tagged data from the “ ar:” namespace. This data is viewable on anyARML-compliant browser, such as Wikitude, or a KML browser, such as Google Earth.(Figure 1.9)Figure 1.9: Augmented Reality Markup Language (ARML)2220Morville, Peter. Ambient Findability. O’Reilly Media, 2005. p. 15.Open ARML. www.openarml.org.22Advanced ARML Document, using Wikitude, taken from http://www.openarml.org/21

JessicaSantana10Another language schema, created by the Graz University of Technology, is theAugmented Presentation and Interaction Authoring Language (APRIL).23 Beyond POIs,APRIL vocabulary includes virtual objects, multi-media content, real world artifacts, anduser information that encompass more of the user experience than ARML. APRIL isdesigned in such a way to represent the AR experience as a drama, including componentssuch as setup, cast, story, behaviors, and interactions, where behaviors are the actions ofthe cast and interactions are the actions of the user. APRIL is based on XML and itsstructure can be described in either an XML Schema or a DTD.24 (Figure 1.10)Figure 1.10: Augmented Presentation and Interaction Authoring Language (APRIL)Another example of a language schema that has been authored for ubiquitous computingin general is SOUPA, the Standard Ontology for Ubiquitous and PervasiveApplications.25 SOUPA, based on Web Ontology Language (OWL)26, can be extended toincorporate augmented reality properties and classes, as Carlos Toro et al. did using23See http://studierstube.icg.tu-graz.ac.at/april/.An XML Schema defines the legal structure of an XML document, such as the document’s elements andattributes. DTDs (Document Type Definitions) are a predecessor to the XML Schema. Seehttp://www.w3schools.com/schema/schema intro.asp for more information.25Harry Chen, Filip Perich, Tim Finin, Anupam Joshi, "SOUPA: Standard Ontology for Ubiquitous andPervasive Applications," mobiquitous, pp.258-267, First Annual International Conference on Mobile andUbiquitous Systems: Networking and Services (MobiQuitous'04), 2004.26OWL is a knowledge representation language for defining Web ontologies.24

JessicaSantana11Protégé to create a User, Device, Knowledge and Experience (UDKE) AR architecture.27A visualization of the device layer of the UDKE AR ontology can be seen below. (Figure1.11)Figure 1.11: AR Extension of SOUPA Core, Device LayerRegardless of their availability, AR languages will be useless without proper ARmetadata management. This need is not yet addressed.AR innovation will explode once AR design and development tools become accessible tothe layperson. Unfortunately, AR development environments are not easily readablebeyond the core group of developers who have been building the environment. AR itselfis not well known to the design community, who still views it as a technical uncertainty.Beyond awareness of AR as a tool, designers and other architects need to understand ARas a totally new medium. For this to be credible, AR must be identified as a viablebusiness strategy.II. Augmented Reality Business StrategyInvestment in Augmented RealityAugmented reality provides tremendous value by revealing information about objects thatwas previously hidden or difficult to access. The technology has wide application, fromhealth services and industrial processes, to education and tourism, to entertainment andmarketing. Wherever extra information about an object is handy, AR can add value.While augmented reality has huge potential, investors are reserved in funding its growth.Some believe, short-sightedly, that it is a passing fad or too immature.28 This is far fromthe truth.27Toro, Carlos et al. “User Experience, Ambient Intelligence and Virtual Reality in an IndustrialMaintenance domain using Protégé”. 11th International Protégé Conference. June 2009.28See Robert Rice’s “Top 12 Things I’ve heard from VCs when pitching AR in 2008/9” ure-capital-and-augmented-reality.html.

JessicaSantana12While many opportunistic AR business models will come and go, the core value that ARadds to our lives will prove to be a viable, sustainable model for many others. Large firmsthat benefit from established software or hardware are casually taking advantage of therecent publicity around AR to invest in AR projects, such as Google’s Goggles orNokia’s Point and Find. These firms will be interested in acquiring relevant businessmodels, just as Amazon did with image recognition startup SnapTell. However, asChristensen asserts in “The Innovator’s Dilemma”,29 startups will be able to competeagainst these large firms by approaching seemingly unattractive markets that fly underthe large firm’s radar. This is what happened with Foursquare, the mobile socialnetworking check-in business model. Foursquare began its existence as Dodgeball,founded by Dennis Crowley, and was bought by Google in 2005. Google however did notprioritize Dodgeball, forcing the team to leave Google and expand the company on theirown.30 Google did not think Dodgeball’s market was large or profitable enough to matter.So the company, as Foursquare, was ultimately able to compete against Google.Christensen’s “emerging or insignificant market” in the context of current augmentedreality is smartphone owners. Many investors, particularly in the United States, considermobile markets too small for serious consideration. A strategic investor will observe thatworldwide smartphone ownership will approach 1.74 billion by 2013, and that the mobileaugmented reality market is expected to reach anywhere from 350 million to 732million by 2014.31 But despite MAR’s market potential, investors are waiting for thetechnology and market to ripen before they dedicate significant money to any businessmodel. The notable exception is Layar, the European MAR browser company that boastsover 80 developers who have created over 500 “layers” of information, downloaded over1.5 million times through the Android and iPhone appstores.32 Layar received 3.4million in a series A capital investment from European investors Sunstone Capital andPrime Technology Ventures.33Investors are slowly beginning to turn their eyes to mobile augmented reality as astrategic investment. At the recent AR Conference hosted by the EmergingCommunications Conference & Awards, David Blumberg of Blumberg Capital discussedwhat venture capitalists are looking for in an emerging technology investment.34 Despiteinvestors’ cold feet in giving money to a nascent market, Blumberg asserted thatcompanies should not focus on what is currently available but rather what will beavailable tomorrow. Investors want to know that the company is dedicated to theirmission, rather than seeking to make quick money off of a gimmicky business model.“VCs don’t fund science projects”, says Blumberg, “we want to know: What is the latent29Christensen, Clayton M. The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail.Harvard Business School Press. 1997.30Siegler, MG. “Dodgeball founder pegs Google in the face with Foursquare.” Venture Beat. March 10,2009. stribution-agreement/34http://arconf.com/

JessicaSantana13demand? Who will buy this?” In other words, investors who are warming up toaugmented reality are interested in business models that have a clear, proven market, arethinking strategically rather than opportunistically, and have invested in a prime userexperience.Mobile Augmented Reality in MarketingIn order to test out the guidelines identified in this paper, I worked with two MBAstudents, Leonel Olicon and Federico Marquez, at the London Business School toimplement an AR business model. We decided to pursue an AR in marketing model forthree reasons: 1) the London team had strong connections with London-based marketingfirms, 2) marketing is an identified source of revenue for a two-sided market likeaugmented reality, and 3) we knew that many others would seek a marketing-basedbusiness model and we wanted to take advantage of our academic security to explore aninnovative model that did not simply apply traditional marketing to AR.Marketing remains cautious in testing innovative media like augmented reality. Mostmarketing organizations want proven campaigns before they will invest in a new method.This is ultimately harmful for brands seeking to engage with their customers, ascompeting brands that are less risk-averse are willing to experiment with augmentedreality and gain the competitive advantage.Augmented reality has the capacity to turn marketing on its head. AR is a completely newway of fostering a relationship between the consumer, the product, and the brand. MobileAR takes advantage of many of the benefits of mobile, location-based services: it knowswhere the user is, what they are doing, what they have done, what they would like to do,and is an immediate connection between the user and the provider. As such, mobilemarketing is a sensitive game because of the personal nature of the smartphone. Imaginea marketer entering your home, unannounced and uninvited, digging through your desk,pulling out your receipt from the last pharmacy visit, and announcing to whoever isstanding nearby that you can buy a cheaper tube of antifungal cream brand X at thepharmacy three blocks from your home. Because our lives are increasingly stored on ourmobile devices, marketers must be more sensitive to the context with which they aredealing.AR takes the Internet to the next level, providing the user with an immediate interfacebetween the product and the brand. Physical space is no longer a constraint with AR.With an integrated AR experience, consumers are not only unhindered from exploring,identifying, and acquiring a product, but are also able to continue a lasting relationshipwith the brand through maintenance, support, and community-building. Unfortunately,AR is currently undervalued in marketing, with resemblance to marketing’s transitionfrom brochures to websites in the 1990s: marketers seek to apply traditional marketingmodels on a new medium. By not taking the time to explore the innovative nature of ARin marketing, as well as its unique constraints, marketers do not take full advantage of thecapability of AR.

JessicaSantana14Most AR marketing campaigns seek to push advertising to the user via screen space.Unfortunately, this model does not take the user’s perspective into account. By viewingthe application from the user’s perspective, the AR architect will be able to design anapplication that makes the user happy and leads to greater sells and customer loyalty forthe brand that sponsors the interaction.Over the course of the semester I worked with Marquez and Olicon to build an AR inmarketing application that would incorporate lessons learned from our research. Inparallel to my research, Marquez and Olicon also explored mobile augmented reality inmarketing and published their findings in a final report as their capstone project. Marquezand Olicon dedicated their time to researching the business value of MAR in marketing. Ihave incorporated their research below.Mobile Augmented R

iPhone development, but it points to good generic resources on 3-D programming, including “The Black Art of 3D Game Programming”.8 OpenGL is a tool for general graphics development. Toolkits that build 3-D functionality into OpenGL exist, such as Open Inventor. Figure