by Malcolm McCullough
Associate Professor
Taubman College of Architecture and Planning
University of Michigan
2000 Bonisteel Boulevard
Ann Arbor, MI, 48109
U.S.A.
mmmc [@] umich [dot] edu

KEYWORDS
urbanism, embodiment, ability, ontology, practice.

ABSTRACT
Phenomena such as embodiment, spatial ability, scale, and persistent physical pattern provide deep bases for a shift from universal mobility toward a more socially-centered approach to situated computing, and from merely positional media toward a semantic component of location models. The goal of this shift is to support the agility with intentions in context that characterizes rich urban life. This paper examines that change as a challenge in bottom-up cultural processes, and suggests "urban markup" as a way to understand the goals in knowledge representation.

SITUATION: NEW MEDIA URBANISM
Inscriptions have characterized almost all cities in history. Whether as grand expressions carved in stone facades, mundane signage in the streets, or the various props used by communities of practice, an information layer has shaped urban experience. Now that layer intensifies. Much as electrification did for power infrastructure a century before it, pervasive computing brings mobility, precision, personalization, and embedding to urban annotation.

The key change has been the capacity to provide, through mobile and embedded systems, information about places in those places. Urban information has traditionally been held with limited access in agencies such as tax assessors and insurance underwriters. When consumable it has been with a particular purpose built in, e.g. tourist guides published by companies with a stake in increasing automobile use. Now these constraints fall away. This change is evident in the recent rush to indexing (geocoding) many more kinds of spatial features for much more variety of use. Locative media increase access to geographic information, add flexibility in filtering and presenting it according to activity, and challenge us to annotate sites less obtrusively.

Mobile computing has obviously been key to this change. Much of the world has gained its first communications access with the handheld device. In the process, the use of geographic information systems (GIS) has spread beyond the office to the field, as well as beyond civic and environmental administration to a wider spectrum of business and social uses. Some of the deepest information design challenges in locative media involve delivering geographic information from its heavier desktop windowed forms to the lighter, thinner technologies of mobile and embedded systems.

Usual notions of "ubiquitous" computing leave the shift to the embodied world incomplete, however.

In particular, an overemphasis on mobility assumes access anytime, anyplace to a medium that is the same everywhere, and that is otherwise little changed from windows-and-pointers computing as we know it. A preference for one-size-fits-all graphical user interfaces precludes much of the development of skillfully embodied practice that invites escape beyond the desktop in the first place. The notion that any device could have access to the Internet overlooks how many devices interoperate by use of much less communications overhead. The emphasis on hardware that is stylishly worn or carried neglects its counterpart that is invisibly embedded into a site. The aggressive pursuit of distributed workplaces and electronic commerce denies the prospect, so evident among younger populations, that the primary role of the net is social and recreational [5]. The obsession with messaging treats pervasive computing as yet another attention-seeking medium in the foreground, whereas the original idea of the move beyond the desktop was to get computers out of the way, into the background.

Consider background. Interaction designers know that the ease and appeal of a task depend on its context, within information flows, within organizations and practices, and within physical and intentional settings. The ability to interpret cues from contexts becomes only more important as tasks multiply. How we manage many streams of activity and information in everyday life depends intensively on environment. Configuring these contextual cues has always been a main purpose of urban space. The way cities have formed over long periods of time is tacitly perceptible, and helps cue our instantaneous shifts of intent [4]. Much as a multilingual speaker might change languages in mid-sentence to emphasize an idea, so in any variety of circumstances you might change between work and play, socialization and solitude, production and consumption, identification and alienation, or onstage and offstage attitudes. But such ability with environment depends on the capacity to keep known and persistent cues ready in the background. It is the persistence and embodiment of the city that makes this possible. But who decides when, where, and how locative media layer onto this cultural memory device? As more and more annotation occurs, does public space, (whose decline has often been lamented lately) become enriched or further cheapened by developments in electronic media? After all, the city remains, as Lewis Mumford put it, the most possible variety in the least possible area, and as Aristotle put it, the best means toward realizing our human nature.

AGENDA: SITUATED SEMANTICS
Thus by now the main research problem in ambient and social computing has become the development of appropriate abstraction layers. Encapsulations are necessary in the use of any infrastructure — printing on your local area network for instance - so that you notice the only technology you need and receive only the information that you want. Locative media increase the need for abstractions that give scope; otherwise the number of links possible even in an ad-hoc and local setting may grow exponentially.

The problem of contextual scope is much more difficult than problems in global positioning systems (GPS). Locative media can involve many more inferences than usual GIS queries have done. This can work by site — for example you might turn off your phone while in a theater. Representing activity can become much more ambiguous than representing static features. Even within just the latter, much more needs to happen by proximity — such as that which is in view or within urbanists’ favorite measure, the five-minute walk.

Increasingly, then, the challenge of locative media is an ontology challenge [1, 2, 9, 17]. Recent work in topics such as object GIS, geographic markup language (GML), radio frequency identification (RFID) tagging, and distributed environmental sensing, to name a few, generally lead the way to the need for a shared conceptual semantic structure (which is all that ontology means in some information technology applications) for contexts. Knowledge representations might begin with position, but then extend to categories of sites, operations of context-based tasks, and more general inferences about more specific activity domains, e.g. conference room, stadium, hotel lobby, express lane [17]. The goal of "urban markup" is a way to characterize this complex problem as a bottom-up rather than top-down process - that is, like the World Wide Web, to invite many more people to become authors and producers, and offers many more filters for those who remain consumers. A bottom-up approach addresses how people play, appropriate, and occasionally resist the situations they encounter.

The ability to juggle frames of reference seems an important step toward that goal. Having a more substantial semantic abstraction of frames of reference should help in developing different cultures of mobile and embedded computing for difference domains of activity. For example, tourism has developed its own spectrum of spatial representations, and these are far more differentiate by outlook and niche than they were just a decade ago. Even without significant ontological standards and practices yet, the move toward urban markup goes beyond usual assumptions about navigation. Consider three components of this change.

- Beyond universality: Fixed and embedded objects must modify mobile interactivity locally. If the metaphor of "human cursor in the city" has any relevance, then what you "roll over" must affect what you do. Well-tuned accumulations of technology create islands of better engagement — not just islands of better bandwidth. Where you go affects how you interact.

- Beyond positioning: the GPS and WiFi (802.11) systems behind the rise of locative media are only the beginning. At least until Europe’s Galileo turns on, GPS remains notoriously prone to shadow and scatter among large buildings. WiFi systems have been demonstrated for distance triangulation where coverage is thorough enough [3]. But in some situations, it is quicker and easier to read a nearby tag than to triangulate your coordinates. Or more to the point, tags and coordinates serve complementary purposes. (Figure 2 illustrates different representations of location).

- Beyond static description: More difficult still, locative media involve tasks and domains. Commonly-held geographic representations, such as a street grid commonly found in an object GIS, provide only the top-level orientation. Locative media must somehow also recognize that context can be a dynamic production of engaged activities, and not just a preexisting arrangement of destinations [13]. For robustness, multiple streams of positions, tags, and sensor fusions might have to be corroborated. The use of tagging in locative media raises privacy issues. How much are you willing to declare what you are doing to obtain relevant data?

Meeting these challenges has the incentive of richer locative media experience: not just finding the proverbial nearby Thai restaurant; but also polling for who of your friends is nearby and hungry; not just being scanned by EZ Pass on a bridge, but also signaling an embedded guestbook on a sidewalk as a form of sport; not just wayfinding for the tourists, but also the recreational exercise of spatial and kinesthetic intelligences for the inhabitants. The latter include might street games such as capture-the-flag, environmental learning such as local plant identification, historic perspectives such as then-and-now images, everyday logistics such as food shopping, and ethnic representations such as neighborhood lore. Treating people as if they DO have a spatial mental model and favorite local artifacts has richer cultural implications than assuming that just about everybody is lost.

SUBSTRATE: EMBODIED ENGAGEMENT
Underlying this drive toward the exercise of spatial ability, the discipline of interaction design has made foundational changes in the understanding of activity in context. As reflected by so much recent emphasis on embodiment, contextual factors matter more than early researchers in interactivity had anticipated [14]. Obviously the study of purposeful activity has long risen past the creed of mechanical efficiency that once dominated machine design; for although the counting of keystrokes in goal-directed operations has provided quantitative checks and balances against too-familiar software bloat, it has not accounted well enough for the frames of reference.

More recent activity theory describes a more intrinsic unity of context, activity, and intentionality. Ethnographers remind us that experts do not follow rules so much as they play their settings. As people learn from their settings, they come to associate them with particular states of intent. This is important because it causes engagement of context to be, as Nardi [16] has best explained it, "about" something. Interfaces not only assist tasks, but also represent them. Available possibilities do not have to compete for our limited attention when they can be apprehended in known intentional contexts as background.

As Dourish [7, 8] has most thoroughly explored it, phenomenological theories argue that abstract categories are often things that need to be imposed on the world through our interactions with it and with each other, rather than things that exist within it. Context arises as an occasion of an activity, through consensus about representations, especially of embodied phenomena. Quite often those embodiments emerge intrinsically from interactions — such as on a market day in an urban plaza.

The phenomenology of embodiment has distinct expression in everyday architecture and urbanism. The proportions, image, and embellishments of the body are reflected in the proportions, image, and embellishments of buildings. Scale, which is perhaps the most essential trait in architecture, is brokered carefully by time-tested formal types in urbanism. Lobbies, courtyards, walk-ups, assembly halls, conference tables, service counters, stairways, arcades, and promenades all reflect embodiment. Changing the scale of any of these alters its reality. Maintaining and interpreting any of these over multiple human lifetimes, i.e., culturally, slowly grounds perceptivity and shapes predispositions [14].

Embodiment, culturally and philosophically interpreted, has thus become a cornerstone for the projects of pervasive computing and locative media. Perhaps the essential trait of this condition is internalization. As expressed for example in Piaget’s classic theories of developmental learning, mental representations grow more from manipulation than from mimesis — embodied interactions surpass visual perceptions in the cumulative construction of abilities [8]. Then as expressed in Gardner’s [10] classic theory of multiple intelligences, spatial and kinesthetic abilities intermingle with, for example, linguistic or logico-mathematical abilities, in a manner that rewards active shifts in frame of reference.

Accumulated environmental experience appears beneficial to the assimilation of some new activities as well as to the enjoyment of habitual ones. Arrangements of tools, props, and process ephemera become second-nature. Periphery, as the seminal version of pervasive computing so well understood it, is about creating an information context in which things are brought to deliberative attention only when necessary. Spatial mental models of well-tuned peripheries not only make activities more effective, but also can make them more satisfying. Physical and intentional frames of reference unite; a rich context represents and invites a particular practice. To interact amid a community of practice is to see the world according to the expertise held there [18].

Persistent structures, whether symbolic or physical, facilitate developmental learning of differences in protocols. The process of "construct adjustment" thought so central to this learning operates largely on frames of reference [11, 14, 16]. Spatial mental models assist many forms of activity comprehension - common examples include counting, crafting, and storytelling. Increasingly it is understood that the mental models so essential to usability are quite often spatial or situational.

The wayfinding operations so often studied by behavioralists and built as locative media applications only begin to tap embodied predispositions. Deeper experiences are available in the play of interpersonal distances, participation in the sites of knowledge communities, and in the recreational application of haptic masteries, to name a few [14].

The city’s institutions, official and unofficial, confer identity on those who take part in them, largely by means of context cues, spatial analogies, and social navigations [6]. Its persistent structures provide an effective armature not only for personal, organization and social memory, but also for intent. Engaging these through locative media presents fertile ground for cultural production.

CONCLUSION: PLAYING YOUR STREAMS
Each of us now belongs to multiple places and communities, partially, by degree [12, 14, 15]. Individually and socially, we manage our many such associations in daily life in increasingly mediated ways. Our ability to manage streams of information, goods, and services indirectly and asynchronously has become vital, and much more extensive [15]. Locative media let us combine these mediations with organizations in space. That in turn combines many senses of the word "architecture". For although the information technologies of any era presumably construct nonmaterial cultural spaces, they have generally transformed urban experience rather than abandoned it. History provides us with few instances of communication causing de-urbanization. New forms of communication more often attach as new layers to the forms and flows of the city. Interplays of social construction and technological determinism produce their richest effects there. Activity there is less mechanistic than many researchers’ models have assumed; the law of unintended consequences holds more broadly. For those who cherish cultural unpredictability, the good life remains an urban life.

CAPTIONS
Figure 1. Not just solo mobility: Storyboard project titles from author’s recent design seminars graphed by location and participation.

Figure 2. Not just coordinates: Semantic tags and process targets as components of location models.
_____________________________

REFERENCES

1. Becker, Christian and Frank Durr, "On location models for ubiquitous computing", in *Personal and Ubiquitous Computing*, Vol. 9, pp. 20-31 (2005).

2. Michael Beigl, Tobias Zimmer and Christian Decker, "A Location Model for Communicating and Processing of Context", in *Personal and Ubiquitous Computing*, Vol. 6, pp. 341–357 (2002).

3. Gaetano Borriello, Matthew Chalmers, Anthony LaMarca, and Paddy Nixon, "Delivering Real-World Ubiquitous Location Systems", in *Communications of the ACM*, Vol. 48, No. 3 (March 2005).

4. Christine Boyer, *The City of Collective Memory* (Cambridge: MIT Press, 1994).

5. Manuel Castells, *The Internet Galaxy—Reflections on the Internet, Business, and Society* (New York: Oxford University Press, 2001).

6. Mary Douglas, *How Institutions Think* (Syracuse: Syracuse University Press, 1986).

7. Paul Dourish, "What we talk about when we talk about context", in *Personal and Ubiquitous Computing*, Vol. 8, pp. 19-30 (2004).

8. Paul Dourish, *Where the Action Is* (Cambridge: MIT Press, 2001).

9. Federica Fonseca, Max Egenhofer, Clodoveu Davis, and K. Borges, "Ontologies and Knowledge Sharing in Urban GIS", in *GIS. Computer, Environment and Urban Systems*, Vol. 24 No. 3, pp.251-272 (2000).

10. Howard Gardner, *Frames of Mind — The Theory of Multiple Intelligences* (New York: Basic Books, 1983).

11. Reginald Golledge and Robert Stimson, *Spatial Behavior — A Geographic Perspective* (New York: Guilford Press, 1997).

12. Steven Graham and Simon Martin, *Splintering Urbanism—Networked Infrastructures, Technological Mobilities, and the Urban Condition* (London: Routledge, 2001).

13. Quentin Jones and others, "Putting Systems Into Place: A Qualitative Study of Design Requirements for Location-Aware Community Systems", in *ACM: Computer-Supporter Cooperative Work 2004*, Vol. 6, No. 3.

14. Malcolm McCullough, *Digital Ground—Architecture, Pervasive Computing, and Environmental Knowing* (Cambridge: MIT Press, 2004).

15. William Mitchell, *Me++ The Cyborg Self and the Networked City* (Cambridge: MIT Press, 2003).

16. Bonnie Nardi (ed.) *Context and Consciousness: Activity Theory and Human-Computer Interaction* (Cambridge: MIT Press, 1996).

17. Norbert Weißenberg, Agnès Voisard, Rüdiger Gartmann, "Using Ontologies in Personalized Mobile Applications", in *ACM GIS'04*, (2004).

18. Etienne Wenger, *Communities of Practice: Learning, Meaning, and Identity* (Cambridge, U.K.: Cambridge University Press, 1998).

AUTHOR BIOGRAPHY
Malcolm McCullough is a design educator at the University of Michigan. Prior to joining the faculty there, McCullough served at Carnegie Mellon; and for 10 years at Harvard. He has experienced Silicon Valley briefly as a product manager at early Autodesk and later as a visitor-in-residence at Xerox PARC. His 1996 book *Abstracting Craft* found an interdisciplinary audience for the creative work practices behind the digital economy, including those of artists, urbanists, environmental psychologists, digital fabricators, and usability professionals. His latest book, *Digital Ground - Architecture, Pervasive Computing, and Environmental Knowing*, offers a theory of place for interaction design.