Visualizing Cultural Patterns

The interdisciplinary project Visualizing Cultural Patterns (see also Visualizing Cultural Patterns and video) was created on the novel opportunities to studying social and cultural processes from large amounts of data about people’s cultural behavior and preferences as well as cultural assets themselves in digital form. In the same vein as Bruno Latour’s essay on digital footprints (and other sources), this project starts from the observation that:

The joint availability of (a) large cultural data sets (through the Web and digitization efforts by museums and libraries) and (b) tools already employed in the sciences to analyze big data makes feasible a new methodology for the study of cultural processes and artifacts. If humanities have typically relied on the manual analysis of a small number of cultural objects, we can now create information visualizations of large cultural data sets to discover patterns that have not been visible previously.

Lev Manovich present interface design for Cultural Analytics Research Environment running on HiperWall. Photo courtesy of Anne Helmond

Relation to my thesis: A growing number of researchers start take advantage of digital footprints . This one proposes an interdisciplinary approach with a combination of expertise in Visual Arts, Communication, Cognitive Science, and Structural Engineering. I am really intrigued by their use of computer-based analysis and visualization of large data sets and data flows used that should go beyond eye-candy mashups.

The Data-Driven Urban Computing

Whenever I introduce the domain of “urban computing” I mention three highly intertwined themes. First the design of technologies that integrate into the urban environment (still) driven by ubicomp-optimists; second the study of impact of these technologies on the city observed by social scientists and human geographers; and third the research on revealing the invisible urban dynamics led by hybrid groups of individuals with architecture, design, urban planning, GIS, social sciences or computer sciences backgrounds. Two major drivers of this this theme are the emerging and massive availability of georeferenced data and the popularization of mapping softwares (not to say GIS). Kazys Varnelis and Leah Meisterlin’s essay “The invisible city: Design in the age of intelligent maps” exemplifies very well these new opportunities for designers to describe cities in a novel ways. Yet, this text shows a confusion the novel map previously uncollectible and inaccessible data to the possibility to produce “intelligent maps” (the urban computing version of the ubicomp’s “intelligent fridge”?). It overlooks some critical considerations that the current state of “mash-ups” is still trying to figure out: 0. What part of reality the data reveal? 1. What we can do with them? 2. How to communicate them for people to acquire information (still a far stretch from “intelligent”).

My own “Let’s map and see…”

Relation to my thesis: As much as ubicomp’s vision has been driven by the emergence of mobile, wireless and sensors technologies, I notice that part of the urban computing vision are driven by the newly available data, with a “let’s map and see” trial and error approach rather than starting from human and urban considerations. Of course, my work has not been completely foreign to that, but the clear current steps are about exploring points 0. et 1. They were discussed this Spring at the Real-Time Cities Round Table.

See also Nicolas’ “Design in the age of intelligent maps”

You are a Sensor

This week’s LBS360.net podcast “You are a Sensor” discusses volunteer geographic information and “taking advantage of people doing what they do” to detect diseases, natural disasters, traffic jams, and zones of social activities.

Researchers have determined that you, even without a portable device can be an effective geographic sensor. This week we explore examples of how individuals, sometimes along with their electronic gadgets, can act as effective sensors for disease or natural disaster. Our editors share some proven techniques and explore how this type of data collection might play out in the future.

Relation to my thesis: “human as sensors” getting mainstream

Talk at Telefonica R&D

This week I gave a talk at Nuria Oliver‘s recently created Multimedia Scientific Research group within Telefonica R&D in Barcelona. In this presentation, I discussed some of my ongoing research work at MIT and Universitat Pompeu Fabra on “Exploiting digital footprints to describe urban dynamics“.

Thanks to Mauro Cherubini and Nuria Oliver for their kind invitation.

Marc Smith Illustrating Digital Traces

Extracted from Marc Smith‘s keynote presentation on “Illustrating Digital Traces: Visualizations of patterns generated by computer-mediated collective action systems” at the Third International Conference on Communities and Technologies 2007:

Traditional observation of people communication behavior on the street, from City: Rediscovering the Center by William H. Whyte (1990)

Issues with relying on explicit and implicit “reputation systems”

Activity Recommendations With Real-Time Location Data

The launch of Citysense, a real-time social navigation and nightlife discovery application for Blackberry and iPhone confirms the trend of analyzing massive amounts of real-time and historic location data from mobile devices for predictive analytics. The idea is similar to automobile GPS systems sharing and pooling current road speed conditions so that everyone can avoid congestion (see Real-Time Traffic Routing from the Comfort of Your Car). The algorithms behind Citysense indexes the active places in a city and characterized them by activity, versus proximity or demographics, to better understand the context of people behavior. In the same range, using the iPhone’s map and self-location features, as well as information about the prior activities of the user’s friends, Whrrl proposes new places to explore or activities to try. Last year, PARC developed, Magitti, a mobile application that uses a combination of cues to infer her interests. With the time of day, a person’s location, her past behaviors, and even her text messages the application suggests concerts, movies, bookstores, and restaurants.

The recent release of these social softwares is also a signal of a shift from answering “where are my friends” to “where is everybody” or “what is everybody doing” (A theme I discussed in the paper “Leveraging Urban Digital Footprints with Social Navigation and Seamful Design“). But they also raise the questios: are people interested in being entertained with real-time information and can machine learning algorithms provide solutions?

Citysense screenshot on a Blackberry

Relation to my thesis: Sense Networks, the company behind Citysense also developed Macrosense, a direct technology transfer from Sandy Pentland’s work on “Reality Mining” (see Nathan Eagle’s talk at Lift on the subject). It demonstrats the real interest in combining massive amounts of anonymous, aggregate location data to understand people dynamics (see Understanding Human Mobility Patterns). However, for powerful predictions there is still a lot of work to do in 1) understanding what motivates individuals to behave the way they do and 2) how users perceive and interact with the information and recommendation.

From Shoeboxes to Digital Footprints and Digital Shadows

The seminal report From Being Human addresses the issues around the explicit and implicit storage of our interactions and activities in public places, while shopping or on the web. The authors question how we will manage and harness the enormous digital footprints and shadows that are being created by and for everyone and cover many subjects are the core of my research.
The scale of the phenomenon

Furthermore, huge amounts of information are being recorded and stored daily about people’s behaviour, as they walk through the streets, drive their cars and use the Web. While much of this may be erased after a period of time, some is stored more permanently, about which people may be naively unaware. In 2020, it is likely that our digital footprints will be gigantic, distributed everywhere, and in all manner of places and forms.

Explicit digital footprints (emptying the shoeboxes)

The decreasing cost and increasing capacity of digital storage also goes hand-in-hand with new and cheap methods for capturing, creating and viewing digital media. The effect on our behaviour has been quite dramatic: people are taking thousands of pictures rather than hundreds each year. They no longer keep them in shoeboxes or stick them in albums but keep them as ever growing digital collections, often online. The use of Web services for photo-sharing is transforming why we take photos by reinventing what we do with them.

Implicit digital footprints (our digital shadows)

Data are also being collected on our behalf or about us for no apparent reason other than because the technology enables it – our digital shadows, if you like. Personal video recorders (PVRs) record TV programmes chosen by the viewer but also automatically store them based on the viewer’s viewing profile or other criteria. Similarly, new devices are beginning to appear, such as SenseCam (see ‘A Digital Life’, below), that can automatically capture all kinds of traces of everyday life, in the form of images, video, conversations and sounds. The same is true for GPS devices which now appear in cars, in mobile phones and even embedded into clothing. All of these are capable of producing and storing large volumes of location data about our comings and goings without any conscious effort on behalf of their owners.

Data are also being deliberately recorded about us by governments, banks and other institutions using technologies such as CCTV, ATMs and phone logging. In the UK, CCTV often generates recorded ‘feeds’ of conversations and actions, as well as logging exactly where these conversations and actions took place. Some workplaces have meeting rooms that capture the content of and activities around discussions held within them. Many public debates are recorded for posterity by editorialising CCTV: in the UK, the Houses of Parliament are captured on behalf of the nation by the BBC, for example. Most people’s financial transactions are logged too, each time a credit card is used. International phone calls from the US are routinely tapped and analysed for suspicious ‘terrorist’ topics (with advanced word-recognition software allowing interrogators to locate possible conversational threads which are then focused on more attentively).

Questions on the new challenges for how we design technologies
But digital records are merciless: a silly prank captured on a mobile phone and then uploaded to a photosharing site may haunt someone for the rest of their lives in a way it never did before.

• Will it be possible for people to delete digital memories captured by others? Now that there are digital tools that can record everything we say or do, how will this affect our own abilities and ways of remembering?
• What tools and technologies are needed to effectively manage vast quantities of personal data?
• How can the privacy and security of digital footprints be ensured to prevent misuse but at the same time allow them to be shared with others when needed?
• How do people find out about their digital footprint and what tools should be provided?

Questions of broader impact

• How should society manage the storage and access of human data ethically and responsibly?
• Will people have the right to have information removed from their digital footprints?
• What are the legal implications of a growing digital footprint that maintains a record of our present and past?
• Should people be informed of the information that is being captured about them, who has access to it and how it is being used?
• To what extent do we need to design technology that allows people both control and feedback about what kinds of data are being monitored?

Relation to my thesis: The report address the notions of implicit and explicit digital footprint (with the term digital shadows for the notion of implicit footprint) and the implications in for their management, the privacy of the people generating them and their misuse (in extension to Be Counted! Return Your Census!). However, it does not mention the benefits beyond the individual (storing for memory) and social (life sharing, coordination) interests such as better understanding societies (the massive consequences on social sciences) and cities.

Report on the Real-Time Cities Round Table

The round table on Real-Time Cities that took place last month ended up being quite a success. The aim of this event was to gather experts that influence the visions of real-time cities and discuss about the issues, promises and implications inherent to their development. About 25 scholars, practitioners and students from the fields of urban planning, social sciences, architecture, geography, cartography, computer science, interaction design, industrial design and digital media filled the room. We asked 6 main speakers (Georg Gartner, Adam Greenfield, Jonathan Raper, Carlo, Raj Singh, and Paul Torrens) from different disciplines to talk about their work and the resulting implications to real-time cities.

I have summarized the interventions and discussions into 8-pages report now available on the event web page. I mixed Andrea Vaccari‘s details transcripts with Bernd Resch and Jon Reades notes with my own recollection of thoughts generated by this afternoon. It covers the key themes presented and discussed: new information resources for cities, describe real-time dynamics of the city, smart environments (the example of wayfinding), ambient information (the example of Location-Based Services), the city as a Real-Time Control System, and the vision on the opportunities and their implications.

The introduction to the topic of the round table goes as follows:

A city is, of course, by default real-time as exemplified by the street sell of umbrellas when it starts to rain in Barcelona (Figure 1). However, people moving and acting in a city base their decisions on information that is, in most cases, not instantaneous as rain drops and not synchronized with their present time and place. In recent years, the increasing deployment of sensors and handheld electronic devices environments has reshaped these processes and impacted the urbanization of the city. In a real-time city, citizens can be aware and react to events that they can’t see in their immediate vicinity or that took place days before. While humans still set the boundaries, more and more of the critical life support systems of the city are instrumented to both sense and make sense of the world around them . Or as in the “Synchronic Society” envisioned by Bruce Sterling every object worthy of human or machine consideration generates a small history. These histories are not dusty archives locked away on ink and paper. They are informational resources, manipulable in real time .

In the literature on ubiquitous computing and urban planning, the descriptions of the real-time city often employs the terms: pulsing cloud of data, instantaneous information, seamlessness integration, empowerment of the citizens, enhancement of our perception, reveal the city as we experience it, patterns of behavior, observe and improve. They highlight the revolution in urban informatics that gets embedded in the fabric of our lives and giving us the ability to show previously invisible urban processes. Moreover, real-time data have the ability to reveal a city as a whole, instantaneously, in excruciating detail, but for the first time also alive. This information become crucial to monitor the urban system and react to its conditions instantaneously.

Presentation: The co-evolution of taxi drivers and their in-car navigation systems

Yesterday, I presented at the 2008 Association of American Geographers Annual Meeting, the preliminary results of my ethnographic study on the use, adoption, and appropriation of satellite navigation systems by taxi drivers in Barcelona (slides). The abstract of the paper “The co-evolution of taxi drivers and their in-car navigation systems” co-authored with Josep Blat goes as follows:

In recent years, the relative market success of in-car navigation systems has symbolized the emergence of location-based services for wayfinding. This market success creates the opportunity to learn from real-world use of current location-aware systems in order to inform the design of future applications. With this aim, we are using an ethnographic approach to study the different ways taxi drivers rely on their navigation system. This work describes how location technologies impact the wayfinding practices and also how practices influence the appropriation of navigation systems. This co-evolution goes from the acquisition and setup of a navigation system to mastering the system shortcomings and limitations. Next, we study the reasons upon which a driver selects among the different modes of a navigation system and the other artifacts and tools (e.g. maps, street directories, landmarks) he or she uses for location awareness and wayfinding. Moreover, we analyze the role of context in this dynamics, i.e., where and when a driver accesses location information from the system, the external supports and the surrounding environment. We present the findings that emerged from 12 interviews augmented by in-car observations within the community of taxi drivers of the city of Barcelona, Spain. This community forms a massive population of early adopters of in-car navigation systems with a strong past practice of relying on mobile technologies and maps to support their work.

Accepted: Leveraging Explicitly Disclosed Location Information to Understand Tourist Dynamics: A Case Study

My paper “Leveraging Explicitly Disclosed Location Information to Understand Tourist Dynamics: A Case Study”, co-authored with Josep Blat, Filippo Dal Fiore and Carlo Ratti, has been accepted for publication in the Journal of Location Based Services. The abstract goes as follows:

In recent years, the large deployment of mobile devices has led to a massive increase in the volume of records of where people have been and when they were there. The analysis of these spatio-temporal data can supply high-level human behavior information valuable to urban planners, local authorities, and designer of location-based services. In this paper, we describe our approach to collect and analyze the history of physical presence of tourists from the digital footprints they publicly disclose on the web. Our work takes place in the Province of Florence in Italy, where the insights on the visitors’ flows and on the nationalities of the tourists who do not sleep in town has been limited to information from survey-based hotel and museums frequentation. In fact, most local authorities in the world must face this dearth of data on tourist dynamics. In this case study, we used a corpus of geographically referenced photos taken in the province by 4280 photographers over a period of 2 years. Based on the disclosure of the location of the photos, we design geovisualizations to reveal the tourist concentration and spatio-temporal flows. Our initial results provide insights on the density of tourists, the points of interests they visit as well as the most common trajectories they follow.

The reviews validate the direction of my theis. They ask me to explore how the insights insight gained from this project can be transferred to other user groups and compare the outcome with the available tourist services. They propose to continue exploring the issues around the quality of the data (i.e. how to reduce the uncerntainty in a bit detailed manner). Finally, I make a case that the visualization validate my hypothesis, but I could also point out the anomalies or unexpected behavior patterns (journalists somehow requested similar outcomes). And, yeah, not to forget the encouraging comment… “This is an excellent paper, covering a very timely and interesting topic“.

Data flow, from data recording, retrieving, storing to the visualizations.