Space, cognition, interaction 3: person and artifacts relationships

This is the third blogpost of a serie that concerns my thoughts about the topic “Space, cognition, interaction” that I address in my dissertation . Step 3 is about the person and artifacts relationships (see step 1 and step 2).

Another topic the literature about spatiality addresses is the relationship between people and artifacts located in the vicinity of the participants of a social interaction. Indeed, when a speaker talks about an object to his hearer, they are involved in a collaborative process termed referential communication (Krauss and Weinheimer, 1966). As a matter of fact, the practice of pointing, looking, touching or gesturing to indicate a nearby object mentioned in conversation is essential to human conversation; it is called deictic reference. This spatial knowledge can be used for mutual spatial orientation. Schober (1993) points out that it is easier to build mutual orientations toward a physical space (versus a shared conceptual perspective) because the addressee’s point of view is more easily identified in the physical world. There has been very little research focusing on referential communication in virtual space. Computer approach, like “What You See Is What I See” has been designed in order to support this process but studies show that such tools are not as powerful as deictic hand gestures (Newlands et al., 2002). The authors found fewer deictic acts in computer-mediated interaction; a possible reason for that can be the lack of adequate tools. Researchers, for example, attested that it is actually more difficult to see where avatars are pointing in a 3D virtual environment compared to the real world (Fraser et al., 2000). Consequently, if we think about the role of mutual location-awareness (MLA), knowing the location of others can allow one to make sense of deictic acts and promote referential communication. By projecting oneself to the known partner’s location, one can infer meaning from the deictic references.

Moreover, how the spatial environment is used in abstract cognition is a fundamental issue addressed in cognitive psychology (Kirsh and Maglio, 1994; Kirsh, 1995). These authors explain to what extent space between objects and people is used as a resource in problem solving. According to them, actions like pointing, writing things down, manipulating artifacts or arranging the positions and orientations of nearly objects are examples of how people encode the state of a process or simplify perception. Studies in virtual environments have shown similar results concerning the use of tools in space (Biocca et al., 2001). Biocca explores how people organized virtual tools in an augmented environment. Users had to repair a piece of equipment in a virtual environment. The way they used virtual tools showed patterns of simplifying perception and object manipulation (for instance by placing reference material like clipboard well within the visual field on their right). MLA should then be seen as another set of resources to augment cognitive processes such as memorization or problem solving.

What is also interesting with regard to human activity is the notion of social navigation (Dourish and Chalmers, 1994), which refers to situations in which a user’s navigation through an information space is guided and structured by the activities of others within that space. Social navigation can be defined as “navigation towards a cluster of people or navigation because other people have looked at something” (Munro et al., 1999, p. 3). This refers to the notion of “social space” inferred from the traces left in the environment (virtual orphysical) by people’s activity. As a matter of fact, we all leave signals insocial space that can be decoded by others as traces of a previous use: fingerprints, crowds, footsteps, graffiti, annotations and so on. From these cues, other persons can infer powerful things: others were here, this was popular, where can I find something, and so forth. This process takes place in both virtual and physical settings through recommender/voting systems or collaborative filtering. The most known example of such filtering is the Amazon’s recommendation system, which gives us pointers on books that may interest us based on others’ previous purchases.

References:

Biocca, F., Tang, A., Lamas, D., Gregg, J., Gai, P., & Brady, R. (2001): How do users organize virtual tools around their body in immersive virtual and augmented reality environments? Technical Report: Media Interface and Network Design Laboratories, East Lansing, MI.

Dourish, P. & Chalmers, M., (1994): Running Out of Space: Models of Information Navigation. In Proceedings of (HCI’94): Human Computer Interaction, Glasgow. New York: ACM Press.

Fraser, M., T. Glover, I. Vaghi, S. Benford, C. Greenhalgh, J. Hindmarsh and C. Heath (2000): Revealing the Realities of Collaborative Virtual Reality. Collaborative Virtual Environments. In Proceedings of Collaborative Virtual Environments (CVE 2000), San Francisco, CA, New York: ACM, pp. 29-37.

Kirsh, D., & Maglio, P. (1994). On distinguish between epistemic from pragmatic action. Cognitive Science, 18, 513-549.

Kirsh, D. (1995). The Intelligent Use of Space. Artificial Intelligence, 73(1-2), 31-68.

Krauss, R. M., & Weinheimer, S. (1966). Concurrent feedback, confirmation, and the encoding of referents in verbal communication. Journal of Personality and Social Psychology, 4(3), 343-346.

Munro, A.J., Höök, K., & Benyon, D. (1999). Footprints in the Snow. In A. Munro, K. Höök and D. Benyon (Eds.) Social Navigation of Information Space (pp.1-14). London: Springer.

Newlands, A., Anderson, A., Thomson, A., & Dickson N. (2002). Using Speech Related Gestures to Aid Referential Communication in Face-to-face and Computer-Supported Collaborative Work. In Proceedings of the First congress of the International Society for Gesture Studies, University of Texas at Austin, June 5 – 8, 2002.

Schober, M. F. (1993). Spatial perspective-taking in conversation. Cognition, 47, 1-24.

A dog reaction to an AIBO, an AIBO with fur, a remote-controlled car and a puppy

Kubinyi, E. , Miklosi, A. Kaplan, F. Gacsi, M. Topal, J. Csanyi, V. (2004) Social behaviour of dogs encountering AIBO, an animal-like robot in a neutral and in a feeding situation; Behavioural Processes, 65(3) : 231-239.

The paper is an intriguing account of applying robotics in animal behavior test. The goals of these ethologists is to determine whether the a dog-like robot of the Sony can be used to study animal interactions.

“Twenty-four adult and sixteen 4–5 months old pet dogs were tested in two situations where subjects encountered one of four different test-partners: (1) a remote controlled car; (2) an AIBO robot; (3) AIBO with a puppy-scented furry cover; and (4) a 2-month-old puppy. In the neutral situation the dog could interact freely with one of the partners for 1 min in a closed arena in the presence of its owner. In the feeding situation the encounters were started while the dog was eating food.”

The results shows that age and context influence the social behaviour of dogs. Moreover, the furry AIBO semed to evoke a higher number of responses in comparison to the car. Other aspects that I found of interest, as described by the authors:

A social partner is not only the carrier of species-specific characters to evoke behaviour on the part of the subject but also actively reacts to the actions of the other. In order to mimic interactive situations, the robot has to be able to detect and react to, at least, some elements of the environment that it shares with the tested animal
(…)
AIBO did not turn out to be a ‘real’ social partner for the dogs in all respects, but the change of its appearance, the improvement of its movements and speed could make this possible.
(…)
A further interesting question is whether puppies with experience restricted only to the robot (AIBO
“raised” dog-litters) would consider the robot as a social partner.

Why do I blog this? working on a near future laboratory project with julian, I am gathering some material about pet-technology interactions. In this article, I was interested less by the idea of using a robot for behavioral tests (ethology is not my concern) but rather about this sort of study reveal about interactions between pets and technologies.

Zoolander phone

According to the Wikipedia:

Zoolander Phone is a term often used to describe any extremely small and new mobile phone.
The term is a reference to the film “Zoolander” (Ben Stiller), in which the title character’s (played by Ben Stiller) humorously miniature cell phone is a joke on the continually smaller phones released by phone manufacturers.

(picture taken on eatliver, but this is not the real Zoolander phone, only a look-alike)

Why do I blog this? I was only looking for a picture of an extremely small cell phone to illustrate how mobile UI are tough.

Prevent people from XXXX

When affordances of object prevent people to act. In this case, this garbage in Geneva does not allow people to trash big objects (bombs? private trash?). In a sense, this is about delegating to non-humans a certain function.

Be educated by objects.

Update: look at the two other examples that are from France below: on the left, a simple piece of cardboard, on the right, a lucid/translucent trash bag. Both are interesting examples of forcing a transparent behavior. There are also rigid plastic garbage that are transparent (no picture though), all of those appeared in the VIGIPIRATE frenziness in the 90s. This led to lots of curious behavior that I don’t have time to address here (low number of trash = garbage everywhere around the one that people found…). No to mention the different way to cover/close the public trash to avoid them being filled (!) with bombs.

Areas of play

In “The space to play“, Matt Jones (Nokia Design Multimedia) interestingly describes his group work process when exploring the theme of “play”.

First, it starts with spotting some signals that “play” is a driving force (“Through weak signals found by our trends research group we had a hunch that “play” as a force in the world was becoming stronger, so we got the go-ahead for a research and design conception project“). Then, they gathered of a multi-disciplinary team (“myself, a technical consultant, Janne Jalkanen and a business consultant, Minh Tran. Our ranks were swelled by academics, independent experts, researchers and designers throughout the span of the project.“).

The team worked with user experience experts to refine the driving forces behind “play” (“One of the main components was research carried out with behavioral trend experts, Sense Worldwide. In this collaboration we identified areas of the ever-present driver of play in global culture.“) which led to set 4 relevant areas:
1) the playful engagement people have technology by hacking/modifying/tinkering things
2) the reprogramation of space through technologies (turning a metro into a gig or railway station into a pillow fight)
3) the carefully-designed space that engage people in new experiences (serendipituous meetings)
4) re-imagining the urban experience

The next step was to work on how this can fuel interaction design and mobile applications development/mobile devices design (“What would it mean to create truly playful space in our systems, services and devices? “). Matt, for that matter, describes how they wanted to go beyond user-centered design by taking into account concepts such as Csikszentmihalyi’s “flow” or examples like Parkour, Elektroplankton.

Why do I blog this? though very classic, it’s interesting to see how this work process is described and implemented. My only concern is that I would be happy to know more about how this is turned into applications/products But this is relevant:

“What does this have to do with interaction design or mobile devices? Well, as I’ve said, in play we explore, try new things and push our limits more than in any other state. The practice of experience design often tries to prescribe set paths for the end-user of the device, rather than allow the frustrations of a free exploration of the system. What would it mean to create truly playful space in our systems, services and devices? To create digital weather projects, not just thrilling but constrained slides?”

Early instances of 1st/2nd life connections and intersections

Like V-migo, Teku Teku Angel allows the user to take advantage of his/her movements in the physical environment to make grow a a virtual pet (in the form of an angel) on the Nintendo DS. It’s a pedometer that measure the daily steps and turn them into a mean to make an heavenly creature evolving.

As described on Gizmodo, Otoizm is an impressive product presented at the Toy Forum 2006 in Tokyo. Basically, connected to a music player, this device embeds a tamagotchi-like character that grows according to the genre of music you listen to and memorize phrases or compose tunes. It also has multi-user capabilities: when connected the characters dance with each other.

So, why this is important? simply, it’s another kind of application that engage users in a first life/second life experience. Unlike Teku Teku Angel that is based on a physical experience, Otoizm rather work as the intersection between digital experiences: the intricate relationship between the virtual pet and the music in the form of digital bits. What is missing is an application that would take advantage of both.

Hasbro and innovation

An article in yesterday edition of the WSJ about game company Hasbro and their innovation practices (by Carol Hymowitz). Some excerpts:

To spur innovation, Hasbro managers keep in touch with a global network of game inventors, do online surveys of customers and observe thousands of children and adults playing games developed in a new lab called GameWorks at the division’s headquarters. They also talk with prospective customers about their lives and how they want to spend leisure time
(…)
“People don’t have time to play a game for three hours, so we’re asking ourselves how we can leverage brands so they can be played in smaller time frames,” says Jill Hambley, a vice president of marketing.
(…)
Hasbro is also gunning for technology-savvy customers. Sales of videogames outpace board games by more than six to one, so Hasbro makes versions of its board games that can be played on laptops, cellphones or in video format.

Why do I blog this? no big breakthrough here but it’s interesting to understand how they work/innovate and of course the results are not surprising: small time chunk devoted for gaming (same a video game industry), use of tech to create new experience.

Space, cognition, interaction 2: Person to person relationship in space

This is the second blogpost of a serie that concerns my thoughts about the topic “Space, cognition, interaction” that I address in my dissertation . Step 2 is about the person to person relationship in space (see step 1).

A large amount of research about how spatiality shapes one’s behavior focused on co-present settings since it is the most recurrent situation of our lives. The best-known example of how space structures social interaction is proxemics: the distance between people is indeed a marker that expresses the kind of interaction that occurs, and reveals the social relationships between the interactants (Hall, 1966). Depending on the distance, Hall proposed four kinds of spheres (intimate, personal, social and public) that each affords different types of interactions. His point was also to show how theses interactions are culturally dependent and how distance constrains the types of interactions that are likely to occur. The perception of the “others” in space thus communicates to participants as well as to observers, the nature of the relationships between the interactants and their activity. Studies of 3D worlds show that proxemics are maintained in virtual environments (Jeffrey and Mark, 1998; Krikorian et al. 2000). These authors found that, even in virtual worlds, a certain social distance is kept between participants’ avatars. They noticed how spatial invasions produced anxiety-arousing behavior (like verbal responses, discomfort and overt signs of stress) with attempts to re-establish a preferred physical distance similar to the distance obverted in the physical world.

(picture courtesy of the Library of Congress, Prints and Photograph Division, FSA-OWI Collection taken as an example of how people will maintain differing degrees of distance depending on the social setting and their cultural backgrounds)

Proximity has also proved to improve various processes like conversation initiation. Communication is easier in physical settings than in mediated contexts. The physical environment increases the frequency of meetings, the likelihood of chance encounters and therefore community membership and group awareness thanks to informal conversations triggered by repeated encounters (Kraut et al., 2002). Furthermore, distance between people has an important influence on friendship formation, persuasion and perceived expertise (Latané, 1981). Latane shows that people are more likely to deceive, be less persuaded by and initially cooperate less with someone they believe to be distant.

References:

Hall, E.T. (1966). The Hidden Dimension: Man’s Use of Space in Public and Private. Garden City, N.Y.: Doubleday.

Jeffrey, P., & Mark, G. (1998). Constructing Social Spaces in Virtual Environments: A Study of Navigation and Interaction. In K. Höök, A. Munro, D. Benyon, (Eds.) Personalized and Social Navigation in Information Space, March 16-17, 1998, Stockholm (SICS Technical Report T98:02) 1998) , Stockholm: Swedish Institute of Computer Science (SICS), pp. 24-38.

Krikorian, D.H., Lee, J.S., Makana Chock T., & Harms, C. (2000). Isn’t That Spatial?: Distance and Communication in a 2-D Virtual Environment. Journal of Computer Mediated Communication, 5(4).

Kraut, R. E., Fussell, S. R., Brennan, S. E., & Siegel, J. (2002). Understanding effects of proximity on collaboration: Implications for technologies to support remote collaborative work. In P. Hinds & S. Kiesler (Eds.) Distributed Work (pp.137-162), Cambridge: MA: MIT Press.

Latané, B. (1981). The psychology of social impact. American Psychologist, 36(4), 343-356.

Flavonoid

Speaking about 1st Life and 2nd Life connections, the Flavonoid project by Near-Future Laboratory colleague Julian Bleecker is of great interest. To put it shortly, it’s a mechanism for translating embodied, kinesthetic activity into 2nd Life actions.

A homebrew, Internet-enabled kinesthetic sensor, conceptually similar to a traditional pedometer, is being designed as a networked object that bridges the geophysical worlds (1st Life) and online digitally networked worlds (2nd Life). By providing data feeds about the kinesthetic activities of the person wearing Flavonoid, various embodiments representing that data can be created in 2nd Life, such as the appearance of online avatars, or that avatar’s wealth or capabilities.

So how does it work?

The Flavonoid Kinesthometer, a wearable networkable device, can transfer data as a networked object, providing simple data feeds of one’s movement over long periods of time. This data provides a channel of RSS information used as a baseline of information that can be translated to 2nd Life representations.
(…)
Flavonoid is envisioned as a platform, using standard, open feed technologies, for a variety of embodiments. The initial embodiment being a dynamic site “badge” — a small snippet of HTML that can be embedded on virtually any site, such as one’s blog or social networking home page.

The Flavonoid project proposal gives a more thorough description of what is aimed at here.

Why do I blog this because this project takes the “Internet of Things” in a more interesting mode that what we’ve seen so far. By creating a framework for linking digital environments and the material world (“the leakage of digital networks into the physical world turns that world into a framework for a hybrid 1st Life/2nd Life”), it redefines the notion of embodiment in both environments.

This is an issue that interest me both to think about the future of ubiquitous applications and also as a user experience researcher. From a psychological point of view, there are intriguing questions to address here; especially regarding the overlap of spatial environments, their perception and how the interaction in each of them have an influence in the others.

Building a discourse about design and foresight

Currently completing my PhD program (thesis defense is next week), it gave me the occasion of looking back and think about what interest me. My original background is cognitive sciences (with a strong emphasis on psychology, psycholinguistics and what the french calls ergonomie) and the PhD will be in computer sciences/human computer interaction. In most of my work, I have been confronted to multidisciplinary/interdisciplinarity (even in my undergraduate studies).

It took me a while to understand that my interest less laid in pure cognitive science research (for example the investigation of processes such as intersubjectivity, and its relation to technologies) but rather about the potential effects of technologies on human behavior and cognitive process. In a sense this is a more applied goal, and it led me to take into account diverse theories or methods. Of course, this is challenging since mixing oil and water is often troublesome in academia. Given that my research object is embedded in space (technology goes out of the box with ubicomp) and social (technology is deployed in multi-user applications), there was indeed a need to expand from pure cogsci methods and including methods and theories from other disciplines. The most important issues regarding my work for that matter were the never-ending qualitative versus quantitative methods confrontation (I stand in-between using a combination of both, depending on the purpose) AND the situated versus mentalist approach (to put it shortly: is cognition about mind’s representation? or is it situated in context?). So, this was a kind of struggle in my phd research.

However, things do not end here. Working in parallel of my PhD as a consultant/user experience researcher for some companies (IT, videogames), I had to keep up with some demands/expectations that are often much more applied… and bound to how this research would affect NPD/design or foresight (the sort of project I work on). Hence, there was a need to have a discourse about these 2 issues: design and foresight. No matter that I was interested in both, it was not that easy to understand how the research results/methods can be turned into material for designers or foresight scenarios. SO, three years of talking with designers, developers, organizing design/foresight workshops, conferences helped a bit but I am still not clear about it (I mean I don’t even know how to draw something on paper).

Recently, I tried to clear up my mind about this and the crux issue here is the constant shifting between research and design (or foresight, sorry for putting both in the same bag here but it applies to both). The balance between research that can be reductionist (very focused problem studied, limits in generalizing or time-consuming) and design that needs a global perspective is fundamental. The other day,I had a fruitful discussion with a friend working on consumer insight projects for a big company. Coming from a cognitive science background as this friend, I was interested in his thoughts concerning how he shifted from psychology to management of innovation/design of near-future products/strategy.

I asked him about “turning points” or moments that changed his perspective. He mentioned two highlights. The first one was the paradigm shift in cognitive science in the late 80s when the notion of distributed cognition (Dcog) appeared. Dcog basically posited that cognition was rather a systemic phenomenon that concerned individuals, objects as well as the environment and not only the individual’s brain with mental representation. To him, this is an important shift because once we accept the idea that cognition/problem solving/decisions are not an individual process, it’s easier to bring social, cultural and organizational issues to the table.

The second highlight he described me is when he use to work for a user experience company that conducted international studies, he figure out that the added value not only laid in those studies but also in the cumulative knowledge they could draw out of them: the trend that emerged, the intrinsical motivation people had for using certain technologies, the moment innovation appeared. This helped him change the way he apprehended the evolution of innovations and made him question the fact that they can follows long s-curves.

Why do I blog this? random thoughts on a rainy sunday afternoon about what I am doing. This is not very structured but I am still trying to organize my thoughts about UX/design/foresight and how I handle that. I guess this is a complex problem that can be addressed by talking with people working on design/foresight/innovation. What impresses me is observing how individual’s history helps to understand how certain elements encountered shape each others’ perspective.

The picture simply exemplify the idea that conducting design/foresight projects need a constant change of focus between micro and macro perspectives. This reflects the sort of concern I am interested in by taking into account very focused perspectives (user interface, user experience, cognitive processes) and broader issues (socio-cultural elements, organizational constraints…).