Adaptive Methods for the Development of Interaction in Artistic Installations
Click here to download pdf version. (88kb)

by Raquel Paricio García
Technical University of Catalunya (UPC)
info [at] evolvable [dot] net 
http://www.evolvable.net

J. Manuel Moreno Aróstegui
Technical University of Catalunya (UPC)
moreno [at] eel [dot] upc [dot] edu

Keywords
 bio-inspired systems, evolvable art, multi-sensory artistic installations, adaptation, apperception, emergency

Abstract
This paper presents the implementation of evolvable hardware in artistic installations. The goal is to achieve immersive environments for the whole body where the user interaction drives the evolution of the artwork.

Most of the examples of evolvable art, starting in the 90s of the past century, have been based in evolvable software principles. In our case the major goal is to develop these applications using a hardware substrate. The “POEtic-Cubes” robotic installation will be explained as an example. It is a work in progress that is currently in its final implementation stage.

Introduction
The work presented in this paper is based in the use of mechanisms able to adapt to the environment in order to improve the actual awareness on perceptions or qualia [1].

In the POEtic-Cubes installation these mechanisms are used in autonomous elements (robots) that are able to adapt or evolve their behavior depending on the specific actions performed by the user. In this way the cooperation between these elements constitutes an artificial organism that is changing continuously.

The main difference of the POEtic-Cubes installation with respect to previous approaches lies in the fact that these bio-inspired mechanisms are actually implemented in hardware devices that operate autonomously.

In the installation the user interacts with an artificial organism whose external features are dictated by the movements of the body. As a consequence, more attention is paid to the environment, thus increasing the perception or apperception levels. Even if the robots are individual objects their interactions give rise to emergent properties that are reflected in an artificial organism whose overall behavior results from this cooperation. In this way the installation permits to perceive adaptive emergent processes, like those that appear in living organs as a consequence of the interaction of their constituent cells, in the same way a vortex is actually the result of coupled smaller whirlpools.

First we shall review previous approaches aimed at the development of adaptive artistic installations. Then the major features we consider mandatory to attain actual adaptive artworks will be presented. Thereafter we shall explain how all these principles are being applied in the POEtic-Cubes installation. Before the conclusions we shall present the POEtic tissue, the basic hardware substrate that constitutes the core of the installation.

Related Works
We can define two major categories of projects whose features are a direct antecedent of those present in the POEtic-Cubes installation: the first group is constituted by approaches that use evolutive techniques for their development and in the second we can find installations with immersive features.

The use of evolvable software techniques has its conceptual origin in the middle of the 20th century, when one of the major goals is to provide a perception of both space and time. Artists start then to pay attention especially to evolutive processes and to the change in nature. In the seventies and eighties artists like Yoichiro Kawaguchi and Karl Sims are considered pioneers in the application of artificial evolutionary algorithms to the development of infographic creations. Karl Sims defines the use of evolution in art in the following way [2]: “Evolution is a method for creating and exploring complexity that does not require human understanding of the specific process evolved”. Later in the 90s, artists like Golan Lewin or Casey Reas use evolutive principles for designing digital interfaces.

The artistic projects related to immersive spaces have experienced several stages and different technologies, ranging from the virtual reality to the extension of perceptual processes in the network. Osmose (1995) and Ephémère (1998) from Char Davies [3] are two pioneering immersive installations based on interactive environments. Their major goal is to improve the experience of embodiment in the space-time, and also to amplify emotions and feelings. The Autopoiesis project (1995) from Ken Rinaldo [4] is a robotic installation based on artificial life principles composed of 15 sculpture-robots. It is inspired in the Autopoiesis (self-making) theories described by Maturana and Varela. In this case the user interacts with a system showing evolutive and emergence aspects. However, the evolutive features are not inspired in the mechanisms that can be found in the biological realm, but results from a randomized reaction to the received stimuli. Some of the behaviors expressed in this installation, resulting from the pre-programmed low-level rules, drive the selection of a given robotic arm as a leader that is followed and imitated by the remaining ones.

It is especially with respect to this last work that the POEtic-Cubes installation demonstrates a step forward in the actual application of evolutive processes to physical installations.

Proposed Approach
Bearing in mind the results produced by previous approaches in the fields of evolutive art and immersive art works, our intention is to develop an installation model with the following features:

Immersive: It has to contribute to the development of both kinestesic experimentation models of the space and environments that are constructed from the user actions.

Adaptive: It has to change according to the specific features of the environment, so that its parts should interact between them and also with the users. The behavior of every component is influenced by the actions done by the remaining ones, thus resulting in a structure with emergent features. Furthermore it is necessary for the installation to adapt to the specific features of a given user. This may be achieved by the use of electronic devices able to support bio-inspired learning mechanisms, that is, unsupervised adaptive principles able to extract information from the temporal dynamics associated with the stimuli (the user actions) received by the system. From this information it will be possible to modulate the behavior (movement dynamics and graphical representation) of the installation, so as to fit the concrete features of the users with whom it has to interact.

Evolutive: The parameters that drive the behavior of the installation should not be static, but they may be modified in order to achieve an interaction dynamics adapted to the users. This will permit to conceive long term adaptation methods, similar to those on which the natural evolution acting on living beings is based. Actually the parameter set can be considered as the genetic contents (the genome) of the installation. Therefore, it will be possible to derive a specific fitness function with a specific objective to be optimized (energy consumption, interaction speed or physical shape of the developed phenotype, among others). The evaluation of this fitness function for each realization of the installation with a specific set of users will permit to determine the adaptation achieved. In this way, from several cycles of interaction-evaluation it will be possible to determine the realizations that are best adapted to the proposed objective. By applying evolutive mechanisms (crossover and mutation) on the genetic contents of the best solutions new parameter sets could be obtained offering more efficient alternatives for a given goal. This will result in a long-term adaptation process that will permit to discover, as it happens with the natural evolution, new possibilities that are beyond the design procedures based on a limited set of pre-defined rules.

Non-invasive: The user does not need to incorporate a specific sensor in the body. The natural features provided by the user (like temperature, position or motion pattern) drive the behavior of the installation.

Transportable: Given the context of artistic production and exhibition, it is mandatory to optimize the portability features.

Following this model an interactive multi-sensory artistic installation has been developed, called POEtic-Cubes. It is composed of nine autonomous robots whose behavior is monitored and driven by the POEtic electronic tissue [5], a new electronic substrate with bio-inspired capabilities. This installation will achieve two goals that are absent in previous approaches. First of all, it will implement the development and self-organization processes of an artificial organism, being these driven by the actions performed by the users present in the installation. Furthermore, it will support short-term (unsupervised learning) as well as long-term (mutation, crossover and fitness evaluation of the genetic content of the components of the installation) adaptation mechanisms in order to achieve a global behavior as close as possible to a specific target.

POETIC -CUBES

Description of the installation
The installation consists of nine autonomous robots constructed in the form of cubes with displays in five out of their six sides. The cubes are in a dark room, so that when the room is empty they are grouped together as a 3 x 3 array, constituting a cell. When a person or a group of people enter the room the cubes start to move and place themselves around the people (holding the same distance between them). Therefore, the cell divides itself and differentiates to create an organism. If the person or the group of people moves in the room (or even if one person moves the arms) the cubes move and the colors that are depicted in the displays change. Therefore, in this installation the people can observe how their actions determine the physical aspect of the organism (constituted by the set of nine cubes), i.e., the phenotype, being thus a clear illustration of the genotype to phenotype mapping process. Learning (epigenetic) mechanisms can also be demonstrated since the reaction of the cubes (i.e., their movement) can be modulated depending on the actions done by people. The cubes also determine autonomously the state of the battery, and upon detecting a low battery threshold they go to a specific place in the room where the battery may be charged.

The POEtic tissue
The main goal of the POEtic project was the development of a flexible hardware substrate able to provide capabilities similar to those present in living beings, like evolution, development, self-replication, self-repair and learning. One of the major outcomes of the project was an integrated programmable electronic system, the POEtic chip, that provides the features required for a physical implementation of these principles. POEtic chips may be combined to construct an electronic tissue whose size can be accommodated to the specific needs of the application to be handled.

As it happens with living beings, in the POEtic-Cubes installation it is possible to observe a genotype to phenotype mapping process, so that the epigenetic principles that result from the interaction with the environment provide for the external observable features of the organism. As the environment changes so does the organism. The hardware driving the elements constituting the installation takes into account the specific constraints posed by the environment, therefore driving the changes in the organism.

The POEtic-Cubes prototype
After completing the design and testing of the POEtic chips a prototype for the POEtic-Cubes installation is being constructed. The first stage in the development of this prototype is the construction of a complete cube with all the necessary sensors and actuators. Figure 1 (page 4) depicts the structural setup devised for this prototype. Every edge of the cube is 30cm long.

	Figure 1: Structural set-up of the POEtic-Cubes prototype Copyright © Raquel Paricio Garcia and J. Manuel Moreno Aróstegui

The cubes are totally autonomous. This means that they have all the sensors and actuators required to take proper decisions when interacting with the people present in the installation.

One of the major challenges to solve in order to have actual autonomous elements is the determination of the position of every cube in the installation.

This is achieved by means of a synchronized loop controlled by infrared and ultrasound transceivers.

Conclusions
Natural principles may constitute an important inspiration source for the development of short and long term adaptation methods able to improve interaction processes in artistic installations. In this paper we have developed the basic features of a model that would permit to advance on step towards the construction of artworks able to adapt to specific goals and also to the concrete features of their users. In this way it will be possible to achieve a deeper immersion and as a consequence an improved perception. A demonstration installation, called POEtic-Cubes, has been developed using these principles. It is composed of nine autonomous robots whose behavior can be adapted as a result of their interaction with a group of users. The prototypes that constitute the installation have been successfully built and are currently undergoing the final field tests.

Acknowledgements
Our thanks to the Departament de Cultura de la Generalitat de Catalunya for granting the research presented in this paper.

References
1. R. Paricio, J.M. Moreno, “POEtic-Cubes: Acquisition of new qualia through apperception using a bio-inspired electronic tissue” in Proceedings of the 13th ACM International Conference on Multimedia, Singapore (2005), pp. 783-789.

2. Karl Sims, “Computer Graphics”, ACM SIGGRAPH '91 Conference Proceedings, pp. 319-328, Las Vegas, Nevada, 1991.

3. http://www.immersence.com/

4. http://www.kenrinaldo.com/

5. A.M. Tyrrell, E. Sanchez, D. Floreano, G. Tempesti, D. Mange, J.M. Moreno, J. Rosenberg, A.E.P. Villa, POEtic Tissue: An Integrated Architecture for Bio-Inspired Hardware. Evolvable Systems: From Biology to Hardware. Lecture Notes in Computer Science, Vol.  2606. Springer-Verlag, Berlin Heidelberg New York, pp. 129-140, (2003).

Author Biographies
Raquel Paricio Garcia graduated in Fine Arts and is currently a Ph.D candidate at the Technical University of Catalunya (UPC) in Barcelona about Evolvable applications in Art installations. Her research interests include body consciousness and expression and multi-sensory environments and interfaces, apperception and new qualia.
She is the creator of www.res-qualia.net on-line space.

She is currently granted by the Catalunya Goverment to develop her work and research. She has been teaching in different University schools of Barcelona. She has participated in exhibitions held in Mendel Art Gallery Saskatoon, Fundacio Tapies, Digital Culture Festival, at the Museum of Contemporary Art Chicago, E-literature Festival, Primavera Sound, Barcelona, Agora Möbius and University of Valencia. She has participated in several conferences and publications such as Planetary Collegium,  ACM multimedia, Complex Systems Network of Excellence, LEA (Leonardo Electronic Almanac).

Dr. Juan Manuel Moreno Arostegui, Ph.D in Telecommunications Engineering, holds currently an Associate Professor position at the Department of Electronic Engineering, Technical University of Catalunya (UPC), in Barcelona, Spain. He is the coordinator of the POETIC IST project, and in the past he has participated in other projects funded by the EC in the area of programmable devices (RECONF2 and FIPSOC projects) and artificial neural networks models (ELENA project). His research interest includes bio-inspired computing techniques; artificial neural networks models and architectures for programmable devices.

Citation reference for this Leonardo Electronic Almanac Essay

MLA Style
Paricio García, Raquel and Aróstegui, J. Manuel Moreno. “Adaptive methods for the development of interaction in artistic installations.” “LEA-ACM Multimedia” Special Issue, Leonardo Electronic Almanac Vol. 15, No. 5 - 6 (2007). 10 May 2007 <http://leoalmanac.org/journal/vol_15/lea_v15_n05-06/RParicioMArostegui.asp>.

APA Style
Paricio García, Raquel and Aróstegui, J. Manuel Moreno. (March 2007) “Adaptive methods for the development of interaction in artistic installations,” “LEA-ACM Multimedia” Special Issue, Leonardo Electronic Almanac Vol. 15, No. 5 - 6 (2007). Retrieved 10 May 2007 from <http://leoalmanac.org/journal/vol_15/lea_v15_n05-06/RParicioMArostegui.asp>.