Slugfest

Chiron Mottram and Alan Penn

The VR Centre for the Built Environment, University College London

e-mail: c.mottram@ucl.ac.uk

 

 

 

 

 

Abstract

This paper describes the creation of interactive and responsive digital art. By adding a message handling interpreter to SGI's Performer software we have created a medium within which interactions between virtual objects and interactions with the user are made possible. By making objects within the medium responsive to each other it is possible to create emergent effects. By making the medium responsive to the user the viewer is more intimately involved in the artwork. However, at the same time this requires the artist to work with the intrinsic properties of emergence in the system to and to develop the audience relationship by means of involvement in a manner more akin to performance than the plastic arts.

The use of virtual reality allows the creation of a profusion of different forms and behaviours not possible in conventional plastic media. The aesthetic of these forms can be governed in two ways, either by in-built rules or by direct intervention by the artist/audience personae. The built-in rules can be changed dynamically as can the objects' actions as mediated by the viewer's intervention. Underlying this is the aesthetic produced by the machinery of the computer, which can both impede and improve this process, this is the illusion of 3D provided by the Performer libraries in conjuction with the SGI box, and the many constraints implied by this. As such we are also limited by the input devices, though the mouse has now almost become synonymous with the pen, as a means by which our interaction with computers is governed. This paper describes a series of pieces which explore the boundaries of generated and evolving artworks in which kinetics and morphogenesis are integral to investigations of social interaction between virtual performers.

1. Introduction

“I would like now to examine the dimensions of the object in a new light and so try to show how it is that the artist frequently arrives at what appears to be such an arbitrary ‘deformation’ of natural forms.

First, he does not attach such intense importance to natural form as do so many realist critics, because, for him, these final forms are not the real stuff of the process of natural creation. For he places more value on the powers that do the forming than on the final forms themselves....

Thus he surveys with a penetrating eye the finished forms which nature places before him.

Then he permits himself the thought that the process of creation can today hardly be complete and he sees the act of world creation stretching from the past to the future. Genesis eternal!”   (Paul Klee, On Modern Art, 1924, p43-45)

Speaking at an exhibition opening in Jena in 1924 Paul Klee gave one of the first desriptions of modern art considered as an analytical process. In this he described the artist’s role as a transmitter of underlying structures within nature through the body of his work. He described the ‘artist’s workshop’ or pallate of tools: line, tone, colour; and the way they interelate into constructions of figure and compositions of objects.  In this paper we argue that virtual reality is a medium waiting for its artists, but bringing a greatly extended set of tools to the artist’s pallate. Virtual reality is, perceptually at least, three dimensional, and so to line, tone and colour we add form and space. However, virtual reality is also interactive and responsive to the viewer, and so behaviour must be added, along with the dynamics involved in that. One aspect of behaviour involves the ability to control growth and change and so morphological evolution also forms a part of the pallate. These dynamic and responsive dimensions supplementing those of line, tone and colour, bring a range of entirely new possibilities to the artist. In spite of these possibilities, however, the take up of virtual reality for creative purposes seems remarkably slow. Our work in this area stems from an interest in the use of virtual reality as a tool for creative architectural design, however in pursuing this interest we first came to realise that the development of the medium for creative design depends on its becoming a medium for the plastic arts. The investigations described in this paper explore some of the implications of adding dynamic dimensions and responsiveness to the artists pallate, however, we first give a brief description of the steps we have had to take to create the tools we needed.

2. Tools

 

Most tools for virtual reality visualisation focus on the development of the illusion of three dimensions. The problem with these is that they are usually entirely separate from the CAD tools within which form is created. The break imposed by this division between form generation and visualisation has a disasterous effect on design and creativity, and so to one of these visualisation packages, the Perfomer API on Silicon Graphics, we have added the Gaiatalk ‘scripting’ language developed initially on the Intelligent Architecture Project and further adapted as part of a project for the VR Centre for the Built Environment through a series of extensions to the language’s capability (Penn et al, 1996).  The scripting language essentially gives to objects (3D shapes within the ‘world’ one is creating) the ability to send, receive and act on ‘messages’.  For instance, a script might consist of the following text:

on mousedown

     rotateby 10,10,10

end mousedown

When the object containing the script receives a ‘mousedown’ message to say that it has been clicked on by the user it then acts by rotating through 10 degrees in each plane. Gaiatalk consists of a series of messages that objects understand, and these messages can be sent and received by the user, the application, the document or ‘world’ within the application or any object within the world. Since each object in the world has a unique identifier messages can be addressed directly to objects, and so objects can communicate with each other and pass on actions or data. Using a scripting language, means that new behaviours are easy to prototype. It also means that, the objects can modify each others' behaviour.

In the recent enhancements of the Gaiatalk language scripting messages allow objects to change shape dynamically using functional parameters. These messages can be addressed to individual vertices on objects, allowing for scriptable ‘morphing’.  We have also given a degree of limited "vision" to objects within the world using the Performer API’s functions that return the identity of a poligon at a given location in the scene. The Performer API is probably the best 3D library currently available and allows for a wide range of very rapid rendering functions that can be accessed directly from scripting. It runs primarily on Silicon Graphics machines under Irix where it is optimised for the SGIs hardware, and also, somewhat less efficently, under Linux on PCs.

These new functions in the scripting language have allowed us to produce ‘squidgy’ shapes that respond to each other, or the user, either visually or by ‘touch’. Performer’s intersection testing functions allow objects to detect whether they are touching each other and so respond appropriately, and the user can touch objects by tracking the mouse postion or with a 3D cursor which is in fact just another shape in the world. These are the elements of functionality that allow us to expand the dimensions explored in plastic arts beyond those available to Paul Klee to encompass an exploration of the dynamic aspects of behaviour and response between objects in a virtual world.

3. Pieces

Three main pieces have been made to date, the slugs, the jellyfish and the intelligent shapes. These highlight different aspects of the way that behaviour and response relate to our perceptions of intelligence and ‘life’. Through these pieces we explore different movement, visual, reponsive and message passing criteria, and have begun to investigate the constraints and material properties inherent in the scriptable ‘stuff’ of our new medium. At the same time we are trying to develop an awareness of the criteria which produce effective aesthetic systems which exploit the constraints to creative effect.

3.1 Slugfest

A colony of slugs is created. Their motion consists of several different parts: the initial phase uses a sine function running down the body of the creature from just behind the head, input to this is the distance of the vertices of the whole object moving to a fixed point, this is done "n" number of times with the distance factor being multiplied by a factor of "n" to create a slithering extension. The second phase allows all the vertices to return to their original position relative to each other, and this is done in a number of steps to maintain smoothness of movement. The slug moves forward at the same time proportional to the distance of movement of the vertices, thus an illusion that the slugs’ movement is due to the vertices movement is created.

The dynamics of their individual motion is only one aspect of their movement,  the other, the navigation and relationship between slugs is probably more important for establishing the eviseral qualities of this piece. The basic algorithm gives the slugs limited (tunnel) vision and a simple rule on motion. The rule is that if no other slug is directly ahead, move forward whilst randomly changing direction by an increment, otherwise if there is another slug ahead move straight towards it. If there is another slug within a small distance ahead turn sideways away from the other creature and squirm. The result is that the colony of slugs ‘cohere’ into an apparently primitive social group while each individual is moving apparently with free will.

The viewer can interact with the slugs since they are responsive not only to each other but also to a mouseclick. At each mouseclick an individual slug changes its colouration and speeds up. If you speed up a slug enough its character apparently transforms and it becomes, instead of a slug an excited puppy, rushing through the crowd and wriggling wildly. It seems that our attribution of animal characteristics relates as much to the dynamic as the form.

3.2 Jellyfish

This was a simple attempt to investigate what was necessary for the viewer to infer an environment or medium with physical characteristics from the behaviours of objects apparently located within it. A series of bell shaped objects was created each hanging within the viewing volume, and each morphing slowly, but consistently with respect to each other. Thus a motion in one would be repeated in its neighbours after a slight delay. The result is that one interprets there to be an intervening medium which is somehow transfering motion from one to the next. Here we also attempted also to create the illusion of sunlight coming through the surface of rippling water by constantly, randomly altering the colour of the creatures. The piece is unfinished in two main regards. Firstly a more complex interaction with the the viewer is desirable. It should be possible to ‘stir’ the medium and to see the results. Secondly, that secondary effects should be visible on the creatures not of their own making. This would be like watching sea weed move in the waves.

 

3.3 Neural shapes

Our neural shapes try to create complexity through touch, sight and the use of simple neural networks to remember shapes and seek to return to them when deformed. The input to the neural network is the objects shape and how each vertex moved at the last iteration. The error is defined by how far away the vertex is from its original shape. Vertices are coloured by a factor of its movement, and so both colour and motion are synchronised. Due to the propogation of errors within the neural network, no shape ever comes totally to rest, but they wobble gently seeking equilbrium, colours rippling.

The larger scale motion of these objects is similar to the slugs, with a couple of differences. They have a vertical movement, if they are intersecting another object they will gently rise, if not they will sink until they intersect with the floor. If they ‘see’ another object close ahead they will send a script message which moves one of the other objects vertices out of place randomly, and at this they also make a sound. In some ways the resulting motion is like a lava lamp with occasional aggitated flurries of movement and shape changing in which the shapes take on incteasingly dynamic and angular forms, each coloured in vibrant hues, followed by a return to calm and almost stasis and more subtle and subdued colouration.

4. Conclusion

“In everyday life , ‘if’ is a fiction, in the theatre ‘if’ is an experiment. In everyday life, ‘if’ is an evasion, in the theatre ‘if’ is the truth.”  (Peter Brook, The Empty Space, 1968, p157)

The first two pieces are largely effective due to the complexity and order of space which is created by their movements and interactions. There is an appearance of seemingly intelligent behaviour by the slugs in their movement and interaction, and interest is maintained by ones attribution to them of a degree of perception and cognition. With the jellyfish the illusion is of a heavy deep moving medium, which effectively surrounds the viewer as they move through the scene. Again the interest lies in the viewer’s attribution to the piece of properties which in fact do not exist.

The neural shapes attain depth in the randomness and order that are invested in the neural networks, which are never completely predictable, but whose behaviours tend towards a norm. The timescales over which the motions achieve regularity are larger than that for the other peices, and so the interest is maintained for longer. The mind is kept working trying to unpack the interactions and regularities within the piece.

Generally one is mostly unlucky in the production of a Performance using virtual reality, there is always the unseen parameter that becomes too slow to compute. It seems that almost all conventionally choreographed methods at some point use some feature of the natural world which is instinctive to us, but is virtually uncomputable by the machine. The craft skill lies in manipulating the parameters in the computation so that the desired effect is achieved without the system breaking down. This is like the skill of the sculptor in following the grain of the stone whilst avoiding shattering the sculpture. The trick then, lies in creating the illusion that of depth and complexity when the computations concerned are in fact rlatively simple. At the same time one must limit the viewer’s interaction so this does not become apparent to them, or more interestingly perhaps, to use their own input to choreograph the scene. Their responses and reactions to the virtual environment take over from the computer. This is related to a somewhat conventional view of art, where the viewers imagination or social response becomes a feature of the validity of the art work.

This has to be fairly well underpinned by a sense of expectation or familiarity. In some ways one expects computer games to provide this, though in some ways their own interactive language is too rich, which can overload peoples expectation, our response here is to provide a mimimialist or unpredictable interface. The less there is to do the better, the user has time to contemplate and predict without distraction, interruptions become more measured and expectations are dulled. This gives room for surprise.

The computer medium for a plastic art has thus lessons to learn from the perfoming arts. The exploration of dynamics and interactions between discrete objects, whether animate or inanimate for the viewer implies a search for meaning and intention on the part of those objects. We look for causes and consequences, and it is in the search for these that we find ourselves intrigued. When we can interact in the scene and become directly involved we experiment and explore still further. As Peter Brook argued, the “theatre has one special characteristic. It is always possible to start again. In life this is a myth; we ourselves can never go back on anything.” (1968, p157).

 

References

Brook, P. (1968), The Empty Space, Pelican Books, 1972, ISBN 0 14 021415 1.

Klee, P. (1924), On Modern Art, tr. Findlay, P. 1948, Faber & Faber, 1966, ISBN 0 571 06682 8.

Penn, A. et al. (1996) Intelligent Architecture: Pangea - an intelligent workbench for architectral sketch design, in Intelligent Systems Integration Programme ES96 Papers,  I Watson, A Morgan (eds), BCS SGES Publications, 1996, ISBN 1 899621 14 8.