This article is divided into three parts: First a listing of some problems/challenges that might be considered by people working within computer music, second, the author’s personal answer to these challenges, and third, a brief discussion on how the new paradigms could be used creatively to further develop the field of computer music and maintain artistic intent and presentation during a time where the onslaught of user interactivity is stronger than ever before
The long way around
In evaluating or discussing new developments within art, technology, or both, it is helpful to consider existing problem areas. When considering contemporary practice in the art music domain, for example, it might be a good idea to remember that almost nobody actually likes this kind of music. Especially not electroacoustic music. Audiences are small, both in the USA (where contemporary art music survives only through the more or less generous patronage of the educational institutions) and in Europe. However, in comparison with the bottled, opened and recapped versions of older art music found in the larger concert halls, this is the reality for this relatively small and specialized field. Contemporary art music audiences gather around this music out of an interest (and because they know what is good for them) that is not based on the social conventions that drive patron audiences to Lincoln Center. Nonetheless, the fact remains: Very few people enjoy listening to contemporary electroacoustic music. Perhaps it is more fun to make the music than to listen to it, as a Swedish friend of mine once remarked. Perhaps the creative process for the composer who «performs» the music in his/her head while structuring it is more rewarding than the creative process that the listener goes through when trying to sort out and make sense of the unfamiliar barrage of frequencies hitting the eardrums.
Everyone constructs their personal version of reality by putting together what interests them and deselecting what does not, and this fundamental human ability to restructure the neural highways in the brain has been the most important basis for electroacoustic music since its inception; the desire to liberate music from time, place and acoustic circumstance (or to liberate oneself from the same? What are we?). To get away from those dreadfully boring instrumental sounds, whether produced by bow or mouthpiece, or by kicking, screaming or shouting. The step from this point to Internet as concept art is only one of magnitude not of principle. The desire to construct one’s own reality independent of material circumstance is what all dreams are made of.
However, the material world does exist, and it is here that we find our building blocks and the materials they are composed of. The word appropriation comes to mind. Appropriation is a special mode of mediation, where one takes something out of its context and places it elsewhere, perhaps also «making it one’s own» in the process. Many have expressed problems with this approach in art music, labeling it «cultural imperialism» in reference to the geographical axis north-south. This is their privilege, yet these same persons tend to not have the same kinds of problems with composers writing in the «old style»; in other words, appropriations from the past. Why is that? Electroacoustic music has found much of its material through appropriation, both in the recording of sounds from physical events and in the application of various principles found in the natural sciences. Is this kind of appropriation principally different from «cultural» appropriations, such as those taken from music history – or art history and philosophy, for that matter?
Nonetheless, one thing is certain; appropriation, or the particular instance of cross-disciplinary mapping, has attracted a good deal of interest – particularly in computer music – and serves as a natural foundation for various analysis/resynthesis algorithms as well as physical modeling. Electroacoustic music has, since its inception, had its home plate in a cross-disciplinary no-mans land, with areas of interest spread in all directions; from ideas of musical structures ordered through traditional pitch-classes to data flow where timelines are developed through fractal geometry. Digitalization serves as the great common denominator although it might be argued from a signal processing perspective that it is not, in principle, a new phenomenon. Digitalization reduces everything to the same two numbers, rendering the data structure itself into an object; a pliable mass that can be molded into visible, audible and tangible shapes. Digitalization opens new perspectives that allow us to penetrate to that which lies «behind»; to the structure that art consists of. There is a broad correspondence between different kinds of art expressions – on stage or canvas, on screen or through speakers – and the artistic gesture, time and content seem to gather energy from the same types of motions and relationships.
My attention has been drawn to the cross-disciplinary field through an interest in areas other than my own profession. However, most multimedia productions are based on one idea (either visual or audible), which is then supplemented by other means of expression that play support roles. Although there is nothing necessarily wrong with this approach, I find a more strictly executed cross-disciplinary mapping more interesting in the mediation of different aspects of an idea. There is still plenty of headroom for artistic decisions and interpretations but the challenge is greater since one is bound by the data sets. While mediation is a complex problem not easily discussed here, it is perhaps important to bear in mind that the viewer/listener – the receiving end – is also interesting, and that the experience is created by the receiver, not the artist. The artist has many tools to guide this reception if he/she so desires and the more defined he/she tools the idea, the better the chance of communicating artistic intent. Thus, presenting the same data structure using several means of expression betters the chance of influencing the re-creation process that occurs in the art user’s brain and nervous system.
to What
The main impulse has been to make something that can best be experienced without words in a prosaic sense. Cross-disciplinary productions such as «When Timbre Comes Apart», «Planet (Terra)», «Moment» and «Concrete Net» function well in animated games with time; with music as a means of transportation. A display of the illusion itself shows that the musical experience is virtual and thus borders on some of the fundamental ideas of VR technology. In a pedagogical sense, these animations can also serve as tools for music representation by displaying spectral content, and in addition contribute to understanding fundamental ideas in musical works; their «program» content. The illusion becomes transparent, and we are left with the data structure – the art content?
In most of these productions, the music is analyzed and the sound data is imported into programs to be mapped onto models, where movement and perspective from a viewer’s position can be generated. «When Timbre Comes Apart» is an example of a horizontal plane where two minutes of music are displayed at a time, and where the visual material is fed onto the plane in pace with the music. By manipulating the plane itself and studying it from different angles and perspectives, and interpolating between the points, an illusion of a «fly-through» is created. The color and camera movement has been determined based on a desire to define different sections of the work, to suggest connections between elements in the music, and to accentuate sensations of drama and reflection where they exist in the music, etc. The work has been performed in widely different types of audience situations, and has functioned surprisingly well for art audiences as well as more «commercially» oriented groups of people. The sync between the timeline of the music and the moving images is deliberately kept «loose» except for two or three points, in order to state that a close reading is not the point; the camera placement in the timeline is normally directly «under» the camera, so most of the time you don’t see what you hear, but you see what you are about to hear. But you are supposed to think about it.
«Planet (Terra)» is created differently; sound data is mapped onto a sphere and a «floor» around the sphere is gradually «lowered» until the mapped spectrum comes into complete view. The color pallet of the spectrum was strictly an aesthetic choice, and the swirling camera movement combined with the sphere’s rotation was used to suggest an energy-laden atmosphere. Naturally, the visual result does not appear to be the Earth as we know it from satellite photos but the idea for the work is to render an illusion of creation – after the Ice Age. The mountains and plains are generated by the music, and although the colors suggest nature, the geological formations are intended to evoke a sense of wonder and speculation, rather than to serve as factual documentation. The music is taken from a short study I made in the 1980s.
The idea for the most recent work, «Concrete Net,» combines spatial distances from our solar system (which serve to define spectral content and to some extent the timeline), short text fragments from a science fiction novel (Concrete Island by J.G.Ballard), and finally physical remodeling of actual long steel wires that exist in the Western part of Norway to send hay down the mountain sides to the valley floors – winter fodder for goats and sheep. The synthesis of these three elements is motivated by a desire to say something about wires, nets, communication and isolation – a grid of paradoxes that interest me. There is no mapping of sound data in this animation but the visual model displays some of the structuring principles that are used in constructing the sounds. There are also visual models of the virtual instruments designed through physical modeling. The «journey» of the animation is executed as a cartoon script, from a storyboard. This concert video grasps directly at that which «lies behind,» and is also designed for VR implementation.
and Why.
My principal reason for making animations is the desire to depict intellectual motion in several ways simultaneously. «Gesamtkunstwerk» is not exactly a novel idea any more, however, a multimedia approach does seem to be natural for composers of electroacoustic music. All musical ideas come from somewhere, and other forms of expression are able to serve as mirrors to further define these ideas. There are many attempts to make sense out of the possibilities that exist in this field, and it seems that popular music has been the most aggressive in pursuing the technology at present. However, CD-ROM productions are growing rapidly in number in art as well as in university environments, and interactive WWW-projects using Java shoot up like weeds – and are perhaps also examples of the relative ease with which data sets can be shared and tooled into different representations. There is a renewed focus on the receiving end, but this time in Gibsonian cyberspace.
Digitalization makes it relatively easy to consider the same data structure in different ways, and makes it easy to show the «journey» of the music, to show musical movement both in the time- and frequency domain, and also to add to the temporal experience the ability to focus more clearly on important musical events. Different analysis algorithms permit the artist to tool the spectrum representation to serve his/her needs as determined by the musical material (for example, rhythm, flow in timeline, spectral movement and density, energy profile). This form of mediation breaks away from the normal cycles of tension/relaxation and anticipation/fulfillment that one expects to find in music (except in much popular music, where constant ecstasy seems to be the agenda), and it might be argued that this changes the experience of the sound, although I don’t think that is true. Good composers typically leave plenty of clues as to approximately what will happen next.
The ambitions for «When Timbre Comes Apart» were higher; the purpose here (artistic intentions aside) was to delve into music representation, and the work thus assumes a pedagogic aspect. The current debate concerning music representation reflects a growing interest in dealing with the sound object itself, not only its representation as black dots on little lines. This is a noble project, and with regards to electroacoustic music quite necessary, since the most interesting aspects of this music cannot be captured by notes. Large numbers of data can be extracted from sound, and the use of different representations will provide better tools for the analysis of our music. (Does anyone know of a copyright agency that can come up with reasonable evaluations in determining the complexity of composition in electroacoustic music?)
Thus, by creating a journey through some aspects the music, it is possible to focus attention on musical parameters and thoughts – through zooming, turning the camera forward/backwards/sideways, and by aesthetic decisions designed to reveal the «imaginary part» – the virtuality – of the experience. One can view the data from the underside, change colors, crash through it and map physical properties to it that deviate from those found in nature or the material world at large. This guiding of the viewer’s/listener’s attention takes him/her by the hand and leads him/her through both the music and the understanding of it at the same time. A logical point for further development in this field is to sever the temporal ties, making it possible for the viewer/listener to move around in the image (both spectrum and time), to choose and focus on what seems interesting and to take as much time as desired with the music. This emphasis on the «receiving end» may perhaps leave many composers feeling unsettled by the loss of control in shaping the art content – of setting the premises. But that is their problem, not the audience’s.
It is the possibility of taking music apart (by transporting it to the frequency domain) that seems intimidating, and this deep well for hoisting up tutorial strategies is for the most part hidden from the analogue world. In taking sound apart, one sees which frequency components exist, and by using a synthesis- or an analysis/resynthesis program one can teach through destruction/recreation. This permits one to re-order existing material and the creative impulse is thus free to take off in whichever direction the muse is inclined. This approach to learning, as «uncovering,» is radically different from the traditional methods used in schools – where, all too often, the all-knowing teacher is expected to infuse his/her wisdom into an inept student mass. This «backwards» educational approach serves as the pedagogic basis for a CD-ROM project currently underway at the Norwegian network for Technology, Acoustics and Music (NoTAM), a project that is designed for children (6th to 9th graders).
Now what?
As mentioned above, the desire to combine visual elements with music is not in any way new, but digitalization facilitates the transfer of information between different domains and makes it is easier to directly include mathematics in the creative process. Faster processing and manipulation of data also expands artistic horizons. This makes for new definitions of composition and mediation, and the ease with which user participation can be incorporated also contributes to the opening of new definitions of art and its content and value.
So? Will the field of electroacoustic music become more widely accepted? Can digitalization be expected to bring about change in the cultural conventions regarding concert rituals, social exchange and institutionalized mediation? Will the definitions of individual reality change? Will appropriations continue to be considered «bad form?» Will the new (and enhanced) presentation of ideas result in more profound insights? I think «yes» to all questions except the last one where «maybe» is a better answer.
What we will see is a decentralization of valuable information, and consequently a breaking down of traditional authority and borders. Despite the fact that large portions of the world’s populace is yet not on the net, and can not be expected to log on in the near future, the «motors» of current civilization have adapted it’s methods of communication to the new media at such a pace that it is already becoming a valuable commercial industry. Knowledge becomes a commodity, and actual instrumental knowledge changes so rapidly that the focus instead needs to be placed one step further up, on a strategy for gathering information and managing the stream of data that is flowing around each and one of us.
Our ways of gathering input into our neural nets – be it information, art or both, is changing. The experience of surfing the net is radically different from that of dropping by the library, and while surfing the net is not necessarily better than visits to the library, it is a phenomenon which encourages foraging into territories bordering our own through links reflecting other individuals’ realities. It used to be one searched for information when the need arose, but nowadays one is flooded with information from various news- and mailgroups on a constant basis, and a Websearch on almost any subject brings up new URLs to the growing bookmark-file. Time takes on a new dimension, as everything exists concurrently (or a few seconds apart) in the best postmodern fashion, and one single trend no longer monopolizes the representation of what the future will hold. Human consciousness becomes increasingly mobile as, paradoxically, greater numbers of people share the same world (or information space). Or better; The same «way» of relating to the world. The tribal instinct previously sustained by local flavor or print is brought into the global village, as a continuation of a centuries-old trend. There is no longer any distinction between then and now, copy and original. The blueprint is the same in the machine. But on the outside?
And what happens to our art, if composers and «audients» use the same tools for experience, for example, a virtual reality environment for authoring and experiencing audiovisual experiences – does it not change? I think it does, and the challenge for those of us who believe that art reflects the society from which it emerges, this is an enormous challenge. When audiences can move their ear freely in the spectrum, freeze time and play music backwards and sideways to arrive at their favorite «spot», then our task has certainly changed. How do we mediate musical ideas and what exactly will they consist of? How can we discuss time’s occupation of space when the «audient» can stop time and move around in it? The situation calls for valuable artistic replies if we want to counter the industrial onslaught of entertainment in constantly new configurations.
If no one plays with the technology (and artists are particularly well-qualified and often well-trained players) it will never become «ours,» never become part of our vocabulary. We have no choice but play – alienation is the alternative. This is the principal point – and our attitudes must be checked accordingly. The answers to the questions above do not lie in increased processing power, which is perhaps what the digital era is all about, but in the power embedded in artistic expression.