The words "Sein und Zeit," the title of a book by M. Heidegger, have been adopted as the key words of this project. Distinct from their usage in the publication, they represent an attempt to consider what forms audio and visual information take when accompanied by a time axis, and how they are interrelated; this with respect to how humans appreciate information that is a reflection (phenomenon) of our computerized existence. By means of breaking up the continuity of an image, and placing a subject on a different time axis, this project attempts to expose a segmented time gap to an audience who see a continuous image that constitutes thirty still frames a second. It is also an attempt to explore the interactive potential of audio and visual information systems and physical beings that exist in real time, when the systems themselves are on a different time axis.



Mass and Media

The performance piece "Mass and Media for Percussion" is the first concrete result of the "Sein & Zeit" project, and at the same time a prototype that will form a base for future developments.

"Mass Media" is a performance piece that features the recital of a percussionist on stage. It both breaks down and ‘operates’ ground based television broadcasts that are representative of current mass media. The physical movement of the percussionist is merely a ‘recital’ to the performer himself, however to the machines it is an ‘operation’, and to the computers a ‘command.’ In this regard the linear musical score dictates the music performed on stage, and at the same time represents instructions for the operation of the machines.

The percussionist is a person who manipulates the progress of the performance on stage, similar to a priest holding mass, adhering faithfully to what is written in the musical score. Even though the performer is able to freely manipulate the sampled television broadcast, he acts purely as a medium, and does not know anything about the content of the programs that just happen to be broadcast at the time.

The more skillful the performance of the trained percussionist on stage, the more likely he is able to accurately recite the musical score like an un-fluctuating machine and realize natural, stable video playback. In other words, since the development of the whole performance revolves around the performer, the more superior the skill of the performer, the less his existence affects the quality of the playback.








Equipment

This performance work is firstly a concert piece. It consists of a device that samples and scratches television signals broadcast at the time depending on the drumming of the performer.

Television and video images are usually displayed at a speed of 30 frames a second. This is a "hand operated video replay device" that replays these images in keeping with the rhythm of the percussion instruments. If the percussionist strikes his instruments slowly the image replay slows, and if the percussionist strikes his instruments hard the images grow brighter. The audio that accompanies the images follows closely the progress of the image frames, which are themselves dependent on the actions of the percussionist. The setup of the percussion instruments revolves around bongo drums that have been assigned single frame advance and reverse functions, and other instruments that have been assigned functions such as changing the television channel, and starting and stopping the television broadcast sampling. The performer recites the piece adhering faithfully to the musical score. Towards the end of the performance the image memory, that is full of samples of the television broadcasts, is written over with images of the percussionist himself, upon which the percussionist scratches the images of himself by continuing to recite the music dictated by the musical score.




System

The performance system comprises two computers. One is used solely for digital image processing and the other is used for audio processing and control of the whole system.

- A DV video sampling and playback software package called “videotronics” was developed especially for this project. Together with MIDI Control, this software is used on the image-processing computer to record and replay video signals, and for capturing and scratching digital signals in conjunction with the system control computer. It also controls the brightness of the images, which in turn depends on the volume of the drums.

- The audio-processing and system control computer is used firstly to read and replay the video, and accompanying audio signals. It is also used to simultaneously control the image-processing computer, and process the performance information (transformed into MIDI data) obtained from multiple microphones and drum triggers. The video and audio playback speed is unrelated to the original audio signals. Granular synthesis is used in the playback algorithms to facilitate playback of the signals from the percussion instrument triggers according to overall time progression.

The video recorder used for receiving the television broadcasts, changing channels and the image effecter are all controlled remotely using a MAX/MSP patch specially developed for this project.





Stage

On the stage there is a large video image screen, a percussion kit, and PA speakers on the left and right of the stage. All other equipment including the computers is placed to one side of the percussion kit, underneath the screen. The performer controls the whole system during the performance and there is no separate operator. There is a PA mixer located in the central area of the auditorium.


Potential for development

The software and performance system developed for "Mass & Media"” is not necessarily restricted to percussion instruments. It is a versatile system with ample room for development. Examples of other uses include the modern revival of live production using manually operated cameras that have been used in the past, or a system that controls frame advance according to the movement of dancers.

Amongst these possibilities, one that I would most like to see realized is the “Mass & Media Tele-presence version”. As the reader would understand from the system outline, all the performance date generated by the performer is digitalized, which opens the possibility for remote control, or control by multiple systems. That is, by simultaneously transmitting the images of the recital and all other performance data, it is possible to generate multiple versions of the work, all different from the original, via using local television in other countries around the globe. One of the special characteristics of this project is how, in principle, the time delay that usually occurs when using a network for transmission does not present a problem. The delay in transmission of image information when sending pictures can not be avoided, however due to delays in, or by purposely delaying the transmission of the MIDI information, it is possible to fully coordinate the local video materials with the movement of the percussionist contained in the video images transmitted from the performance "base."