Arie Altena

The Knowledge of Musical Time

Interview with Joel Ryan
Arie Altena


Joel Ryan, composer, inventor and scientist is a pioneer in the design of musical instruments based on real-time digital signal processing. Starting from a scientific rather than a musical education, he moved into music by degrees from physics via philosophy. He came to STEIM (Studio for Electro Instrumental Music) in Amsterdam in the 1980s, and currently works there. He also tours with the Frankfurt Ballet and is a teacher at the Institute of Sonology in The Hague. Joel Ryan seeks to bring concreteness to digital electronic media through the intelligent touch of the performer. He regularly performs live in improvised duos, and with Evan Parker’s Electro Acoustic Ensemble. The interview was conducted over Skype, early January 2012, when Joel Ryan was temporarily in San Francisco.

Arie Altena: There are at least three reasons why I wanted to have some of your ideas in this book. One is the way you deal with improvisation. You have built computer-systems to improvise with live musicians, and I know that in your opinion timing is crucial. So I’m interested in your ideas about time and timing, as informed by your performance practice and the design of your set-up. Then there is your knowledge of physics, and the notions of time used in contemporary physics. I wonder how that informs your ideas about music. And finally, let’s start with this: I remember that during a conversation you said how much split-second timing in improvisation relies on muscle memory, instead of rational decisions in the choice of notes, let’s first play a B-flat, followed by two low A’s slightly out of tune.

Joel Ryan: First of all, I don’t like using those terms. Talking about reflexes seems to me to be something from the nineteenth century, or referring to Pavlov. It’s just about the meat-side; I think such an approach misrepresents the cognitive situation of making music. It makes it seem as if improvisation in music is just based on some brutal instinct, on some sort of unconscious, irrational process, that it happens unthinkingly, instead of being what it is: one of the primary references that we have to actual knowing anything intelligently about time and timing. You can have a rational theory about rhythm or timing, but that’s not enough to account for the amazing thing that musicians do with time. Only musicians know this, and it is knowledge that to a large extent is not accessible to intellectualisation, certainly not within continental philosophy. I have read a lot of books recently about time in Western music and most, if not all of it is incredibly unsophisticated and simplistic when you compare it to what neurologists are doing with time, or what mathematicians are doing with numbers. A Greek from 2000 years ago would be embarrassed by the way time is approached in Western conservatories, it is shockingly primitive. Most of the writing about musical time is either about the relations of simple numbers or about rubato as traditions of interpretation. Rubato seems to mean that when you have all these wonderful times happening, you just adjust it a little bit to make it more ‘expressive’. It doesn’t represent how music happens, obviously, and it doesn’t represent how composers invent music. Rubato is not a concept to quantitatively capture the richness and complexity of music. It is not a rich enough representation of what time in music is. It’s difficult to have a good conversation about this with musicians – though they are interested – because they don’t have the language. Things are very different in the study of harmony. A lot of research has been conducted, some of which is very ‘recherché’ stuff that only string theory physicists can understand. It’s quite impressive in its ambitions. But there isn’t anything like that for timing and rhythm.

AA: It seems as if a lot of Western music has derived its idea of time and timing from the metronome and the clock. What you’re saying is that real time as it happens for a musician is way more complex. In what sense is clock-based time and rhythm not even scratching the surface of time?

JR: There are two issues with clock time. The first is the objectification of time. Clocks have given us the ability to agree on what time is. Clocks are really a social invention, and very practical ones at that. Before the 1880s, every town had its own local time, but when trains started travelling longer distances it became necessary to agree on one time. This notion of objective time is extended to music, though what we really look for in music is something else, something personal and local. The flip side of this, related to musical time, is the issue of synchronisation. How to synchronise 30 or 40 musicians? It isn’t the most important thing in music, and neither is rhythm, but it has had a huge influence on Western perceptions of music. Clearly synchronisation is something that is already sought after and achieved between two or three musicians, but there the sync is totally intuitive – it’s not about tracking some objective notion of time. A string quartet doesn’t need a conductor. The feeling for time in a quartet is so tight – it’s in microseconds – that it’s amazing. You watch a string quartet and you think: how do they do it? This kind of synchronisation has nothing to do with clocks or simple rationalisations of time, it has everything to do with feedback, perception and the direct cognition and intuition of time. It is a knowledge of time that cannot be improved by any of the rationalisations of time we currently employ. You can say, time is only what can be written – the time indicated in the score – but when you look at this as mathematics, as form, it doesn’t capture what goes on in music. Talking about interpretation, about how a little rubato really makes a piece, while reverencing virtuosity, is still really about diminishing the role of performance as the primary source of knowledge of musical time. This seems very similar to the way physicists used to talk about ‘transient’ behaviour when they were unable to get to grips with the real complexities of dynamic systems. So while a great deal of time is ‘constructed’ in modern music, in the end, when it comes to performance, this can only act as a kind perturbation or testing of our internal sense of time. Compare it to pitch: you cannot say an A is 440 Hz in the key of C-sharp – that’s just not going to happen. It is what it has to be for that musical instrument, for that singer, and at that moment. To say someone is off pitch is not about whether or not the pitch is objectively correct, it is about whether or not it sounds right to the person, a performer. It is intuitive knowledge. The mistake in European music history is the belief that systems are always superior to individual idiosyncrasy. But for me music is about trusting mere individual minds, instead of promoting pedagogical approximations as law. But good musicians are always doing their music, regardless of this.

AA: You mentioned that research into harmony is much further developed...

JR: You could certainly make that argument. Harmony is an algebra, which needs a prior rationalisation. Once you have the rationalisations, you can begin to experiment. You can go back to Brahms, or Haydn to see how this works. They did harmonic experiments within the algebra, within the conceptualisations of harmony, which led to music that they would never have come up with if they had just been singing along by themselves. I am not 100 per cent sure about this. You have to ask a musicologist to get a really good answer to this question. It’s connected to the argument that writing is necessary for creating harmony. Writing music has enlarged the possibilities for harmony, but I don’t think that writing has similarly improved our ability to make time in music. It has produced synchronisation and the richness of synchronisation that happens in a symphony, but it doesn’t seem to have produced more interesting rhythms than a Cuban band, a band from Tajikistan, or a jazz band produce. Notation doesn’t seem to have contributed significantly to our knowledge of time in music.

AA: But that is especially the case for classically composed music.

JR: When you listen to Alban Berg or Pierre Boulez – who is very proud of his rhythmical stuff – it doesn’t really sound that spectacular as time. Rhythmically, it isn’t anywhere near as interesting as a jazz big band can be. Maybe it’s because it’s too difficult for musicians to play, maybe not difficult enough. I think we are at a low tide in European music: there is too much discourse without actual reference to the knowledge and skills of musicians – which were still central to composition until at least the late nineteenth century. Now we have amateur musicians aspiring to be composers. This is even encouraged and considered ‘cool’ at conservatories. You are a ‘neo-primitivist’, ‘conceptual’, and you canbe as unskilled as you want. It isn’t the same in the jazz departments where it is all about the reproduction of historical skills, which is equally uninteresting.

AA: All that might be true for academic classical music, but you can also argue that over the past few years the subtlety of using time in music has increased, especially since musicians and composers are much more exposed to music from non-Western traditions, and also to strange computer-generated rhythms... This might not reflect in the classical department at the conservatory, but it does in music...

JR: Yes, and there are also composers – like Richard Barrett, for example – who do greatly respect the special talents of musicians.

AA: Could you imagine neurological or mathematical models that come closer to a representation of what time is in music than the notation systems we have now?

JR: It sounds a little bit ambitious. I’m not a musicologist, and I do not really think they don’t understand musical time, but I would suggest that the representation of musical time is still primitive, and perhaps something else, something new, wherever it comes from, would be more useful. There is no question that we try everything in music. It is the nature of music to embrace new approaches, from new materials to new rational systems. Musicians are sluts for tech and new crazy ideas – almost everything that comes along seems to be incorporated in a musical experiment. I’m not convinced that the idea of rhythm is better advanced using representations such as those in mathematics, but it’s an enormous stimulus to invention. In the West there is a great desire to find unity in music, and mathematics offers an image of that. Western musicians are Pythagoreans, because they fall for the rational models. But while musical Pythagoreanism explains what we share, it doesn’t account for the great diversity. Music is more heterogeneous than we like to admit, its structure is intuitive, but we want to simplify it in a rational way. There was a famous music conference in Cairo in the early twentieth century which brought together people from all over the world (including a lot of French and German musicologists), and they tried to agree on a universal tuning system for ‘Arabic music’. What the motivation was for doing this, I don’t know. It resulted in tuning systems for each of the representative Arab states: one for Syria, one for Lebanon, one for Iraq and Pakistan and Turkey, and three for Egypt. That was the reality. The problem with theoretical music is this desire to be universal. It is confusing the map with the territory. This belief is so robust that it can almost only be explained as a religious thing. It isn’t accounted for by practice. Practice shows that in music there are lots of solutions to a problem. The idea that you can have a mathematical model, representation or understanding of rhythm... if you have that, how then do you deal with music as a construction and as being based on human choices? Perhaps this just reflects a philosophy of mathematics that has been adopted by music. It’s completely different in the other arts. We see poets freely inventing language and grammar, and we value them for their taking liberties with language, our language. Today we’re all aware of how street language changes the way we speak. And we feel free to accept or reject that change. I think that this is a better model for music than any Pythagorean system. I talk so ardently about this because I have a guilty conscience: in computer music you are constantly faced with strong but simplistic representations. You have no other choice than to work with these representations, or devote your time to making instruments, which turn those representations back into something you can play with.

AA: You have a guilty conscience because a computer cannot be as subtle, malleable and elastic as a human being...

JR: Not by itself at this point.

AA: In working with computers, you start with representations and then you deal with them in a way so that they become flexible?

JR: Yes, you listen to what is happening, not what your text (score or code) says it is doing. One of the most important things is that I never wanted to look at anything when I was listening to music. It shouldn’t be visual.

AA: Mark Fell relates in another interview in this book that he has an issue with the timeline. He says that he cannot come up with interesting music when he puts notes on a timeline. That’s one of the reasons for writing process-based music.

JR: I like to think of time as an elastic system. Maybe that’s because I was a physicist. I like my system to have a kind of natural, rhythmical, flexible quality, a time of its own, with a physical nature, not a formal time imposed on the system, not a Cartesian time. I tell students: try and find the time in the system. A system can be quite large. It doesn’t have to be a device, or software, it can be a band too. The power of a lot of great quartets is that they understand how they work as a rhythmic system. The members understand each other’s limits and abilities, and they can push and excite that system. That’s an interesting type of process-based music too: based on communication between people as systems. Such a system would be the ideal for me. I think that it’s also socially interesting because in a way it’s a model of what we have to achieve politically as well, how we have to integrate into, well, a society – system is such an ugly word for it. We have no good vocabulary for this; maybe we have to keep checking with the biologists to see if they can come up with better words to describe it.

AA: You have terms like the hive, the dynamic unstable network...

JR: But none of these words capture the complexity of it in an operational way, they don’t include the flexibility that I’m after, the elasticity of time. The time of the system is made out of the pieces of the system– you can never say that the system imposes the time on its pieces. The point of a system is that there is no whole, no centre. Time emerges from the pieces. I have been reading a lot on turbulence and the weather. How do you determine the velocity of a packet of air? There is no absolute velocity, you can look at one small piece and say that it is moving West at ten m/second past this fixed point, but that piece is also within a slightly larger chunk that is currently moving North, which is itself part of a larger front which is rotating and drifting East and so on. That is analogous to the problem of absolute time in music. Music consists of so many independent processes, all of which have temporal expression, some in close harmony, some autonomous. It becomes problematic once you try to describe it as if time is one objective thing. Your left and right hands keep a different time, your left foot is again doing a different time, the reed player is in a different time from the guitar player... this is not a talent, it is a given.

AA: Is there something like absolute time?

JR: That was the great dispute between Leibniz and Newton in the seventeenth century. It was about whether time exists independently of us. Leibniz claimed that time was relative, Newton believed in absolute time. There was a deep-rooted belief in the eighteenth century that we could discover the clockwork of the universe. It implies that the universe works regularly and everything about it is knowable in some sense, which was reassuring in an age of doubt. We have had to give up on that idea; we know things are much more complicated. But at the moment music is ideologically very backwards looking. The idea of absolute time in music reflects an ideological issue, that of centralising, authoritarian concepts.

AA: You like the material side of music, even though you work digitally...

JR: I like mechanical devices, not for any clockwork idea, but for the material aspect. Some time ago I tried to find out more about ancient metallurgy, because it must have such a huge impact on the sound world and on sound perception. Metals make sounds that do not occur in nature. It must have been quite amazing for Neolithic people when they started realising the sonic possibilities of tin and copper, silver and nickel, and started making bells. These new materials also made standardisation possible, because they were more stable and reproducible. After the invention of metallurgy the Meso-Americans changed their cosmology to include metal: their gods chose to make humans out of metal. This could be because metal is responsive, it is reciprocal, metal echoes, you give it something, and it gives you back something else. That’s a musical observation, and it is the wonder we look for in music. We want to tap on our computer and have it do something surprising, something we cannot do ourselves. It is an old dream, a magical, animist desire. Electronic music is historically based on seeking new sounds and new possibilities for sound organisation, but to get that from the material itself: that is the great wish. Musicians have great respect for material objects, which is why they stick strange pieces of shaped wood and metal in their mouths. Musicians love their instruments and look for improvement in the otherness they represent. Music has a strong quest for the connection to otherness, to the non-human part of existence and the universe – more so than other artforms. The embracing of materiality and the agency of things is intrinsic to music. The problem for computer music is that this agency is very poorly developed, because it derives from late twentieth- century hyper-rationalism.

AA: Which is a problem?

JR When I got into the Institute of Sonology in 1987, there was a recently retired professor who had been teaching students Bertrand Russell’s sentential calculus as a path to making music with computers. This is a horrible notation that renders even the relations of logic itself obscure. Of course this was a long time ago but a belief still persists that composing should look like an opaque language rather than reflect or enable what musicians know. Technology trumps music every time.

AA: Your quest has always been to make the computer into something that allows you to interact with ‘materiality’.

JR: Perhaps it’s because my education is in physics. In physics it’s normal to think this way. It is okay to think in physical models, because we believe in the reality of the physical world. The models are always seen as models of something larger than themselves, and you don’t confuse the model with the physical reality. You don’t confuse the calculation with the thing you are calculating. You would never confuse a model of the weather with the weather, or a model of a 747 with the actual aircraft. But in music nowadays, the simulation seems to be just as good as the music. No youngster today doubts that a synthesiser or a sampler is a musical instrument, nor do they distinguish between music made purely with samples in a tracker, and its precursor made by people with instruments in a storefront in Detroit. For most people there is no difference between a stimulated sound and a simulated sound. I believe you have to constantly refer back to nature and to people (musicians) to see and hear time, and not surrender to some simplistic representation.

AA: That also relates to the major issue in early synthesiser design, the Buchla versus Moog thing. Buchla tried to invent new sounds, Moog added a keyboard to the synthesiser, and hence it was promoted as a technology to emulate existing instruments.

JR: And it sold. I just saw Buchla’s latest synthesiser. It’s huge, it’s everything he tried to do combined in one machine for 40,000 dollars, and it makes beautiful sound. Don Buchla is interesting because he was a super smart engineer. My astronomy professor who was Buchla’s classmate at Berkeley, told me that he was the most brilliant engineer he’d ever met, he was intuitive, a Nikola Tesla type. He didn’t have to finish his degree. They just dragged him into the physics lab. His gear is really different from other synthesisers. It does weird things. It has ideas in it that you don’t find anywhere else. Even the sequencer is much more complicated that any other sequencer I’ve ever seen. Instead of trying to simulate existing music, a Buchla gives you unique behaviour.

This text was published in Travelling Time, 2012.

some rights reserved
Arie Altena