Computer simulation of a musician playing an instrument

Sometimes this Everyday Musician blog will offer some glimpses into the future of technology for musicians.  This one came across my desk today.  Researchers (including Mark Bocko) at the University of Rochester have simulated a musician playing an instrument.

The claim of simulating a musician might lend itself to controversy, as well as humor along the lines of drummer jokes.  I’ll let you all have the fun of making up jokes about this your blog comments.  This is very interesting stuff, that I’ll consider here seriously.

For many years, there has been a method of simulating instruments that is termed “physical modeling.”  The physical modeling of a wind instrument, such as a clarinet, is in part a computerized simulation of the wind tunnel in the clarinet, with wave pressures of air bouncing around.  The physical modeling of a violin string is a simulation of the string vibrations that are picked up by the body of the violin as the bow crosses the string.   Physical modeling has been available in commercially available electronic (MIDI) keyboards as early as the mid-90s.  When you hit a key soft or hard on the keyboard, your action is interpreted as blowing soft or hard on the clarinet, or bowing soft of hard on the violin string.

So, what’s new here?   Two days ago, the researchers announced at the International Conference on Acoustics Speech and Signal Processing that their computer simulations not only mimic what’s happening inside the musical instrument but also mimic the physical actions of the musician himself playing that instrument. 

For example, this University of Rochester project simulated the physical activity of musician playing the clarinet, including “the fingerings, the force of breath, and the pressure of the player’s lips to determine how they would affect the response of the virtual clarinet.”

This simulation of the musician playing the instrument offers a far richer way of controlling the virtual instrument.  Instead of controlling the sound of the clarinet by just hitting keys soft or hard on the keyboard, and perhaps using a joystick on the electronic keyboard, the virtual clarinetist actually has a virtual mouth cavity, and lung blowing air, and tongue articulating the notes. 

As a keyboard player, I’ll be interested in how this technology might evolve to enable a keyboard player to “pretend” he is a clarinetist, by controlling the virtual musician portion of this simulated musician-instrument combination.

Tags: Music Technology