RAY LI: My name is Ray Li. I'm studying applied physics with a minor in music, and I have been working on the aura, which is a new musical instrument that I think can really change the way that people see musical interfaces.
[SYNTHESIZER MUSIC, TIED TO HAND MOVEMENTS]
RAY LI: So the first thing I made was an instrument called the saber, which was a cello-based instrument that had four strings, which were basically conductive fabric that I mounted on to fingerboard. And then I had sensors on my fingers which would detect where along the strings I was, so I could control pitch that way, and then I also had a joystick for my right hand, and there were touch sensors on that joystick that allowed me to control the sound quality and do really interesting things.
[MUSIC FROM INSTRUMENT]
RAY LI: So I started thinking about, OK, I have this right-hand joystick controller for tone quality. How can I improve on that? What if it wasn't a joystick? I started thinking what if it was like a webcam? What if there was a distance sensor or something like that? And then I went into the next semester thinking that I was going to just build another version of the saber with a motion tracking sensor or something for the right hand instead of a joystick. And after thinking about it more, I just thought about, why even stick to this fingerboard thing? What if the whole thing was just a motion-based instrument? So that's what I did. I just thought, let's make an instrument that's completely just motion-based and throw away the idea of any traditional instrument reference.
[SYNTHESIZER MUSIC, TIED TO HAND MOVEMENTS]
RAY LI: So after I came up with the idea for the aura, I knew that I couldn't do it on my own, because I needed a lot of programming to be done, which I didn't know how to do. So I reached out to a friend of mine who lived in my same suite. His name is Michael Ndubuisi and he's been helping me with the computer science part, all the programming. He's been doing some absolutely fantastic work with programming visualizations and just different sound parameters.
When we first tried to build the aura, we spent an entire semester experimenting with a motion tracking system, which was completely physical, in that there were actually strings attached to a glove that the musician was wearing. There were three strings attached, and each of the strings fed down to a joystick, where the string was retractable in the joystick, and the joystick would move around based on where the string was pointing. So that allowed us to get like the position of the endpoint of the string. We tried that. But that kind of failed miserably, because the sensors that we were using just weren't accurate enough, and there was a lot of complication with things being hot-glued together, falling apart.
So after the string tracking mechanism didn't work out, Mike and I started looking into alternative tracking devices. What we settled on was a magnetic tracking system, where the musician would have a small magnetic sensor attached to their hands, and then there would be a magnetic base, which emits magnetic fields that the sensors on the hands pick up. And it's fantastic. It's so much more accurate, so much faster, and just a much better solution than what we had last semester.
How the aura makes music is similar to how the saber made sounds, in that it just controls software on my laptop. But there's infinite possibility for what kind of sounds the aura can make. It's really just MIDI device, which means it just sends out data, in terms of positions and different parameters and those can be interpreted by any different software that uses MIDI, which is a pretty universal language. So yeah, we're really excited to start experimenting with lots of different software synthesizers, to really try to see what we can do with this thing.
We've received your request
You will be notified by email when the transcript and captions are available. The process may take up to 5 business days. Please contact firstname.lastname@example.org if you have any questions about this request.
Ray Li, BS '14 and Michael Ndubuisi, BS '14, talk about their new musical instrument, Aura, which allows the musician to control sound simply by moving their hands in the air. Video courtesy of the Cornell Daily Sun. Videographer: Ryan Larkin, film major.
Li and Ndubuisi will perform with the Aura, March 26, 2014 at 7:30 p.m. in Barnes Hall Auditorium.