How can we play music with our eyes?

Max MSP | Jitter | cv.Jit | Arduino

Exploring our intimate — and often uncomfortable — connection to the human gaze, Ocular is an electronic musical instrument that explores the paradox between this discomfort and the role eye contact plays in forming true human connection.

Ocular loops different music samples and affects them by tracking the performer's pupil with computer vision. In the beginning, sounds are harsh and designed to make the user uncomfortable, but as more samples play (and more eyes look into the audience), a melody begins to take shape, creating a more forgiving and comfortable environment for the audience, even though the number of eyes might feel more unsettling.

Designed and developed over a span of three months, Ocular was performed in front of a live audience in Brooklyn, NY for ITP NIME on December 3, 2017.



In order to invent a new instrument, I started my process by researching past NIME performances and figuring out which elements of existing performances I liked. Inspiration came from a sound, a gesture, an effect. During my research process, I dove into eye-tracking with computer vision. The idea of playing an instrument with my eye appealed to me, and I began to ideate on what a performance would look and feel like. Through rapid prototyping, I built my first prototype of Ocular and through testing, I iterated to the final performance version.

Initial Sketch


Ocular was performed in front of a live audience at Littlefield in Brooklyn NY.


Special thanks to Aaron Montoya Moraga, Greg Shakar, Yuli Cai, Chino Kim, and the rest of the ITP NIME community.

The max patch for this project is open source and can be found on Github