Codex for Musicians: Notation, Arranging, and Transcription

Last week, Codex helped me with a new area of work as a composer, and it blew my mind again.

A piece of mine called An Agreement was licensed for a performance with a ballet company. They requested parts so live musicians could perform the piece with the dancers.

The issue was that I had improvised all of the music.

I still had the original Ableton Live project files, the piano MIDI, and the audio from the modular synth, but there was no score. To create the parts, I would have to transcribe and arrange the piece after the fact.

By hand, this would probably have taken me 20 to 40 hours. With traditional notation software, I might have reduced that to 15 to 30 hours, assuming the software could handle what I needed. When I tried to open the notation application I already owned, I discovered that it had been deprecated and no longer worked.

For the past several months, I have been doing some pretty far-out experiments with Codex and music software, so this felt like an excellent challenge.

The original recording was made with modular synth and piano. I first improvised four oscillator parts, freehand, without quantization. Then I improvised a piano part over the synths. The entire piece was performed in open time, without a grid or fixed tempo.

To create a performable score, I needed to notate the piano MIDI and turn the modular synth recording into four string parts: cello, viola, violin 2, and violin 1.

The key to the notation came from the left hand of the piano part. There is a steady, slow waltz feeling throughout much of the piece. The triplets were not performed in strict time, but they gave us a musical framework for translating the performance into notation.

I worked back and forth with Codex inside a music software repo I had already been building. The repo included raw audio analysis tools using librosa, though it did not yet include a notation system. For this piece, Codex began building that capability from scratch inside the repo. Using codex-live-bridge, an application API we had previously built together, Codex opened the Ableton Live project and inspected the source material directly.

The solution was to treat the score as a readable approximation of the original performance. It needed to preserve the character and detail of what I had played while giving the musicians something clear enough to rehearse and perform.

Once the piano MIDI was notated, we cleaned up the meter so that the score remained within readable measures of 3/4, 4/4, 5/4, 6/4, and 7/4. The phrasing could still breathe, while the page gave the musicians a usable structure.

For the modular synth parts, we used a combination of audio analysis tools and Ableton Live’s Audio to MIDI functionality to create MIDI material from the original recording. This remains an incredibly difficult task for software, especially with oscillator lines that sweep, overlap, and move freely through time. I still needed to edit the resulting MIDI by ear and make musical decisions about how the parts should be assigned and written.

Codex then created an HTML-based notation page where I could view the complete score and each of the five individual instrument parts. From there, we added articulations, dynamics, phrase markings, measure numbers, and glissandos to approximate the oscillator sweeps between pitches.

To check whether the written rhythms matched the performance, Codex created simple playback synths for the piano and string parts. I could listen to the actual notes from the transcription while looking at the score in real time and make adjustments where the notation felt unclear or inaccurate.

The moment that really got me was playback.

When I pressed play, the simple synths performed the score with the phrasing, dynamics, articulations, and glissandos we had written into it. An improvised modular synth and piano recording had been turned into a score for live musicians, inside a notation system that Codex had helped me create for the piece itself.

I was able to export the PDFs, review them closely, and turn in the score and parts. I was extremely happy with the final result.

The entire process took about five days. My direct time prompting, editing, listening, and reviewing was probably between five and ten hours. Using Codex with the new /goal feature, I was able to give it an extended task and have it continue building and notating across many autonomous hours. During much of that time, I was rehearsing with another dance company. The parts finished while I was performing, and I sent them off the next morning after a final review.

What is especially exciting to me is that this does not end with one piece.

Now that the workflow exists, I have started a new Codex automation that looks through 15 years of my Ableton Live projects, roughly 1.2 terabytes of music, and begins identifying pieces that may be strong candidates for notation.

I have released hundreds of pieces of music through Sound for Movement. Until now, many of those pieces existed primarily as recordings used by dancers, teachers, and choreographers. This process could allow me to begin publishing scores and parts for musicians as well.

If I were able to hire a composition and library assistant, this is exactly the kind of work we would be doing together: going through the archive, identifying pieces with another life in them, preparing notation, creating new formats, and making more of the catalog available to artists.

With Codex, I can begin doing that work now and grow Sound for Movement in a direction I have wanted to explore for a long time.