GPT-6 Astra for Professional Musicians

This year, I have been pushing hard at the edges of music technology. My pedagogy is based on learning what is possible and finding ways to translate that back into music, creativity and teaching. Yes, in Research 1 universities, there was a time when burning a CD from iTunes was part of the curriculum. AI is no different, and what is possible now for independent professional musicians is staggering. What follows goes beyond dodgy generative music systems and into the ways it can support your music and career.

ChatGPT can control any application on my computer. With Astra, Computer Use is incredibly capable of working with Ableton Live. Live is my primary production platform, and in the past, controlling the software was only possible through Max patches, and more recently with Max for Live. This is amazing for learning how to use the software if you are new. I think of this as one of the very surface layers of how AI can assist with music.

By connecting ChatGPT directly to Live through an OSC/UDP bridge with Max for Live, you can get one layer deeper. What I found over the few months of intensive research is that there is no single tool or setup that will cover any and every need from day-to-day work. There are things that Computer Use can do that connecting ChatGPT directly to Live cannot, so combine them. Ableton then dropped the Extensions SDK, and now I have a third layer of ways to interact with Live and ChatGPT.

Live Codex Bridge on GitHub

One of the fascinating discoveries I made was that AI works much more smoothly and at a higher level when it is given its own native control panel. I like to compare it to how well I could make music by telling another person what notes to play, as opposed to when I am playing the instrument myself. The Codex App Server allows me to build a piece of music software and embed Codex into it.

My experience with Astra is that I can talk about an idea like this and then go right to work on making it come to life. I come from a tradition of musicians like Skip la Plante, who, after graduating college in the 60s, swore never to play another instrument that he or a friend did not make from scratch. I lived with Skip in NYC, and it was a bohemian graduate school in microtonal music and instrument building. I see a path now to a whole series of music releases made with just the music software I’ve made with ChatGPT.

I have wanted a native macOS application for my website so that dancers and filmmakers that use my music can play and edit it offline. ChatGPT went step by step to teach me about Swift and how audio works on a Mac. I knew a bunch of this as a musician working with software, but not from the perspective of a software developer. Quickly, I had my full library, streaming, downloading, licensing and filtering launching as a native Mac app. Next, we added the user’s account backend and all of the educational content.

I used to teach a course where folks would collect sounds on their phones, bring them into Ableton Live and then compose with them. So I built my music software to support that approach. You start with making an instrument from an audio sample, then you bring it into the editing area and start making music with it. Users can sign in with a ChatGPT account and access the assistant that can help you with everything in the application.

You can type back and forth with the assistant, and the agent is using Astra, with adjustable reasoning levels. I also installed gpt-realtime-2.1, so I can talk and build with the assistant hands-free and screens-free. The assistant can play the music back, and I can make adjustments manually or by asking the assistant to make the change while the music keeps playing. This was something I thought never ever possible ever in my lifetime for me, unless I hit the lottery and could spend any amount of money on it!

Stoked to be sharing the beta of the app with close friends now.

What led me to take this research seriously and invest so much time was the initial results I found over last Christmas. I connected a GPT Pro model through the API to a MIDI-controlled synth and started trying multi-pass compositions of solo piano music. A pass for the harmonic scheme, one for the left hand, one for the right, one for the choreography between the hands, one for velocity, one for note lengths, one for pedaling and a final one for notating into musical engravings in a browser. Expensive, but it worked, and I made a few beautiful short studies.

I spent the last month going back to those roots and now have a robust music software application as a plugin directly in Codex. It displays an editing area directly in ChatGPT’s Browser. Astra can add audio tracks, MIDI tracks, notation tracks using SMuFL, video tracks, effect tracks, AUs and VSTs on your machine, and can edit as directed or manually. It can load a Kontakt instance and use Computer Use to adjust the parameters, and even write automation for any occasion.

As an added experiment during testing, I had just finished a draft of music for a client and wanted to see if Astra, inside of our music software plugin, could recreate the score and play it back. We used this to figure out some functionality with the software I was making that wasn’t quite up to par, and Astra was able to adjust certain things. For example, I had not tried rendering audio and MIDI at the same time, and Astra found that my software couldn’t do that and added the functionality.

Astra can use the Supabase plugin to research my catalogue of 600 tracks and see what tempos, keys, instrument combinations and meters need new tracks to fill out the library. I had it analyze 1.1 TB of all of my past Ableton Live projects, and Astra was able to take the resulting 250 GB dataset and turn it into learning the plugin could reference while working with me. So when I say, “Let’s add a piano track next”, Astra pulls up my fine-tuned preset of “All that Jazz” in “The Giant”, a sampled upright grand piano by Native Instruments.

Astra is the first model that I have been able to get to improvise over many hours. With the codex-daw plugin, I can set up a 3-minute loop and ask the model to begin by adding one note to the empty MIDI loop each pass. I tell it what key it is in and ask it to improvise with velocities, note lengths, pitches within a key and utilize some of the data points it picked up from analyzing how I choose notes in my past MIDI regions. This is also the first model to help me begin embedding a small 12.5-million-parameter neural network trained on that dataset directly into my music software.

Astra has taken my research into generative models trained on only my raw audio to the next step. In the past, the results from my VAEs were distorted and extremely muffled. Astra’s leaps in coding have allowed us to improve on the fidelity in specific frequency ranges. This is a novel step forward for me, because the latent space is becoming so rich with possibilities and deeply learned choices. To start, these early 5-10 second returns get fed into my macOS app and turned into new instruments that I compose with.

Artistically, to be clear, nothing at all is new with this overall workflow. I never, unless an artist hires me to remix, start with another artist’s piece of audio in Ableton Live and then get down to making music. I find the technology fascinating, but the applied use by generative music companies so far dodgy at best. If I were to make music with some of these tools, I could put myself or my clients at legal risk. And for what? I did not need help with the composition. As a lifelong composer, I search for new sounds, new ways of making sounds, new ways of manipulating sounds and new ways of sharing the music I make with what I find.

I hope to release the plugin in ChatGPT as an open source project free for others to fork soon.

I’ve been releasing music online for over 20 years, and I have boiled the process down to fourteen steps, or standard operating procedures. Step 7 is the only step that includes “making the actual” music. Professional musicians know that the other thirteen steps often take much longer than making the music itself. Music is my craft, and I am good at step 7. People come to me for music because of step 7. Astra is much faster, more accurate and more patient than I am at all of the other thirteen. Some years, I put out 10+ albums, and so my first company hire would be someone who would maintain the released library and manage all of the release steps for new music.

My library contains 74 albums, 606 tracks each with 20 points of metadata, mp3 audio files, WAV audio files and album art, and lives in my company’s database, my third-party distributor’s database, Bandcamp and twice in Stripe for both Test and Live products. Thousands of opportunities to make mistakes and be off by a single letter or space. With Astra, along with Computer Use and plugins by Stripe and Supabase, I now have a single manifest that found about 40 mismatches yesterday and corrected them asynchronously and autonomously end to end.

This is new to Astra, and in my testing, the accuracy is easy to validate. You can use Chrome or ChatGPT Browser to have Astra perform customer journeys and checkout flows. These checks run weekly using Automations, and with Astra, they can return a fix as quickly as they can find an issue. This has extended to using Computer Use through a daily Automation that can surface customer issues, find the error in the code and push a fix while I test and respond that the issue has been resolved.

The performance increase, along with the improved efficiency of Astra, has cracked open fundamental knowledge work tasks that used to take hours. For every surface my company distributes music through, there is a separate dashboard with how/where the music is performing. Another amazing discovery I made during testing was that Astra could reliably go from website to website and daily collect changes.

With this information, Astra started a repo that contains a dashboard for all of my music plays and downloads globally! I combined that with ChatGPT’s Finance tool and with Stripe and can now see how it all exactly flows together to make a livable wage. Then, with all of the intelligence packed into Astra, I can use the new Deep Research plugin each day to surface advice and coaching around ways of improving my services and staying focused on music that is responding really well.

Yes…the bane of the artist’s day-to-day! If you have practiced your own bookkeeping and managed most of your accounting, AI can take over the systems you have made. If you combine the QuickBooks plugin with ChatGPT, Computer Use can assist in reconciliation, bank connections, expense tracking, cash flow and account health across vendors and clients. Astra is so much more reliable when I am trying to fix something in a buggy balance sheet than watching several YouTube videos or searching for the docs.

What I have found is that Astra has a sense of a website’s entire sitemap even before the site finishes loading. This has proven so helpful, because it knows exactly what submenu contains the only link to a page that contains the only menu with the only link to the page you need. For example, my banking company just requested all users to transition to a new account platform. My company has 5 separate accounts, many more vendors and exponentially more clients. Computer Use can open directly up to the page where I need to update ACH info, or credit card info, saving time when needing to do this over and over again.

After I performed a few of the account updates myself, I decided to allow Astra a shot at doing one for me. It was more exciting than embarrassing how much faster it completed the data entry task than I ever had. If I could hire an employee, this is exactly the job I would give them: “Here are the old numbers, here are the new numbers and here is everywhere that you need to change them out…I think. If you find any others, let me know.”

I watched in profound delight as Astra opened over twenty tabs at once and started updating everything at that velocity. That was about a week’s worth of work, and it finished in a few hours while I practiced my harmonic minor scales.

I have been working on a History of Music and descriptions of every instrument as educational materials for years. AI has helped me go much deeper into that specific area of my domain expertise. In the past, we would spend weeks before a semester at the copier making packets for students for the semester. That turned into social media groups where we could share online resources as a class, and now I can share articles, videos and all types of educational media.

Instrument by instrument, musician by musician, I am using ChatGPT and Codex to research and create online resources for students. I write the initial editorial, and AI helps me with the research, image sourcing and software needed to share online. Some friends of mine recently introduced me to Blender, and now I can make MIDI-playable, exploding views, interactive teaching environments around each instrument!

Explore and play the Rhodes

What’s next? So much more. I am busy finalizing the new software to release and share. There are many other amazing discoveries that I was able to make while testing GPT-6 Astra. I am incredibly grateful to OpenAI. They are the most welcoming group of people I have ever met. It is critical that the big companies work alongside both famous artists and, even more so, working artists tending a bespoke tribe of dedicated fans.

Last year, many companies negatively labeled musicians like me working with code as "just vibe-coders", but OpenAI leaned in. They saw that the whole point was empowering folks like me, and they were very right here. Everyone I have ever met at OpenAI has shared a genuine curiosity in my wildest and most impossible ideas. That is true maker and founder vibes, and such a joy to be around.

Come find me at OpenAI Dev Day 2026. I’m so excited to share and talk with more creatives at the event.