[sdiy] ML / AI for music & synthesis (not coding!)

[Pete Hartman]

On Sat, Apr 25, 2026 at 6:35 AM Florian Anwander via Synth-diy <
synth-diy at synth-diy.org> wrote:

> Am 24.04.26 um 20:45 schrieb Eric Brombaugh via Synth-diy:
>
> A more interesting topic to me is the use of machine learning and AI
> methods for realtime synthesis / effects and music creation.
>
> Hmm, there we are again, where Roman wrote "I really like to know exactly
> how the thing I'm making is doing what it's doing in every detail." Also in
> synthesis for me it is the most satisfying part, that I know what I am
> doing. I don't want to use, what comes from the blackbox - even if it
> sounds "cool". I want to have it on my own.
>


Inspired by these discussions, yesterday I picked up Claude code and
applied it to a project that had lain fallow for 3 years, a trigger
sequencer project using a PIC.  It had been 90% done when I set it aside
because of some glitching that I suspected was my code but couldn't find.
Claude found the bugs quickly, explained them clearly so I learned
something about using an ISR and also about the logic mistakes I had made,
and now except for one unimplemented feature, it works perfectly.

This was more "expert advice" than "vibe coding" and the result is
definitely not a black box.

It's also not quite what Eric was meaning 🙂....

For my second trick I applied it to something much more mushy--I have
kicked around an idea for an aeolian harp module for several years now.  Dr
Rod Selfridge did his dissertation on and published a couple of papers
about the physics involved and the math needed to simulate this to generate
such tones.  He also put a pure data implementation out in the world.  I
have reread and studied and tried to pick apart the Pd to understand it,
but the academic writing style and the byzantine nature of deeply layered
Pd code has consistently blocked me from understanding it enough to
implement.
 (to be fair to him, I have very little experience with Pd so that
perception of the implementation could easily be my own limitations)

I pointed Claude at the papers and the Pd implementation and within a few
minutes it was able to digest the complexity and provide a block diagram
laying out the parts of the math and how they fit together.  I recognize
the parts from my own attempts to dig into this, but had never been able to
fully grasp the connections because the connective tissue and idioms of
pure data weren't natural to me.  I now feel I understand it well enough to
implement, and I will probably lean on Claude to assist with that part as
well.

This too is not a black box, and the final result will be sound generation
code that I am intimate enough with to understand and modify to refine it
on my own.  Again it is much more in the vein of expert advice.  Perhaps it
would be feasible for a complete stranger to reach out to Dr Selfridge, and
 his advice would be better and 100% human, but I expect he has better
things to do than tutor some dilettante who is just interested in a very
narrow implementation of what he understands so well.

Still probably not quite what Eric was meaning, I don't think, but my point
is that  these are tools that can be used without them turning into a black
box.  It's about how you engage with them.

Pete




>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://synth-diy.org/pipermail/synth-diy/attachments/20260425/76969ef2/attachment.htm>