Engram gives musicians a knob for bending an AI’s sound

Evan King has put a physical control on a part of generative AI that musicians would usually never touch. In his account of building Engram, a hardware sampler, he describes altering the audio decoder and attaching knobs to the resulting parameters. He calls the technique “model bending”, after the practice of modifying electronic instruments to discover unexpected sounds.[1]

That makes the project interesting for a particular reason: a useful musical sound need not be a convincing imitation of the object named in a prompt. A strange attack, unstable texture or short burst of noise can become something worth keeping once a musician chooses where it belongs. The instrument’s job is to make those choices possible.

The point where a representation becomes sound

King’s February engineering account identifies MusicGen as the basis of the prototype’s sample generation. Meta’s documentation explains the underlying division of labour: MusicGen predicts sequences of compressed audio tokens, while an EnCodec model provides the representation used to encode and decode the sound. Those tokens describe audio, rather than a conventional score of notes for a synthesiser to play.[1][3]

This distinction gives a useful way to understand the control. Imagine keeping a musical instruction broadly similar while changing the machinery that gives it a timbre. That is an analogy, rather than a claim that a decoder is a conventional filter. A learned decoder has its own relationships between internal values and audible output; disturbing them can produce results that are difficult to specify in ordinary musical language.

MusicGen itself supports conditioning on text and, for its melody models, melodic information. Engram’s additional idea is to expose the sound-making process as something to manipulate after choosing a general direction. King describes custom embedded inference code that made his decoder experiments available through the device’s controls.[3][1]

A sound still needs somewhere to go

Thoughtful Things’ current product page describes two routes into the instrument: process a recording through a neural audio codec, or generate short sounds to capture and reshape. It lists sample and pattern recall, a microphone, stereo line input, headphone output, MIDI connections and control-voltage synchronisation. It also proposes interchangeable audio models.[2]

The sampling workflow matters as much as generation. Consider an illustrative session: record a struck object, change its character, keep one appealing fragment and place it into a repeating pattern. The musician then has decisions about duration, rhythm, repetition and the relationship to other instruments. Producing the first sound has created more opportunities to compose.

Physical connections are consequential here. An instrument that can take in another sound source and exchange timing information can fit into an existing setup. The value is partly in continuity: the player can bring familiar recording and sequencing habits to unfamiliar audio material.

The project remains a prototype. King’s September update says its first production run was funded through Kickstarter, while the product page still warns that the design can change. Those public descriptions establish the intended instrument and its development status; they do not provide our own hands-on assessment of its responsiveness or finished hardware.[1][2]

A good question for Engram is therefore what happens after it produces a sound. Can the player find it again, shape it, combine it and make it matter in a piece? That is where an unusual model output can acquire musical purpose—and where this kind of AI instrument has to earn its place.

Report an error or suggest a correction