How Piano Genie turns eight buttons into an 88-key piano

An eight-button keyboard cannot assign a permanent button to every note on an 88-key piano. Piano Genie makes the assignment change as you play. Each press requests one note; the model chooses its pitch from the musical context. The player keeps the rhythm.[1]

Chris Donahue, Ian Simon and Sander Dieleman introduced the system in 2018. Higher and lower buttons suggest melodic direction without fixing the next note.[1]

Teaching the shortcut backwards

Training starts with expert piano performances. One network, the encoder, turns note sequences into button sequences; a second, the decoder, tries to reconstruct the notes. After training, the encoder is removed and a person supplies the buttons. Donahue’s 2021 tutorial walks through this separation.[3]

The network works with digital note instructions. A synthesiser turns the predictions into sound. Choosing a pitch and generating its audible waveform are separate jobs.[3]

The training also discouraged reversals of melodic shape, such as encoding a rising pair of notes as a falling pair of buttons. That gives a newcomer something to steer. A model trained on about 1,400 competition performances supplied the musical vocabulary; the authors noted that the training repertoire shaped the result’s style.[1]

Consider a simple gesture: three quick presses moving towards the right, followed by a long pause and one press towards the left. It expresses pace, space and a change of direction without naming a key signature. The player can hear what comes back and decide whether the next gesture should answer, repeat or interrupt it. This is an illustrative way to think about the interface, not a transcribed performance.

Enjoying the playing is a separate result

A small 2019 study compared a fixed G-major scale, an automatic next-note model unaffected by button identity, and Piano Genie. Eight participants with varied piano experience had up to three minutes on each.[2]

Average enjoyment of performing, on a five-point scale, was 4.375 for Piano Genie, 3.750 for the automatic model and 3.125 for the fixed scale. Enjoyment of the resulting music was 3.125, 3.125 and 3.250 respectively. The tiny study supports no broad statistical conclusion; its useful distinction is between liking the activity and liking the sound.[2]

Several participants also wanted a repeated button pattern to yield the same notes. That frustration identifies a musical capability the changing mapping made harder: returning to an idea deliberately.[2]

A satisfying instrument needs relationships a player can gradually learn. Surprise can invite another phrase; a recognisable response can let someone develop it. The challenge is to offer both without requiring the fingering knowledge the small keyboard was designed to bypass.

A small interface to explore

The author-hosted demo documents keyboard numbers 1–8 or touch controls, with the space bar operating sustain. Its instructions recommend landscape orientation on a phone. It also exposes a four-button option.[4]

A listening exercise to try is a short pattern, a pause and the same pattern played at a different speed. Listen for what remains yours: the entry of each note and the gap before the reply. Then listen for what the model changes.

The researchers’ physical prototype found an unexpected use for the divided controls. A hardware constraint led them to build two boxes of four buttons. Two users took one box each and improvised a duet. The miniature keyboard had made room for another player.[2]

Report an error or suggest a correction