Training a 125M-parameter Model to Autocomplete Piano
- TL;DR: I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15).
- The biggest improvements came from finding the right MIDI representation, cleaning the training data aggressively, and adding DPO post-training.
- Almost a year ago, I started tinkering with an idea: connect my MIDI piano to my phone, play something, and have AI autocomplete the song for me.
Unverified
- TL;DR: I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15).
- The biggest improvements came from finding the right MIDI representation, cleaning the training data aggressively, and adding DPO post-training.
- Almost a year ago, I started tinkering with an idea: connect my MIDI piano to my phone, play something, and have AI autocomplete the song for me.
Sources: Simedw