Home/Blog/Show HN: I Trained a 125M Model to Autocomplete Piano On-Device – A Paradigm Shift for AI Agents in Creative Workspaces
virtual coworking
Show HN: I Trained a 125M Model to Autocomplete Piano On-Device – A Paradigm Shift for AI Agents in Creative Workspaces
Show HN: I Trained a 125M Model to Autocomplete Piano On-Device – A Paradigm Shift for AI Agents in Creative Workspaces The recent Why piano autocomplete? I started with piano beca
13 MIN READ
20 Aug 2026
virtual coworking
Show HN: I Trained a 125M Model to Autocomplete Piano On-Device – A Paradigm Shift for AI Agents in Creative Workspaces
I started with piano because it sits in a sweet spot: the structure is rich enough to be interesting, but the input/output format is still compact enough to fit on a phone or laptop without streaming audio or running a giant model in the cloud.
A piano roll is also a very clean representation of musical intent. Instead of predicting raw waveforms, the model only needs to decide:
which notes are active,
when they start and stop,
how long they sustain,
and how the next few beats should evolve.
That makes it a great testbed for on-device generation. If a model can learn to continue a musical phrase locally, with low latency and without sending data anywhere, then the same basic pattern can be extended to other creative tools: MIDI composition, drum sequencing, live accompaniment, even code or UI generation in constrained environments.
02What “autocomplete” means in practice
The goal was not to generate a full polished composition from scratch. It was closer to the kind of autocomplete people already expect in text editors:
you play or enter a short phrase,
the model observes the context,
it proposes a continuation,
and you can accept, reject, or modify it in real time.
That framing matters. A creative assistant does not need to “be the artist.” It needs to stay out of the way, preserve the user’s style, and respond fast enough to feel like part of the instrument.
For piano, that means the model should be able to continue a melody, suggest harmonic movement, or fill in a rhythmic pattern while keeping the result musically coherent. The most important metric is not just loss or perplexity, but whether the output feels usable to a human musician.
03Dataset and representation
The biggest design decision was how to represent music for the model.
I experimented with a few options, but the most practical one was an event-based token stream derived from MIDI. Instead of trying to model audio directly, I converted performances into sequences of discrete events such as:
note on,
note off,
time shift,
velocity bucket,
and sometimes pedal events.
This approach has a few advantages:
It is compact.
It preserves timing.
It works well with transformer-style language modeling.
It makes generation deterministic enough to run on-device.
The training data came from a mix of classical, jazz, and contemporary piano MIDI files. I wanted enough diversity to avoid overfitting to one style, but not so much variety that the model would lose the ability to learn consistent phrasing. In practice, that meant cleaning the dataset carefully:
removing malformed files,
normalizing tempo where appropriate,
filtering out extremely sparse or noisy sequences,
and splitting long pieces into manageable segments.
The preprocessing pipeline ended up being just as important as the model itself. For creative generation, garbage in really does mean garbage out.
04Why 125M parameters?
The number sounds arbitrary until you try to fit the whole system into a device-friendly budget.
A model around 125 million parameters is large enough to learn meaningful structure, but still small enough to be realistic for local inference with quantization. It is also a familiar scale: big enough to benefit from transformer architecture, small enough that you can actually iterate on latency, memory, and quality without needing datacenter infrastructure.
I did not want a model that only worked in a notebook demo. I wanted something that could plausibly run:
on a laptop without a GPU,
on a tablet,
or inside a desktop app as a background assistant.
That constraint changed several architectural choices. For example, I prioritized:
efficient attention patterns,
short context windows that still captured musical phrases,
aggressive quantization for inference,
and a decoding strategy that could produce stable output quickly.
The result is not “state of the art” in the abstract benchmark sense. But it is very good at the thing I actually care about: practical, local, interactive music continuation.
05Training setup
Training followed the usual language-model recipe, but with a few music-specific adjustments.
The model learned next-token prediction over the event sequence. During training, it saw many partial piano passages and learned to predict the next event given the preceding context. Because music has a strong temporal structure, the model quickly picked up on recurring patterns like:
repeated motifs,
cadences,
rhythmic tension and release,
and left-hand/right-hand coordination.
A few things turned out to matter a lot:
Context length
Too short, and the model forgets the phrase. Too long, and training becomes expensive while inference slows down. I found that a moderate context window gave the best trade-off for on-device use.
Tokenization granularity
If the representation is too coarse, the output sounds robotic. If it is too fine, sequences get bloated and harder to learn. The sweet spot was a tokenization scheme that preserved timing detail without exploding sequence length.
Regularization
Music models can overfit surprisingly fast, especially on smaller datasets. I used standard regularization techniques and monitored not just validation loss, but also qualitative samples. Sometimes a model with slightly worse loss produced much more musical continuations.
Sampling strategy
The decoding method had a huge effect on musicality. Greedy decoding often made the output too repetitive. Too much randomness made it drift away from the input. The best results came from a controlled sampling setup that balanced coherence and variation.
06On-device inference: the real challenge
Training was only half the story. The real engineering problem was making the model usable locally.
On-device inference introduces constraints that cloud demos can ignore:
memory is limited,
latency matters,
battery matters,
and the user expects immediate feedback.
A creative assistant cannot feel like a batch job. If the system takes several seconds to respond, it stops feeling like an instrument and starts feeling like a render queue.
To make this work, I focused on a few optimizations:
Quantization
Reducing model precision was essential. Quantization dramatically lowered memory usage and improved deployment feasibility, especially for consumer hardware.
Efficient runtime
I tested different inference backends and execution paths to minimize overhead. In practice, the runtime mattered almost as much as the model size.
Incremental generation
Instead of recomputing everything from scratch, the system caches past context so it can extend sequences efficiently. That makes a big difference for interactive use.
UI responsiveness
The interface needs to show that the system is “thinking” without blocking the user. Even if generation takes a moment, the experience should still feel fluid.
Once those pieces came together, the model became usable in a very different way from a typical server-hosted AI. It could sit inside a local creative workflow and respond immediately, without network dependence.
07What the model learned well
The most encouraging part was how quickly the model picked up basic musical grammar.
It learned to:
continue repeated motifs with small variations,
maintain rhythmic consistency over short spans,
suggest harmonically plausible note sequences,
and avoid some of the most obvious random-note failures.
It was especially good at short-term coherence. If you gave it a phrase with a clear contour, it often extended that contour in a way that felt intentional. It also handled accompaniment patterns reasonably well, especially when the input already implied a chord progression.
In other words, it did not need to “understand music” in a human sense to be useful. It just needed to model enough local structure to make the next few bars feel natural.
08Where it still fails
Of course, there are limits.
The model can still:
drift into repetitive loops,
lose long-range form,
overuse certain rhythmic patterns,
and occasionally produce awkward voice leading.
It also struggles with very long compositions where global structure matters more than local continuity. That is not surprising. A 125M model with a modest context window is not going to rival a human composer’s sense of architecture.
Another issue is that music quality is subjective. A sequence that looks statistically plausible can still feel dull or derivative. This is one reason I think human-in-the-loop design is essential. The model should propose, not dictate.
09Why this matters beyond music
This project is really about a bigger idea: creative AI does not have to live in the cloud.
There is a strong assumption in many AI products that bigger models, remote inference, and centralized infrastructure are always better. But for many creative tasks, local models offer advantages that are hard to ignore:
lower latency,
better privacy,
offline availability,
lower operating cost,
and tighter integration with the user’s workflow.
For musicians, that means you can improvise without sending your ideas to a server. For writers, it means autocomplete can happen inside the editor instantly. For designers, it means local suggestion engines can work inside the app without waiting on a network round trip.
The broader implication is that small, specialized models may become the default for interactive creative tools, while large cloud models remain useful for heavier reasoning or synthesis tasks.
10A few lessons from the build
If I had to distill the project into a handful of practical lessons, they would be these:
1. Representation matters more than people think
A good tokenization scheme can make a small model look much smarter than a bad one.
2. Latency changes behavior
If the model is slow, users treat it like a generator. If it is fast, they treat it like an instrument.
3. Musicality is not just loss
The best sample is often not the one with the best validation metric. Qualitative evaluation is mandatory.
4. Constraints are productive
Forcing the system to run on-device improved the design. It removed a lot of unnecessary complexity and made the product more focused.
5. Small models can still feel magical
A 125M model is not huge by modern standards, but in the right interface, it can create a genuinely delightful experience.
11What I’d do next
There are several obvious directions to improve the system.
One is better conditioning. Right now, the model can continue a phrase, but it could be much more useful if it accepted richer prompts such as:
key signature,
tempo,
style tags,
chord hints,
or a short user-provided motif.
Another is better control over form. Instead of only predicting the next events, the model could be guided by higher-level structure, such as verse/chorus-like sections or tension curves.
I also want to explore hybrid workflows where the model assists rather than replaces. For example:
generating a left-hand accompaniment while the user plays melody,
suggesting harmonizations for a selected phrase,
or filling in the next bar after a live performance.
Finally, there is room to improve the deployment stack itself. Better quantization, faster runtimes, and more efficient attention mechanisms could make the experience even smoother on consumer hardware.
12Closing thoughts
This experiment convinced me that on-device creative AI is not just viable, but genuinely compelling.
A small model that understands enough musical structure to autocomplete a piano phrase locally is more than a demo. It is a proof of concept for a different kind of AI product: one that is embedded, private, responsive, and shaped around the user’s workflow rather than the provider’s infrastructure.
The future of AI agents in creative workspaces may not be about replacing human creativity with giant remote models. It may be about building compact, specialized systems that behave more like collaborators: fast, local, and context-aware.
And for piano, at least, that future is already starting to sound pretty good.
For Show HN: I trained a 125M model to autocomplete piano on-device, Nonilion can be used as the practical AI-office example: a shared workspace where human teammates and AI agents keep discussion, decisions, and execution connected.
The reason Show HN: I trained a 125M model to autocomplete piano on-device keeps returning to Nonilion is simple: the topic becomes more useful when it turns into coordinated work, not just another article, chat, or dashboard.
13Why This Trend Matters for Nonilion
This trend matters to Nonilion because it points to a bigger change: teams are moving from simple calls toward persistent, AI-supported collaboration spaces. Nonilion can bridge live presence, meeting context, avatars, and follow-up work so the trend becomes a usable workflow instead of a headline.
14Shareable Extracts
The trend is not just "Show HN: I Trained a 125M Model to Autocomplete Piano On-Device – A Paradigm Shift for AI Agents in Creative Workspaces" - it is a signal that team coordination is becoming the next competitive edge.
Hot take: the teams that win from this shift will not be the ones with more meetings; they will be the ones with clearer shared context after every meeting.
If show hn: i trained a 125m model to autocomplete piano on-device – a paradigm shift for ai agents in creative workspaces keeps moving this fast, remote teams need a workspace where conversation, presence, and follow-up stay connected.
Show HN: I Trained a 125M Model to Autocomplete Piano On-Device – A Paradigm Shift for AI Agents in Creative Workspaces The recent Why piano autocomplete?
Instead of predicting raw waveforms, the model only needs to decide: which notes are active, when they start and stop, how long they sustain, and how the next few beats should evolve.
15Social Hooks
Everyone is talking about Show HN: I Trained a 125M Model to Autocomplete Piano On-Device – A Paradigm Shift for AI Agents in Creative Workspaces. The overlooked part is what happens to team workflows after the headline fades.
The uncomfortable question behind Show HN: I Trained a 125M Model to Autocomplete Piano On-Device – A Paradigm Shift for AI Agents in Creative Workspaces: are teams adapting their collaboration systems fast enough?
This is not a meeting trend. It is a coordination trend, and products like Nonilion sit right in the middle of that shift.
Piano Practice, Training, Diagrams & MIDI Note Recognition ...
www.youtube.com/watch
Author
This article on Show HN: I trained a 125M model to autocomplete piano on-device was generated by the Nonilion AI blog workflow using web research inputs and AI-assisted synthesis.
