Stem Separation Explained: How AI Splits Music Into Parts (2026)

If you've used an AI tool to remove vocals or isolate drums from a song, you've experienced stem separation. But how does it actually work? Why does it sound so good on some songs and leave obvious artifacts on others? And what's happening inside these models that makes them better every year?

This article covers the full picture — the science, the practical limitations, and what you can actually expect when you run a song through a stem separator in 2026.

What Is Stem Separation?

Stem separation (also called source separation, audio demixing, or music unmixing) is the computational process of decomposing a mixed audio recording into its individual components. A typical pop song that arrives as a single stereo file contains everything recorded together: the lead vocal, harmonies, kick drum, snare, bass guitar, guitars, keys, and whatever else the producer added. Stem separation attempts to mathematically reverse that mixing process.

The word "stem" comes from music production terminology. In a recording studio, a stem is a submix — a single audio file that contains a group of related instruments mixed together. A producer might export a "drums stem" containing all the drum tracks, a "vocals stem" with all vocal layers, and so on. These stems make it easier to hand off a project to a mastering engineer or sync licensor without sharing every individual track.

Modern AI stem separation produces these same outputs — just working backwards from the final stereo mix rather than from the original multitrack. For a broader look at what you can do with separated stems, see our AI stem splitter guide.

The four standard stems most tools target are:

• Vocals — lead vocals, backing vocals, harmonies, and any spoken word
• Drums — the full drum kit: kick, snare, toms, hi-hats, cymbals, and percussion
• Bass — bass guitar, synth bass, and low-frequency tonal instruments
• Other — everything that doesn't fit the above: guitars, keyboards, synths, strings, horns, effects

More advanced 6-stem models add Guitar and Piano as separate outputs, though these are harder to isolate reliably because guitars and keyboards overlap in frequency range with many other instruments.

Why Old Methods Failed

To appreciate how impressive modern AI separation is, it helps to understand what came before.

Phase Cancellation (1960s–2010s)

The oldest trick was phase cancellation. If you take a stereo file and invert one channel, then sum them to mono, anything that's identical in both channels cancels out. In many pop mixes, the lead vocal was recorded in mono and panned to center — so it appeared identically in the left and right channels. Inverting and summing removed it.

The obvious problem: anything panned to center gets removed along with the vocals. That typically included kick drum, bass guitar, snare, and any other center-panned elements. The "instrumental" you got was hollowed out. And if the vocal had any stereo effects (reverb, chorus), those parts survived the cancellation and left ghostly artifacts behind. This method is why old-school "vocal removers" sounded so bad.

Frequency Filtering

A cruder approach: cut the frequency bands most associated with the human voice (roughly 200 Hz to 3 kHz). This damaged the entire mix because guitars, piano, and drums all occupy those same frequencies. You'd mute the vocal somewhat while destroying everything else in the process.

Spectral Editing

Manual spectral editing using a tool like iZotope RX lets a skilled engineer paint over vocal components in a spectrogram view and replace them with silence or noise-filled approximations. This can work remarkably well for small sections — cleaning up a region where a door slammed, or removing a short spoken phrase. But doing it for an entire song takes hours of expert work, and the results still show obvious seams.

None of these approaches could produce clean, full-length separated stems. AI changed that completely.

How Modern AI Stem Separation Works

Today's systems are deep neural networks trained to recognize the spectral and temporal patterns that distinguish one sound source from another in a mix.

Step 1: Converting Audio to a Spectrogram

Raw audio is a waveform — a time-series of pressure values sampled thousands of times per second. But neural networks work better with a 2D representation that shows how the frequency content of a signal changes over time. The standard tool for this conversion is the Short-Time Fourier Transform (STFT), which produces a spectrogram: a grid where the horizontal axis is time, the vertical axis is frequency, and the brightness (or color) at each point represents how loud that frequency is at that moment.

You can think of it like a piano roll, but instead of showing discrete notes, it shows the full continuous frequency spectrum of everything playing at once. A snare hit looks different from a guitar chord, which looks different from a sustained vocal note — all of which look different from bass guitar. The spectrogram makes those differences visible.

Step 2: What the Neural Network Actually Learns

The model is trained on a dataset of songs where the isolated stems are already known — essentially learning from thousands of multitrack recordings that have been "mixed" in software. For each training example, the model sees the spectrogram of the mixed song alongside the spectrograms of each individual stem.

During training, the network learns that certain spectral shapes, textures, and movement patterns are associated with vocals versus drums versus bass. It learns that bass frequencies tend to have long, smooth sustained notes. That drums create sharp vertical lines (transients) across many frequencies simultaneously. That vocals have distinctive formant structures — resonant frequency patterns specific to the human voice that shift as vowels change.

The most capable modern architectures, like Hybrid Transformer Demucs (HTDemucs), process the audio signal in two parallel streams simultaneously:

• A time-domain stream that operates directly on the raw waveform and is especially good at capturing transients, timing, and phase relationships
• A frequency-domain stream that operates on the spectrogram and excels at identifying tonal content, harmonics, and spectral texture

A transformer-based attention mechanism then allows information to flow between these two streams, letting the model combine what it learns from each. This dual-path approach is why HTDemucs outperforms earlier models that only used one representation.

Step 3: Mask Estimation

Rather than directly predicting the output waveform for each stem, most architectures work by estimating a mask — a grid the same size as the input spectrogram where each cell contains a value between 0 and 1, representing how much of that frequency at that time belongs to that particular stem.

Apply the drums mask to the original spectrogram, and you suppress everything that doesn't look like drums. Apply the vocals mask, and you suppress everything that doesn't look like vocals. Soft masking (where values are fractional rather than binary 0/1) preserves more audio quality than hard cuts, which is why it became standard.

Step 4: Converting Back to Audio

Once the masked spectrograms are computed for each stem, the final step is converting them back to audio waveforms using the Inverse STFT. The result is a set of audio files — one per stem — that together should approximately reconstruct the original mixed track when summed.

The quality difference you hear between tools usually comes down to: how much training data they used, how large and sophisticated the model architecture is, and whether the masking strategy preserves fine details like reverb tails and attack transients.

The Main AI Models Compared

A handful of open-source models have defined the state of the art since 2019. Understanding their differences helps explain why various tools sound the way they do. All SDR figures below are for the vocals stem on the MUSDB18 benchmark, the standard test set used across the research community.

Quick Comparison

Spleeter (Deezer, 2019)

Spleeter was the first widely accessible open-source model for stem separation and it had a huge impact — suddenly anyone could run reasonable-quality separation on their own machine. It uses a U-Net architecture (an encoder-decoder network with skip connections) operating purely in the frequency domain.

Strengths: Fast, lightweight, runs on modest hardware, easy to deploy, still adequate for basic karaoke use.
Weaknesses: Bleed is clearly audible in many songs, struggles with reverb-heavy vocals, 5-stem mode quality drops noticeably compared to 2-stem.
Vocals SDR: ~6.5 dB on MUSDB18.

For a direct head-to-head comparison of Spleeter versus Demucs across genres with SDR tables, see our Spleeter vs Demucs breakdown.

Demucs v3 (Meta/Facebook, 2021)

Demucs v3 introduced time-domain processing alongside frequency-domain and was a significant jump over Spleeter. It processes the raw waveform directly using an encoder-decoder network with gated convolutions, which gives it better phase coherence and sharper transients — particularly noticeable on drum and bass stems where timing matters.

Strengths: Cleaner transients than Spleeter, better phase coherence, good results on CPU without a GPU.
Weaknesses: Slower than Spleeter, doesn't benefit from Transformer context modeling, HTDemucs produces better results for similar compute cost.
Vocals SDR: ~7.3 dB on MUSDB18.

HTDemucs (Meta/Facebook, 2022)

The current mainstream standard. HTDemucs adds a Transformer attention mechanism that lets the model reason about longer temporal contexts — understanding that the bass note playing in bar 1 is probably related to the bass note in bar 3, not randomly changing. This long-range dependency modeling is what makes it handle complex harmonic music better than earlier convolutional-only models.

Strengths: Best overall quality across all four stems, handles dense mixes well, cleaner reverb tails, better bass separation, available as free open-source via the Demucs GitHub.
Weaknesses: Slower than Spleeter, requires more compute, occasional low-frequency bleed on very dense mixes.
Vocals SDR: ~8.7 dB on MUSDB18 — roughly 35% higher quality than Spleeter on the standard metric.

HTDemucs Fine-Tuned (htdemucs_ft) is a version further trained on a smaller curated dataset. On average tracks it performs similarly to base HTDemucs, but on certain genres (particularly pop and R&B) it produces noticeably cleaner results. StemSplit runs htdemucs_ft on all jobs by default. HTDemucs 6s (htdemucs_6s) adds guitar and piano as separate outputs — useful for production work, though accuracy on those extra stems is lower than the standard four.

MDX-Net (Music Demixing Challenge, 2021)

MDX-Net models emerged from the Sound Demixing Challenge and use ensemble approaches — combining multiple models whose outputs are averaged together. The mdx_extra variant was trained on additional data beyond the standard benchmark set.

Strengths: Competitive with or slightly better than HTDemucs on vocal isolation specifically; useful when vocal quality is the only priority.
Weaknesses: Lags behind HTDemucs on drums and bass; slower due to ensemble computation; less general-purpose.
Vocals SDR: ~9.0 dB on MUSDB18 for mdx_extra — strong on vocals, weaker on other stems.

In practice, the difference between htdemucs_ft and mdx_extra on real-world music is smaller than the benchmark numbers suggest. Which is "better" often depends on the specific song's genre and mix characteristics.

BS-RoFormer (2024) — Current State of the Art

The newest generation. BS-RoFormer (Band-Split RoFormer) applies a Roformer (a Transformer variant with rotary position embeddings) to the band-split signal representation, where the audio spectrum is divided into sub-bands and each band is processed with dedicated attention heads. This architecture allows the model to reason simultaneously about fine-grained frequency detail and long-range temporal structure.

Strengths: Best vocal SDR of any published model as of 2026, significantly reduces the metallic artifacting common in older models, handles reverb-heavy vocals exceptionally well.
Weaknesses: Compute-intensive, not yet as widely deployed as HTDemucs, requires newer infrastructure to run efficiently.
Vocals SDR: ~10.9 dB on MUSDB18 — a meaningful improvement over HTDemucs, particularly audible on challenging material.

BS-RoFormer is beginning to appear in commercial tools and represents the direction the field is heading.

What Separation Quality Actually Sounds Like

SDR numbers are useful for comparing models objectively, but they don't tell you what the artifacts sound like in practice. Here's what to expect.

The Bleed Problem

No AI model produces perfectly isolated stems. Some signal from adjacent sources always "bleeds through" into each output. On an isolated vocal track, you'll typically hear a faint ghost of whatever was loudest in the mix — often kick drum or snare. On an instrumental track, you'll hear a faint, somewhat processed-sounding shadow of the vocal.

The bleed is usually subtle enough that it doesn't ruin the use case. For karaoke, a whisper of vocal artifact under the instrumental is far less noticeable than the original full-volume voice. For sampling drum breaks, a trace of bass under the drum track is usually inaudible in context.

Where bleed becomes a real problem is in professional mixing workflows — if you're trying to remix a commercially released track and need truly clean stems for a club release, AI separation typically isn't clean enough without additional manual processing in iZotope RX or similar tools.

Reverb Tails and Transients

Reverb tails are one of the hardest things for AI models to handle. When a vocalist sings a long reverb-heavy phrase, the reverb tail extends past the vocal and smears into the spectrogram in a way that's hard to attribute cleanly to either the vocal or the background. Modern models do a decent job, but you'll often hear the reverb behave slightly differently on the separated stem than it does in the original mix.

Transients — the sharp attacks of drums, guitar picks, and piano keys — are much better handled now than they were with frequency-domain-only models. HTDemucs's time-domain stream preserves attack detail well enough that separated drum tracks typically sound punchy and natural.

Genre Matters Significantly

Separation quality varies substantially by genre:

Studio pop and electronic music tend to separate best because modern production already keeps instruments in relatively distinct frequency zones. Jazz and classical are hardest because everything is deeply harmonically intertwined and recorded with natural room acoustics.

When AI Separation Fails (and Why)

Understanding failure modes helps you predict when you'll get good results and when you won't.

Heterophony and Doubling

When two instruments play the same note simultaneously — a vocal doubled by an octave piano line, or guitar playing in unison with keyboards — the model often can't decide which stem to put it in. You'll get the doubled part split between two outputs, or assigned entirely to the wrong one.

Extreme Effects Processing

Heavy vocal distortion (like auto-tuned, pitch-shifted or vocoded vocals) confuses models that have learned to recognize the natural formant patterns of the human voice. A robot-voice effect that changes the spectral character of the vocal may cause it to bleed into the "other" stem instead of being cleanly isolated.

Similarly, extreme guitar effects like fuzz pedals and octave effects change the spectral fingerprint of the guitar enough that the model may misclassify some of it.

Bass Frequency Confusion

Sub-bass frequencies below around 80 Hz are difficult to separate because everything in that range — bass guitar, kick drum, synthesizer bass — blends into a rumbling mass in the spectrogram. 808-style kick drums that extend down into sub-bass territory are particularly problematic, as they overlap spectrally with bass lines. You'll often hear 808s bleeding into both the drums and bass outputs.

Low-Quality Source Files

Running a 128kbps MP3 through a stem separator doesn't just give you lower-quality output — it actively introduces artifacts because the MP3 compression algorithm has already discarded audio information and introduced ringing artifacts. The model may interpret those artifacts as instrument signal. For best results, use 320kbps MP3 or lossless files (WAV, FLAC).

Practical Use Cases and What to Expect

Karaoke Track Creation

This is the highest-volume use case and where AI separation works best in practice. A clean, modern studio recording separated with HTDemucs will give you an instrumental track where the vocal bleed is subtle enough to not be distracting. Adding a slight low cut and multiband compression to the instrumental output improves results further. See our roundup of the best vocal remover tools for a comparison of services.

StemSplit's vocal remover runs htdemucs_ft to optimize specifically for this use case.

Music Practice and Learning

Isolating stems for practice is another excellent use of the technology. Removing guitar from a recording so you can play along, or extracting just the piano part so you can learn it by ear, works well with modern models. The slight bleed from other instruments doesn't interfere with learning the part. You can also use the isolated stem to hear fine details of a player's technique that are buried in the full mix.

DJ Mashups and Live Performance

DJs use stem-separated acapellas and instrumentals for mashups and live stem mixing. The quality bar here is more forgiving than studio production — in a club environment with a loud soundsystem, vocal bleed at a low level is inaudible. Artists like Girl Talk built entire careers around rough vocal-over-instrumental combinations with far lower quality than modern AI produces.

Production and Sampling

This is where AI separation bumps against real limits. If you're using isolated drums or bass in a new production, you'll need to clean up the stems in iZotope RX or similar before they're release-ready. For sample-based hip-hop production, the drums stem from an AI separation is typically usable after a high-pass filter and some EQ to remove bass bleed.

For commercial releases that strictly require clean samples, AI separation alone usually isn't enough — you need the original multitrack session.

How to Get the Best Results

Use the Highest Quality Source File Available

Lossless is better than MP3. 320kbps MP3 is significantly better than 128kbps. If you have a FLAC or WAV version, always use that. Low-bitrate files introduce compression artifacts that the model can mistake for instrument signal, degrading every stem output.

Avoid Live Recordings and Bootlegs

Studio recordings always separate more cleanly because the mix was controlled. Live recordings have natural room ambience, crowd noise, and stage bleed between microphones that confuse the model. Even a great live recording will produce noticeably worse stems than an average studio mix of the same song.

Test a Short Clip Before Committing

Before processing a full 5-minute track, upload a 30-second section of the most complex part (usually the chorus) to verify the quality is adequate. If the output is problematic on the chorus, it won't improve elsewhere. This is especially useful when you're unsure whether a particular genre or production style will separate well.

Post-Process the Output

A gentle high-pass filter (around 80–100 Hz) on separated vocal stems removes sub-bass rumble from bleed. A transient shaper can restore punch if the drum stem sounds slightly soft. Multiband compression helps tighten an instrumental that has occasional vocal ghosting. These corrections usually take under two minutes and noticeably improve the output's usability.

Calibrate Your Expectations

AI separation in 2026 is genuinely impressive — far beyond what was possible even four years ago. It's not magic. Understanding what it can and can't do helps you use it effectively rather than being surprised by its limits. If you want to run it locally on your own machine, our Demucs setup guide walks through the full installation process. If you prefer a no-setup online option, our Demucs online tutorial covers that workflow.

Try It Yourself

The best way to understand stem separation is to run a few tracks and listen critically. StemSplit's stem splitter runs htdemucs_ft on GPU-backed servers and processes tracks in under two minutes. Upload any song and get a 30-second preview before committing — no account required.

Split a Song Free →

---

FAQ

Is AI stem separation perfect?

No. Current models produce very good results — most users find the output useful for their purposes — but there's always some bleed from adjacent instruments. The quality difference between 2022 and 2026 models is significant, and improvement continues. BS-RoFormer represents the current ceiling; future architectures will push it further.

What's the difference between stem separation and vocal removal?

Vocal removal is a specific application of stem separation focused on the vocals/instrumental split. Stem separation is the broader process of decomposing a mix into any number of components (vocals, drums, bass, other instruments). All vocal removers do stem separation, but not all stem separators are used for vocal removal.

How does it compare to having the original stems from the studio?

Original studio stems are always better — they have perfect isolation because no mixing has occurred. AI-separated stems will have some residual bleed that original stems don't have. For professional production use, original stems are preferable when available. For everything else, AI separation is a practical alternative.

Which model should I use?

For most purposes: htdemucs_ft. It produces the best general-quality results across all four stems. Use htdemucs_6s if you specifically need guitar and piano separated. Use mdx_extra if vocal isolation quality is your only concern and you're willing to accept worse drums/bass output. Spleeter is only worth considering if you're processing at very high volume and speed matters more than quality.

What format should I upload for best results?

WAV or FLAC (lossless) gives the best results. High-quality MP3 (320kbps) is nearly as good. Avoid low-quality MP3s (below 192kbps) as compression artifacts degrade separation quality.

Why does my isolated vocal still have some music under it?

This is the bleed artifact described above — a fundamental limitation of current masking-based approaches. The amount of bleed depends on the genre, mix quality, and model used. HTDemucs-based tools minimize it more than older models, but it doesn't disappear entirely. BS-RoFormer reduces it further, particularly on reverb-heavy vocals.

What will stem separation look like in five years?

The trajectory points toward continued improvement on challenging genres, better handling of extreme effects processing, and likely the ability to separate individual drum elements (kick, snare, hi-hat) with high accuracy. Real-time separation on consumer hardware is already becoming practical. The architectural shift from masking-based CNNs to Transformer and diffusion-based generative approaches will likely produce stems that are difficult to distinguish from original multitracks on most material.