How to Turn Your Voice Into a Synth: Voice to MIDI, Vocoder, Talkbox and More

There are five ways to turn your voice into a synth: convert it to MIDI and play an instrument, put your words on a synth with a vocoder, shape an instrument with your mouth through a talkbox, let a pitch-tracking voice synth play directly from your voice, or resynthesize it with a neural timbre-transfer model. They differ in where the notes come from and what part of your voice survives.

This guide explains each method, what it's good at and what it needs, so you can hum a bassline, beatbox an 808 or sing a lead line and get a synth part back.

Voice to synth methods at a glance

Voice to MIDI: sing the notes, play any instrument

Voice to MIDI detects the pitch and start of each note you sing and writes or sends MIDI notes. The MIDI can then play any synth, sampler or drum rack, and you can fix a wrong note, quantize the timing or move the part to a different sound.

Some DAWs do it offline. Ableton Live can convert a recorded melody to a new MIDI track, and Logic Pro can create a MIDI track from Flex Pitch data. Vochlea's Dubler 2 does it in real time: it's a standalone app that sends MIDI to your DAW, with chord modes, pitch bend, vowel-controlled CCs and drum triggers you train on your own beatbox sounds.

The trade-off is the note decision. The converter has to decide that a note has started and which note it is before the instrument can play, and every slide or wobble becomes either a pitch bend or a new note. Clear, steady singing converts best.

Vocoder: your words on a synth's notes

A vocoder splits your voice into frequency bands and applies their levels to a synth, the carrier. The synth supplies the pitch and your voice supplies the words, so you get a synth that talks or a choir that sings your lyrics on chords you play. Our what is a vocoder explainer covers the bands and envelopes in detail.

The key point for this guide: a vocoder ignores the notes you sing. Choir, for example, plays the Key and Chord you set or the MIDI chords you send it. That makes vocoders the right tool for harmony and robot voices, and the wrong one if you want to hum a melody and hear it as a synth.

Talkbox: an instrument shaped by your mouth

A talkbox pipes an instrument's sound through a tube into your mouth, and a mic picks up the result. The notes come from the keyboard or guitar you're playing, while your mouth shapes the vowels. It sounds wet, bendy and very vocal on a single line, and it has to be performed live: you can't run a finished vocal through one. The full comparison is in vocoder vs talkbox.

Pitch-tracking voice synth: hum to synth with no MIDI step

A pitch-tracking voice synth listens to your voice continuously and drives a synth from it directly. Throat works this way. It tracks your pitch, loudness, vowels, breath and consonants and plays a monophonic synth from them, with no notes detected in between. Slides, vibrato and the way you lean into a note come through as you performed them.

• Pitch sets the note. In Snap mode it locks to a key and scale; in Free mode it follows every bend. An octave control lets you hum in your comfortable range and hear the synth two octaves lower.
• Vowels play the filter. "Oo" is dark, "ah" opens it, "ee" is brightest, so a wah or an acid sweep is a mouth shape.
• Consonants and beatbox hits punch the filter and level. Hard enough, a hit plays the synth on its own, which turns a beatboxed kick into an 808.
• Breath becomes a noise layer, and a talk control puts your words on the synth like a talkbox.

The demo below runs Throat's Hum Bass preset. Hum a low "mm" into your mic and open it to "ah" to hear the filter follow your mouth.

The trade-offs are the opposite of voice to MIDI. It is monophonic, one note at a time, and there are no notes to edit. If a note is wrong, you sing it again, or record your dry voice and let the plugin re-play it with a different preset.

Neural timbre transfer: resynthesize your voice as another sound

Machine-learning models can take a performance and rebuild it in another timbre. Qosmo's Neutone FX is a plugin that loads downloadable models, including RAVE timbre-transfer models that turn a voice into something like a violin or a choir. Dreamtonics' Vocoflex works in a related direction for singers: it morphs your vocal into a different voice built from recordings, in real time.

These tools can produce sounds nothing else can. The results depend on the model and on how close your input is to what it was trained on, and running one usually takes more CPU than a conventional synth.

Which method should you use?

• You want MIDI you can edit, or to play an instrument you already own: voice to MIDI.
• You want chords, choirs or a robot voice: a vocoder.
• You want a sung bassline, lead or 808 that keeps your phrasing: a pitch-tracking voice synth.
• You want an organic talking lead and you'll learn to perform it: a talkbox.
• You want an unusual new timbre or a different singer: neural timbre transfer.

Tips for a clean voice to synth performance

1. Wear headphones. Speakers leak the synth back into the mic, and the tracker hears it as you.
2. Lower your buffer. 64 or 128 samples while performing keeps the round trip through your interface short. Raise it again for mixing.
3. Sing close and dry. A close mic in a quiet room gives the cleanest pitch and vowel readings.
4. Exaggerate. Big vowel changes and crisp consonants move the synth more than subtle ones.
5. Keep the dry take. Recording your voice unprocessed lets you change the sound later without performing again.