Parakeet on sherpa-onnx, in Node.js

What sherpa-onnx is

sherpa-onnx is part of the Next-gen Kaldi family. Its README says it runs speech recognition, text to speech, voice activity detection, speaker diarization and keyword spotting locally, on x86, ARM, RISC-V and several NPUs, and on Linux, macOS, Windows, Android, iOS and HarmonyOS. It lists twelve language bindings, from C++ to Pascal, plus WebAssembly.

For Node the package is sherpa-onnx-node, a node-addon-api wrapper that supports multiple threads and asks for Node 16 or later. The native library arrives as a platform package, and on macOS and Linux the examples README says to point the dynamic loader at that folder first. In our test on macOS with version 1.13.6 the addon also loaded without it, so treat the export as the fix when loading fails.

npm install sherpa-onnx-node

# macOS arm64, from the examples README
export DYLD_LIBRARY_PATH=$PWD/node_modules/sherpa-onnx-darwin-arm64:$DYLD_LIBRARY_PATH

# Linux x64
export LD_LIBRARY_PATH=$PWD/node_modules/sherpa-onnx-linux-x64:$LD_LIBRARY_PATH

Get the Parakeet files

You do not convert anything yourself. The sherpa-onnx project publishes the converted model in its asr-models release as sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8: a 487 MB download that unpacks to about 670 MB of encoder.int8.onnx, decoder.int8.onnx, joiner.int8.onnx, tokens.txt and test_wavs. Silero VAD is a separate 644 KB file in the same release.

NVIDIA's model card describes parakeet-tdt-0.6b-v3 as a 600 million parameter FastConformer encoder with a TDT decoder, released on Hugging Face on 14 August 2025 under CC BY 4.0. It takes 16 kHz mono audio, covers 25 European languages and detects the language itself.

wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8.tar.bz2
tar xvf sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8.tar.bz2

wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx

Transcribe one WAV file

This is the smallest program that works. modelType has to be nemo_transducer and featureDim is 80. acceptWaveform takes float samples between -1 and 1 with their sample rate, decode runs the model, getResult returns the text. sherpa-onnx resamples for you if the file is not 16 kHz, but it wants one channel and 16-bit samples.

Both slow calls have async twins, OfflineRecognizer.createAsync(config) and recognizer.decodeAsync(stream). Use those in anything with a user interface: loading a 650 MB encoder on the main thread will freeze it.

const sherpa_onnx = require('sherpa-onnx-node');

const dir = './sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8';

const recognizer = new sherpa_onnx.OfflineRecognizer({
  featConfig: { sampleRate: 16000, featureDim: 80 },
  modelConfig: {
    transducer: {
      encoder: dir + '/encoder.int8.onnx',
      decoder: dir + '/decoder.int8.onnx',
      joiner: dir + '/joiner.int8.onnx',
    },
    tokens: dir + '/tokens.txt',
    modelType: 'nemo_transducer',
    numThreads: 4,
    provider: 'cpu',
    debug: 0,
  },
});

const wave = sherpa_onnx.readWave(dir + '/test_wavs/en.wav');
const stream = recognizer.createStream();
stream.acceptWaveform({ sampleRate: wave.sampleRate, samples: wave.samples });
recognizer.decode(stream);
console.log(recognizer.getResult(stream).text);

Live audio: put Silero VAD in front

For live captions you cannot wait for a file to end. The sherpa-onnx examples push audio into the VAD in windows of 512 samples, and each time a segment finishes they pull it out with front(), drop it with pop() and decode it. minSilenceDuration sets how long a pause has to be before a segment closes, so it is the dial that trades latency against cut-off words.

The file examples slice a finished WAV, so every window comes out even. A real capture callback does not hand you exact multiples of 512 samples. The official microphone examples park the audio in sherpa-onnx's CircularBuffer; the code below keeps the leftover in a plain array and prepends it to the next chunk, which is what Fraze does. If your capture device is not at 16 kHz, run it through LinearResampler first.

const vad = new sherpa_onnx.Vad({
  sileroVad: {
    model: './silero_vad.onnx',
    threshold: 0.5,
    minSpeechDuration: 0.25,
    minSilenceDuration: 0.5,
    windowSize: 512,
  },
  sampleRate: 16000,
  debug: false,
  numThreads: 1,
}, 60); // seconds of audio the internal buffer holds

const windowSize = vad.config.sileroVad.windowSize;
let pending = new Float32Array(0);

// chunk: Float32Array of 16 kHz mono audio from your capture callback
function feed(chunk) {
  const merged = new Float32Array(pending.length + chunk.length);
  merged.set(pending);
  merged.set(chunk, pending.length);

  let offset = 0;
  while (merged.length - offset >= windowSize) {
    vad.acceptWaveform(merged.subarray(offset, offset + windowSize));
    offset += windowSize;
    drain();
  }
  pending = merged.slice(offset);
}

function drain() {
  while (!vad.isEmpty()) {
    // front(false) turns off N-API external buffers. See the Electron note below.
    const segment = vad.front(false);
    vad.pop();

    const stream = recognizer.createStream();
    stream.acceptWaveform({ sampleRate: 16000, samples: segment.samples });
    recognizer.decode(stream);

    const text = recognizer.getResult(stream).text.trim();
    if (text) console.log(text);
  }
}

// at the end of the session
vad.flush();
drain();

Custom vocabulary, and what bit us

Names and product words come out wrong until you bias the decoder. sherpa-onnx calls this hotwords, and the docs set two conditions: only transducer models support it, and decodingMethod has to change from the default greedy_search to modified_beam_search. Set modelingUnit to bpe and point bpeVocab at a vocabulary file, and you can pass plain words instead of token sequences. A forward slash separates phrases, a colon sets a per-phrase score. The default score is 1.5 and the official Node example uses 2.0. Passing hotwords to createStream needs sherpa-onnx-node 1.13.4 or later.

Two things cost us time. The v3 archive ships no vocabulary file. The official example, written for Parakeet v2, builds one from tokens.txt with equal scores, and the code below does the same for v3. Fraze instead downloads NVIDIA's own tokenizer.vocab from a pinned revision of the model repo, because the file is no longer on the main branch. And the score matters. At 2.0 our biased words came through and nothing else moved. Above about 6 the decoder began swapping correct words for near matches of the hotwords, which is worse than no biasing at all. In one test clip a heavily boosted "county" replaced every "country".

Beam search is not free either. On our English test clips it ran 10 to 15 percent slower per decode, so Fraze turns it on only when someone has entered vocabulary.

const fs = require('fs');

// Build a BPE vocabulary from tokens.txt, as the official hotwords example does.
const bpeVocab = dir + '/bpe.vocab';
if (!fs.existsSync(bpeVocab)) {
  const lines = fs.readFileSync(dir + '/tokens.txt', 'utf8').split('\n');
  const vocab = lines
    .filter((line) => line.trim() !== '')
    .map((line) => line.split(' ')[0] + '\t-1.0');
  fs.writeFileSync(bpeVocab, vocab.join('\n') + '\n');
}

// This replaces the greedy recognizer from the first example.
const recognizer = new sherpa_onnx.OfflineRecognizer({
  featConfig: { sampleRate: 16000, featureDim: 80 },
  modelConfig: {
    transducer: {
      encoder: dir + '/encoder.int8.onnx',
      decoder: dir + '/decoder.int8.onnx',
      joiner: dir + '/joiner.int8.onnx',
    },
    tokens: dir + '/tokens.txt',
    modelType: 'nemo_transducer',
    modelingUnit: 'bpe',
    bpeVocab: bpeVocab,
    numThreads: 4,
    provider: 'cpu',
  },
  decodingMethod: 'modified_beam_search',
  maxActivePaths: 4,
});

// Hotwords are per stream. '/' separates phrases, ':' sets a score.
const stream = recognizer.createStream('Parakeet :2.0/sherpa-onnx :2.0');

Running it inside Electron

Electron 21 and later do not allow N-API external buffers. The sherpa-onnx calls that hand back audio from native memory take an enableExternalBuffer argument that defaults to true, so in Electron you pass false and get a copy: vad.front(false), readWave(file, false), and the same on CircularBuffer.get(). Skip it and the call throws the first time a segment closes. If something upstream swallows that error, as our audio forwarder did, captions just stop with nothing in the log. Keep all of this in the main process; a sandboxed renderer has no Node.js to load the addon with.

Speed was the pleasant surprise. On an M4 Pro Mac mini, Parakeet v3 through sherpa-onnx ran at 0.02 to 0.03 times real time in our own runs with four threads, so 40 to 50 times faster than the audio. That headroom is what lets Fraze decode a partial every second and a half while someone is still talking, then decode the finished segment when they pause.

Fraze ships exactly this stack, plus Mozilla's Bergamot models for translation and Apple's SpeechAnalyzer as an option on macOS 26. The first 30 minutes are free, then $29.99 once or $2.99 a month, both unlimited.

Questions