AssemblyAI JavaScript SDK

The AssemblyAI JavaScript SDK provides an easy-to-use interface for interacting with the AssemblyAI API, which supports async and streaming transcription and text-to-speech. It is written primarily for Node.js in TypeScript with all types exported, but also compatible with other runtimes.

Claude Code

This repository includes a CLAUDE.md file that provides context to Claude Code about this SDK — key APIs, common patterns, and gotchas. When you open this repo in Claude Code, it automatically reads this file to give better assistance.

Using with AI coding agents

If you're integrating this SDK with Claude Code, Cursor, Copilot, or another AI coding assistant, give your agent current API context so it doesn't generate code against outdated model names or parameters.

The most effective option is project instructions. Add this to your CLAUDE.md, .cursorrules, AGENTS.md, or equivalent agent instructions file:

Always fetch https://assemblyai.com/docs/llms.txt before writing AssemblyAI code. The API has changed, do not rely on memorized parameter names.

For on-demand documentation lookups during a session, connect the AssemblyAI docs MCP server:

claude mcp add assemblyai-docs --transport http https://mcp.assemblyai.com/docs
Copy

For deep SDK context in Claude Code specifically, install the AssemblyAI skill:

claude install-skill https://github.com/AssemblyAI/assemblyai-skill
Copy

See Coding agent prompts for Cursor setup, MCP tool details, and tips for best results.

Documentation

Visit the AssemblyAI documentation for step-by-step instructions and a lot more details about our AI models and API. Explore the SDK API reference for more details on the SDK types, functions, and classes.

Quickstart

Install the AssemblyAI SDK using your preferred package manager:

npm install assemblyai
Copy

yarn add assemblyai
Copy

pnpm add assemblyai
Copy

bun add assemblyai
Copy

Then, import the assemblyai module and create an AssemblyAI object with your API key:

import { AssemblyAI } from "assemblyai";

const baseUrl = "https://api.assemblyai.com";

const client = new AssemblyAI({
  apiKey: "YOUR_API_KEY",
  baseUrl: baseUrl,
});
Copy

You can now use the client object to interact with the AssemblyAI API.

Using a CDN

You can use automatic CDNs like UNPKG to load the library from a script tag.

• Replace :version with the desired version or latest.
• Remove .min to load the non-minified version.
• Remove .streaming to load the entire SDK. Keep .streaming to load the Streaming STT specific version.

<!-- Unminified full SDK -->
<script src="https://www.unpkg.com/assemblyai@:version/dist/assemblyai.umd.js"></script>
<!-- Minified full SDK -->
<script src="https://www.unpkg.com/assemblyai@:version/dist/assemblyai.umd.min.js"></script>
<!-- Unminified Streaming STT only -->
<script src="https://www.unpkg.com/assemblyai@:version/dist/assemblyai.streaming.umd.js"></script>
<!-- Minified Streaming STT only -->
<script src="https://www.unpkg.com/assemblyai@:version/dist/assemblyai.streaming.umd.min.js"></script>
Copy

The script creates a global assemblyai variable containing all the services. Here's how you create a StreamingTranscriber object.

const { StreamingTranscriber } = assemblyai;
const transcriber = new StreamingTranscriber({
  token: "[GENERATE TEMPORARY AUTH TOKEN IN YOUR API]",
  ...
});
Copy

For type support in your IDE, see Reference types from JavaScript.

Speech-To-Text

Transcribe audio and video files

// Transcribe file at remote URL
const audioFile = "https://assembly.ai/sports_injuries.mp3";

const params = {
  audio: audioFile,
  speech_models: ["universal-3-5-pro", "universal-2"],
  language_detection: true,
};

const run = async () => {
  const transcript = await client.transcripts.transcribe(params);
  console.log(transcript.text);
};

run();
Copy

let transcript = await client.transcripts.submit({
  audio: "https://assembly.ai/espn.m4a",
  speech_models: ["universal-3-5-pro", "universal-2"],
  language_detection: true,
});
Copy

// Upload a file via local path and transcribe
let transcript = await client.transcripts.transcribe({
  audio: "./news.mp4",
  speech_models: ["universal-3-5-pro", "universal-2"],
  language_detection: true,
});
Copy

let transcript = await client.transcripts.submit({
  audio: "./news.mp4",
  speech_models: ["universal-3-5-pro", "universal-2"],
  language_detection: true,
});
Copy

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({
  apiKey: "<YOUR_API_KEY>",
});

const audioFile = "https://assembly.ai/wildfires.mp3";

const params = {
  audio: audioFile,
  speech_models: ["universal-3-5-pro", "universal-2"],
  language_detection: true,
  speaker_labels: true,
};

const run = async () => {
  const transcript = await client.transcripts.transcribe(params);

  for (const utterance of transcript.utterances!) {
    console.log(`Speaker ${utterance.speaker}: ${utterance.text}`);
  }
};

run();
Copy

const transcript = await client.transcripts.get(transcript.id);
Copy

const transcript = await client.transcripts.waitUntilReady(transcript.id, {
  // How frequently the transcript is polled in ms. Defaults to 3000.
  pollingInterval: 1000,
  // How long to wait in ms until the "Polling timeout" error is thrown. Defaults to infinite (-1).
  pollingTimeout: 5000,
});
Copy

const sentences = await client.transcripts.sentences(transcript.id);
const { paragraphs } = await client.transcripts.paragraphs(transcript.id);

for (const paragraph of paragraphs) {
  console.log(paragraph.text);
}

for (const sentence of sentences) {
  console.log(sentence.text);
}
Copy

const charsPerCaption = 32;
let srt = await client.transcripts.subtitles(transcript.id, "srt");
srt = await client.transcripts.subtitles(transcript.id, "srt", charsPerCaption);

let vtt = await client.transcripts.subtitles(transcript.id, "vtt");
vtt = await client.transcripts.subtitles(transcript.id, "vtt", charsPerCaption);
Copy

const page = await client.transcripts.list();
Copy

let previousPageUrl: string | null = null;
do {
  const page = await client.transcripts.list(previousPageUrl);
  previousPageUrl = page.page_details.prev_url;
} while (previousPageUrl !== null);
Copy

const res = await client.transcripts.delete(transcript.id);
Copy

Transcribe audio synchronously

client.sync sends audio over one live connection and returns the finished transcript in one round trip — no job id, no polling. transcribe() sends a clip you already hold as a single chunk over that connection; transcribeLive() / openLive() upload it as it is still being recorded. Use it for short clips where you want the answer inline; use client.transcripts for long-form audio, URLs, or the rich audio-intelligence features the sync API doesn't expose.

const result = await client.sync.transcribe("./call.wav");
console.log(result.text, result.session_id);
Copy

The input can be a local file path, raw audio bytes, a Blob, or a readable stream — but not a URL.

const result = await client.sync.transcribe("./call.wav", {
  prompt: "Transcribe verbatim. Preserve disfluencies.", // max 6000 chars
  keyterms_prompt: ["AssemblyAI", "Lemur"], // max 100 terms, 8000 chars total
  language_codes: ["es"], // or e.g. ["en", "es"] for multilingual; defaults to English
  conversation_context: [
    // prior turns, oldest first
    "I'd like to book a flight to Denver.",
    "Sure, what date were you thinking?",
  ],
});
Copy

const result = await client.sync.transcribe(rawPcmBytes, {
  sample_rate: 16_000,
  channels: 1,
});
Copy

const result = await client.sync.transcribe("./call.wav", {
  timestamps: true,
});
for (const word of result.words) {
  console.log(word.text, word.start, word.end); // milliseconds
}
Copy

await client.sync.warm(); // fire as recording starts
const audio = await recordUntilDone();
const result = await client.sync.transcribe(audio); // reuses the hot connection
Copy

import { spawn } from "node:child_process";
import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });
const sampleRate = 16_000;

// The SoX recorder the samples use (see samples/sync-live-from-mic), capturing
// raw 16-bit mono PCM on stdout.
const recorder = spawn("sox", [
  "--default-device",
  "--no-show-progress",
  "--rate",
  String(sampleRate),
  "--channels",
  "1",
  "--encoding",
  "signed-integer",
  "--bits",
  "16",
  "--type",
  "raw",
  "-",
]);

await client.sync.warm(); // open the connection ahead of time

// The request starts here. Raw PCM carries no header, so the sample rate and
// channel count go in the config.
const session = client.sync.openLive({ sample_rate: sampleRate, channels: 1 });

// write() never blocks, so it is safe to call from a capture callback.
recorder.stdout.on("data", (chunk) => session.write(chunk));
recorder.stdout.on("end", () => session.close()); // ends the audio

process.on("SIGINT", () => recorder.kill()); // stop speaking, stop recording

const result = await session.result(); // closes if needed, then waits
console.log(result.text);
Copy

import { createReadStream } from "node:fs";

// Don't: the audio already exists, so there is nothing to overlap with.
await client.sync.transcribeLive(createReadStream("./call.wav"));

// Do: transcribe() uploads the whole file at once.
await client.sync.transcribe("./call.wav");
Copy

const recorder = spawn("sox", soxArgs); // still recording
const controller = new AbortController();

const result = await client.sync.transcribeLive(
  recorder.stdout, // a Node Readable; a web ReadableStream works too
  { sample_rate: 16_000, channels: 1 },
  { timeout: 180_000, signal: controller.signal },
);
console.log(result.text);
Copy

import { SyncTranscriptError } from "assemblyai";

try {
  const result = await client.sync.transcribe("./call.wav");
} catch (error) {
  if (error instanceof SyncTranscriptError) {
    console.error(error.status, error.errorCode, error.retryAfter);
  }
}
Copy

Dictation

client.dictation transcribes short spoken notes: a person speaks, the audio uploads as it is spoken, and when they stop you get one finished transcript back — optionally cleaned up or reformatted by an LLM. There is no job id and no polling, and the service takes audio rather than URLs. It targets the dictation API host (https://dictation.assemblyai.com, override with the dictationBaseUrl client option).

const result = await client.dictation.transcribeLive(micStream, {
  sample_rate: 16_000,
  channels: 1,
});
console.log(result.final_text, result.session_id);
Copy

result.final_text is the one to show the user: the LLM rewrite when an llm_instruction ran, the raw transcript otherwise. result.text always holds the raw transcript. Audio must be WAV or raw 16-bit PCM, up to 120 seconds per request.

This is not the streaming API. client.streaming returns words while the speaker is still talking; dictation returns one finished transcript when the audio ends.

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });

await client.dictation.warm(); // open the connection ahead of time

// The request starts here. Raw PCM carries no header, so the sample rate and
// channel count go in the config.
const session = client.dictation.openLive({
  sample_rate: 16_000,
  channels: 1,
});

// write() never blocks, so it is safe to call from a capture callback.
mic.on("data", (chunk) => session.write(chunk));
mic.on("end", () => session.close()); // ends the audio

const result = await session.result(); // closes if needed, then waits
console.log(result.final_text);
Copy

const recorder = spawn("sox", soxArgs); // still recording
const controller = new AbortController();

const result = await client.dictation.transcribeLive(
  recorder.stdout, // a Node Readable; a web ReadableStream works too
  { sample_rate: 16_000, channels: 1 },
  { timeout: 300_000, signal: controller.signal },
);
console.log(result.final_text);
Copy

const result = await client.dictation.transcribeLive("./note.wav");
console.log(result.final_text);
Copy

const result = await client.dictation.transcribeLive(micStream, {
  llm_instruction: "Format this as a SOAP note.", // max 2048 chars
});

console.log(result.text); // what the speaker said
console.log(result.llm_response); // the rewrite, or null if no instruction ran
console.log(result.final_text); // llm_response when there is one, text otherwise
Copy

const result = await client.dictation.transcribeLive(micStream, {
  llm_instruction: "Lowercase unless a name, no final punctuation.",
  rewrite_mode: "smart",
});
Copy

const result = await client.dictation.transcribeLive(micStream, {
  sample_rate: 16_000, // raw 16-bit PCM only; required together with channels
  channels: 1, // 1 mono, 2 stereo; leave both unset for WAV
  language_codes: ["es"], // ISO 639-1; or e.g. ["en", "es"]; defaults to the server's choice
  stt_prompt: "A doctor dictating a patient visit note.", // max 6000 chars
  keyterms_prompt: ["AssemblyAI", "Universal-3"], // max 100 terms, 8000 chars total
  llm_instruction: "Format this as a SOAP note.", // max 2048 chars
  rewrite_mode: "smart", // rewrite model tier: "fast" (default) or "smart"
});
Copy

const ready = await client.dictation.warm(); // true once the socket is open
Copy

import { DictationError } from "assemblyai";

try {
  const result = await client.dictation.transcribeLive(micStream);
} catch (error) {
  if (error instanceof DictationError) {
    console.error(error.status, error.errorCode, error.retryAfter);
  }
}
Copy

Transcribe streaming audio

Refer to AssemblyAI's streaming documentation for full code examples.

Create the streaming transcriber.

const transcriber = client.streaming.transcriber({
  speechModel: "universal-3-5-pro",
  sampleRate: 16_000,
});
Copy

[!WARNING] Storing your API key in client-facing applications exposes your API key. Generate a temporary auth token on the server and pass it to your client. Server code:

const token = await client.streaming.createTemporaryToken({
  expires_in_seconds = 60,
});
// TODO: return token to client
Copy

Client code:

import { StreamingTranscriber } from "assemblyai";
// TODO: implement getToken to retrieve token from server
const token = await getToken();
const transcriber = new StreamingTranscriber({
  token,
});
Copy

You can configure the following events.

transcriber.on("open", ({ id, expires_at }) => console.log('Session ID:', id, 'Expires at:', expires_at));
transcriber.on("close", (code: number, reason: string) => console.log('Closed', code, reason));
transcriber.on("turn", ({ transcript }) => console.log('Transcript:', transcript));
transcriber.on("error", (error: Error) => console.error('Error', error));
Copy

After configuring your events, connect to the server.

await transcriber.connect();
Copy

Send audio data via chunks.

// Pseudo code for getting audio
getAudio((chunk) => {
  transcriber.sendAudio(chunk);
});
Copy

Or send audio data via a stream:

audioStream.pipeTo(transcriber.stream());
Copy

Close the connection when you're finished.

await transcriber.close();
Copy

Text-To-Speech

client.tts speaks text with the Text-to-Speech API. synthesize() returns one text's complete audio, stream() returns it as the API generates it, and session() opens a WebSocket session that speaks text as it is written, for voice agents. Requests go to ttsBaseUrl, https://tts.us.assemblyai.com by default; pass https://tts.eu.assemblyai.com to keep them in the EU data zone.

An HTTP request takes the API's own field names. A session takes the same options in camelCase, like the other session clients of this SDK.

Synthesize a WAV file

import { writeFileSync } from "fs";

const response = await client.tts.synthesize({
  text: "Hello from AssemblyAI.",
  voice_id: "aai-jane",
  container: "wav",
});
writeFileSync("hello.wav", response.audio);
console.log(response.model, response.voiceId, response.audioDurationMs, "ms");
Copy

synthesize() waits for the whole audio, so an API refusal is always a TextToSpeechError and the WAV header carries real sizes. The response headers are on the result: requestId, region, model, voiceId, language, encoding, sampleRate, container, bitRate, billedCharacters, audioDurationMs and unknownParams. The API ignores body fields it does not know, and unknownParams names them (the SDK also warns once), so a camelCase sampleRate in a request held in a variable shows up instead of silently giving 24 kHz audio. A network failure or a timeout rejects with the runtime's own error instead (a TypeError, or a DOMException named TimeoutError or AbortError). On Node, the built-in fetch gives up after 300 seconds without response headers, whatever timeout says; for longer requests use stream(), or raise the limit with setGlobalDispatcher(new Agent({ headersTimeout: 610_000 })) from the undici package.

Branch on a refusal's errorCode, not its message, and on retryable to decide whether to try again (retryAfter says how many seconds to wait). The same codes come back from a session, where the error's closeCode is the WebSocket close code that ended it.

The session rows come from the Error frame the API sends before it closes. A connection that closes without one, such as a network cut, gives an error with closeCode only, retryable for 1006, 1011, 1012, 3005, 3008 and 3009.

Stream audio as it is generated

const speech = await client.tts.stream({
  text: "Streaming straight to the speakers.",
  voice_id: "aai-alba",
});
console.log(
  "speaking with",
  speech.info.model,
  "at",
  speech.info.sampleRate,
  "Hz",
);
for await (const chunk of speech) {
  play(chunk); // raw 24 kHz 16-bit mono PCM, as a Uint8Array
}
Copy

stream() resolves once the API has sent the first audio. Chunk sizes vary and are not aligned to samples or frames: for 16-bit PCM carry an odd trailing byte over to the next chunk, and re-frame for a fixed-size sink (20 ms of 8 kHz mu-law is 160 bytes). The API commits to 200 with the first audio, so a failure after that cuts the body: the loop then throws a TextToSpeechError with retryable: true, and the whole request should be retried. The headers leave with the first audio too, so speech.info.billedCharacters and speech.info.voiceExpiresAt are the values as the stream started (a body cut by a server failure bills 0), and speech.info.audioDurationMs is never set, since the duration is known only once the audio is complete; measure the bytes received instead. A body that sends no audio for 30 seconds while the loop waits for it ends the same way, and its connection is released, so a stalled connection (a dropped NAT mapping, a hung proxy) cannot hang the loop. Set the limit in milliseconds with the readTimeout option (0 turns it off); only the wait counts, so playing a chunk at its own pace never trips it. speech.body is the underlying web ReadableStream, and a read of it waits under the same limit. So stream() needs a global ReadableStream: on a runtime without one (React Native without a polyfill) it throws a TextToSpeechError before sending the request; use synthesize() there.

For a barge-in, pass { signal } as the second argument and abort it: the download stops and the loop throws the signal's reason (an AbortError), not a TextToSpeechError. speech.cancel() stops it too, and the loop just ends. One signal can serve every request of a call. Cancel a stream you will not read, such as a reply requested ahead that a barge-in made obsolete: a stream dropped unread keeps its connection until the runtime garbage-collects it.

Speak text as it is written (voice agents)

A TextToSpeechSession keeps one WebSocket open and speaks text as it is written, so an LLM's reply can be voiced sentence by sentence with no per-request handshake. speak() sends one text and returns its audio; generate() / flush() do the same in two steps, so tokens can be sent as they arrive; cancel() is the barge-in. close() ends the session and resolves with its summary once the API has spoken the text still pending. It stops waiting after terminateTimeout milliseconds (default 35,000) without a message from the API; an audio() iteration still running then rejects with a retryable TextToSpeechError, and close() resolves undefined. Calling close() again, or from two places at once, waits for the same end. Once close() has sent Terminate the API reads nothing more, so generate(), flush(), cancel() and speak() throw instead of sending text that would never be spoken, and keepAlive() does nothing; once the session has ended they say so, and that connect() opens a new one. Text need not wait for connect() to resolve: what is sent while the socket opens is held and sent the moment it opens (see Keep the time to first audio low).

const session = client.tts.session({ voiceId: "aai-jane" });
await session.connect();

for (const sentence of llmReplySentences()) {
  for await (const chunk of session.speak(sentence)) play(chunk);
}

const summary = await session.close();
console.log(summary?.audio_duration_seconds, "s spoken");
Copy

Streaming the text in as it is generated, with a barge-in. The API speaks a complete sentence before flush() once more text has arrived after it and the request is long enough, about 200 characters, or 50 for the first sentence of text sent in several generate() calls; a shorter request, or one whose last sentence has nothing after it yet, is spoken at flush(). So read session.audio() with no id while the tokens are still being sent:

const session = client.tts.session({ voiceId: "aai-jane" });
await session.connect();

let interrupted = false;
const player = (async () => {
  for await (const chunk of session.audio()) {
    if (userStartedTalking()) {
      interrupted = true;
      session.cancel(); // drops the text sent so far; text sent after it is a new request
      break;
    }
    play(chunk);
  }
})();
for await (const token of llmTokens()) {
  if (interrupted) break; // stops the LLM stream too
  session.generate(token); // joined as sent: include the spaces
}
if (!interrupted) session.flush();
await player;
await session.close();
Copy

The API reads frames in the order sent, so a cancel() retires only the text sent before it. Stop sending the interrupted reply once the user barges in: text sent after the cancel is a new request, which is synthesized and billed, and which close() waits for.

audio() fixes its request when it is called, so the iterator can be made before the first generate() and handed to a player that starts later. Requests can be pipelined: flush() again before reading the previous request's audio, then read each audio(flushId) in order, once; audio() throws a RangeError for an id not flushed yet, or one an iteration already read past, since reading a later request consumes the end of the earlier ones. Every flush() and cancel() takes the next id and returns it, so ids skip one after a cancel, and the requests a cancel retires are those up to its id. Audio of a retired request that the API sent before it read the Cancel can still arrive; the session drops it, so neither audio() nor the audio listener keeps playing the interrupted reply. A Generate frame takes at most 2,000 characters (speak() splits longer text; generate() throws); the API accepts text at about 100 characters per second after an initial 1,500 and holds at most 8,000 unspoken characters (limits.text_max_buffer_chars in Begin), ending the session with a non-retryable 3010 past that, so pace text to the audio. speak() throws a RangeError for a longer text before sending any of it; split it and speak each part once the previous part's audio is read. A session streams raw pcm_s16le, pcm_mulaw or pcm_alaw only, ends after an hour, and closes with a retryable 3006 inactivity_timeout after inactivityTimeout seconds (5 to 120, default 120) without a frame in either direction. Audio arrives much faster than it plays, so the session can idle out while received audio is still playing: call keepAlive() at least once per timeout while the session waits.

Register listeners with session.on(event, listener): begin, flushDone, wordBoundaries (word timings per request, with wordBoundaries: true; start/end count from the start of the session's audio, so subtract the event's audio_start_ms for a time within the request), cancelled, termination, error and close. An audio listener receives the audio frames instead of the pull iterators, which then throw. A listener that throws is logged with console.error and the session goes on.

A failure is a TextToSpeechError. A refusal rejects connect(); a failure mid-session rejects the audio() iteration or the next send (a send in the moment between a lost connection and its close event throws a retryable error saying the connection is closing), reaches the error listener, and, if nothing else observed it, rejects close(); a frame the SDK cannot read fails the session with Unreadable frame from the API and closes the socket. On a session read errorCode and closeCode, the WebSocket close code the API ended it with (see the table above). requestId is the session id from Begin, so a session refused before Begin, such as one with an unknown voice or a bad key, has none. status is set only when the server answered the WebSocket upgrade itself with an HTTP status the runtime reports (Node does, a browser does not): a 404 for a wrong websocketBaseUrl path is not retryable, a 503 is.

retryable: true means a new session can succeed, and the same session object can connect() again, also from the error or close listener, which run once the connection has ended; an audio() iteration of the old connection still rejects. That works as is with an API key; a session opened with a temporary token needs a new token for each connection, passed as connect({ token }), because the API may refuse a token it has seen and always refuses an expired one (close code 1008, errorCode: "unauthorized"). The close listener gets the socket's close code, or the error's close code (1000 without one) when the session released the socket first.

In a browser the API key must stay on your server. Mint a temporary token there, and open the session in the page with new TextToSpeechSession({ token }), which needs no client and no API key:

// Server
const token = await client.tts.createTemporaryToken({ expires_in_seconds: 60 });
Copy

// Page
import { TextToSpeechSession } from "assemblyai";

const session = new TextToSpeechSession({
  token: await getToken(), // fetches a new token from your server
  voiceId: "aai-jane",
  // websocketBaseUrl: "wss://tts.eu.assemblyai.com/v1/ws", // EU data zone
});
await session.connect();
// After a retryable close: await session.connect({ token: await getToken() });
Copy

Mint a new token for every connection, reconnects included, and for every HTTP request (a token works as the apiKey of a client made for that request), and keep expires_in_seconds short. createTemporaryToken() mints on the token host of the client's zone: https://token.eu.assemblyai.com when ttsBaseUrl is https://tts.eu.assemblyai.com, https://token.us.assemblyai.com for https://tts.us.assemblyai.com (the default), and https://token.assemblyai.com for any other host; pass { tokenBaseUrl } as its second argument to choose the host. A token works in every zone. A refusal throws a TextToSpeechError whose message gives the token service's reason, such as Invalid API key or expires_in_seconds: Input should be less than or equal to 600.

Keep the time to first audio low

The wait for first audio is mostly synthesis on our servers, plus whatever your client does before the text reaches them. These habits take the client's part away from the moment a user is waiting:

• Connect the session before the reply is ready, when the call starts or the page loads, and keep it for the whole conversation instead of opening one per turn. A turn on an open session skips the TCP, TLS and WebSocket handshakes and the API's checks before Begin: in the API's region that saved about 25 ms per turn, and about 75 ms on the first turn with a voice the API had not used recently, because the API loads the voice when the session opens. Farther away, each new session also costs three network round trips. Between turns call keepAlive() at least once per inactivityTimeout, and before session.begin.expires_at (at most an hour after Begin) open the next session in a pause and move the following turns to it.

• When a session must be opened for a reply, send the text without waiting for Begin. Text sent while connect() is still pending is held and sent the moment the socket opens, and the API reads it as soon as it has sent Begin, so the text travels while the API checks the session instead of after:

  const connecting = session.connect();
  const audio = session.speak(reply); // sent as soon as the socket opens
  await connecting; // a refused session rejects here
  for await (const chunk of audio) play(chunk);
  Copy

  This saves up to one network round trip, and never more than the API's checks take. With 60 ms of round trip added between our test client and the API's region, first audio came 14 ms sooner (median of 30 sessions per arm); inside the region the difference was too small to measure. Keep and await connect()'s promise all the same: a refused session rejects it, and a rejection nothing handles ends a Node process.

• In a browser, mint the next token before the page needs it, for example when the pointer enters the text box after the idle session closed, so that opening the session does not wait for your server and the token service (about 24 ms per new session in our tests, before the round trip to your server). Mint it only when a session will have to be opened, use it for one connection, and drop it well before it expires.

• On Node, keep HTTP connections between replies if your voice agent calls stream() or synthesize() more than 4 seconds apart. Node's built-in fetch closes an idle connection after about 4 seconds when the server sends no Keep-Alive hint, and the API sends none, so a later reply opens a new TCP and TLS connection. Keep connections for 2 minutes with setGlobalDispatcher(new Agent({ keepAliveTimeout: 120_000 })) from the undici package, which applies to every fetch of the process. With requests 6 seconds apart, that brought the first audio 5 to 10 ms sooner in the API's region (median of 30 requests per arm, Node 20 and 22) and 125 ms sooner with 60 ms of round trip added, since each new connection costs two network round trips. An idle kept connection does not keep the process from exiting. Bun and Deno already keep idle connections for over a minute.

Node needs no shared TLS context for sessions: after its first TLS connection it keeps the parsed root certificates, so later connections set up their TLS context in about a tenth of a millisecond.

Contributing

If you want to contribute to the JavaScript SDK, follow the guidelines in CONTRIBUTING.md.