Building a voice AI agent usually means assembling at least three separate APIs: a telephony provider, a speech-to-text engine, and a text-to-speech service — plus an LLM to connect them. Each one requires its own SDK, authentication setup, error handling, and retry logic. Most developers spend more time on plumbing than on the actual conversation logic.

n8n removes most of that friction. You can wire Twilio, OpenAI Whisper, an LLM (Claude Sonnet or GPT-4o), and ElevenLabs into a single visual workflow using prebuilt nodes and HTTP Request nodes. The result is a voice AI agent that handles inbound calls, transcribes speech, reasons over the transcript, and responds with natural-sounding audio — without writing a custom server or managing a real-time streaming framework.

This guide walks through the full workflow, step by step.

How the Workflow Works

The pipeline has four stages:

1. Receive — a call comes in via Twilio, which records the caller's speech and sends the audio to an n8n webhook
2. Transcribe — n8n forwards the audio to OpenAI Whisper, which returns a clean text transcript
3. Reason — the transcript goes to an LLM node, which generates a reply based on a system prompt you define
4. Respond — the reply text goes to ElevenLabs, which returns an audio file Twilio plays back to the caller

This round-trip typically completes in two to four seconds depending on audio length and model response time.

What You Need

Before building:

• n8n instance (self-hosted or n8n Cloud)
• Twilio account with an active phone number
• OpenAI API key (for the Whisper transcription endpoint)
• ElevenLabs API key with at least one voice configured
• Credentials for your LLM: OpenAI (GPT-4o) or Anthropic (Claude Sonnet) both work well here

Example n8n Workflow

Step 1 — Webhook Trigger

Add a Webhook node as the workflow trigger. Set the HTTP method to POST. Twilio will call this URL each time a call comes in and recording becomes available. Copy the production webhook URL from n8n and paste it into your Twilio phone number's "A call comes in" webhook setting.

Twilio's webhook payload includes the caller ID, call SID, and — after the caller speaks and hangs up (or after a recording stop event) — a RecordingUrl field pointing to the audio file.

Step 2 — Fetch the Call Recording

Add an HTTP Request node after the webhook. Configure it as a GET request to {{ $json.body.RecordingUrl }}.mp3. This downloads the audio as binary data. Set the response format to "File" so n8n holds it as binary output rather than trying to parse it as JSON.

Step 3 — Transcribe with Whisper

Add another HTTP Request node pointing to https://api.openai.com/v1/audio/transcriptions. Set the method to POST and add your OpenAI API key as a Bearer token in the Authorization header. Attach the binary audio from Step 2 as a multipart form field named file. Include a model field set to whisper-1.

Whisper returns a JSON object with a text field containing the full transcript. Add a Set node to extract this into a clean variable — call it callerText — so downstream nodes reference it by name rather than by deeply nested path.

Step 4 — Generate a Reply with an LLM

Add an OpenAI node or an HTTP Request node targeting the Anthropic Messages API. Pass callerText as the user message content. Define a system prompt that scopes your agent's role, for example:

"You are a scheduling assistant for Acme Health. Answer the caller's question concisely in one or two sentences. If you cannot answer, say you will transfer them to a human agent."

The node returns the LLM's text response. Use another Set node to store it as replyText.

Step 5 — Synthesize Speech with ElevenLabs

Add an HTTP Request node targeting the ElevenLabs TTS API: https://api.elevenlabs.io/v1/text-to-speech/{voice_id}. Use POST, pass your ElevenLabs API key in the xi-api-key header, and send a JSON body with text set to {{ $json.replyText }}. Include a model_id field — use eleven_turbo_v2 for the lowest latency or eleven_multilingual_v2 if you need language coverage beyond English.

Set the response format to "File" so n8n receives the synthesized audio as binary output.

Step 6 — Return Audio to Twilio

Add a Respond to Webhook node at the end of the main branch. Set the response type to binary and pass the ElevenLabs audio bytes. Twilio plays this audio back to the caller.

If you prefer to use Twilio's <Play> TwiML verb rather than direct binary response, add an HTTP Request node before the webhook response to upload the audio to Cloudflare R2 or Amazon S3 (PUT request with binary body), retrieve the signed URL, and return a TwiML response pointing at that URL instead.

Step 7 — Log the Conversation

Add a Google Sheets node or Supabase node on a parallel branch (connected from the same webhook output). Write a row for each call with: timestamp, caller phone number, transcript, LLM reply, and Twilio call SID. This log lets you review conversations, catch edge cases, and refine your system prompt over time.

Adding Conversational Memory

By default, each call is stateless. The LLM sees no context from previous turns in the same conversation. For multi-turn calls:

• Session store with Supabase: at the start of each turn, query a table keyed by CallSid to retrieve the conversation history array. Append the new transcript and reply, then write the updated array back after the LLM responds.
• OpenAI Assistants API with threads: create a thread at the start of the call using an HTTP Request node, pass the thread_id through workflow variables, and use the Assistants run endpoint for each turn. Thread context is managed server-side.

Fallback Routing

Add an IF node after Step 4. Check whether the LLM reply contains a handoff signal — for example, the substring "transfer" or a structured flag you ask the model to return. If the condition is true, route to a second path: use a Slack node to alert your team with the caller ID, transcript, and call SID, and return a brief audio message via ElevenLabs telling the caller a human will call back shortly.

Latency and Quality

• Use whisper-1 for the best transcription speed. For noisy recordings, trim leading silence from the audio before uploading — an Execute Command node running ffmpeg -ss 0.5 -i input.mp3 output.mp3 handles this on self-hosted instances.
• Use eleven_turbo_v2 or eleven_flash_v2 for TTS. Both generate short audio in under 400ms.
• Keep system prompts concise and enforce a max reply length (e.g., "respond in at most two sentences"). Shorter outputs reduce LLM generation time and TTS processing time.
• If latency is critical, look at Twilio's Media Streams feature. With an n8n WebSocket trigger, you can stream audio in real time rather than waiting for a full recording — this cuts perceived latency by 0.5 to 1 second.

What You End Up With

Seven nodes and a few hours of setup produce a fully working voice AI agent: inbound call handling, Whisper transcription, LLM reasoning, ElevenLabs speech synthesis, conversation logging, and fallback escalation. The same pattern adapts to outbound dialing, appointment reminders, intake surveys, and order-status lookups — any scenario where a caller needs a fast, consistent, and intelligent response.

Browse voice AI workflow templates and the full AI agents use case library at n8nresources.dev. Ready-to-import community templates are available at n8nresources.dev/templates.