A voice assistant that talks about the news has two hard problems hiding inside it. One is the conversation itself: hearing speech, turning it into words, thinking, and speaking back fast enough that it feels like talking to a person. The other is having anything worth saying. A model on its own can do the first part beautifully and still be wrong about what happened this morning, because it is answering from training data that froze months ago.
This post walks through a small demo that solves both. The conversation runs on OpenAI’s realtime model. The knowledge comes from Seltz, a web knowledge API that returns clean, source-backed documents. What I want to show is how little code sits between those two pieces, and why the Seltz call is the one that turns a smooth-talking model into a useful one.
01.What the demo does
The demo is a single page. Pressing the spacebar opens the microphone, and asking a question like “what’s the latest with Nvidia?” gets a spoken reply a moment later, two or three sentences long. The reply is grounded in articles published recently rather than in whatever the model happened to remember, and a transcript builds up on screen as the voice talks.
Underneath, there are exactly two server endpoints, and they are both small. One mints a short-lived token so the browser can open a voice session. The other runs a Seltz search. Everything else, including the entire audio conversation, happens directly between the browser and OpenAI. That division of labor is the whole design, so it is worth making it explicit before looking at any code.
02.The architecture: who does what
The instinct with a voice app is to route everything through your own server: audio in, audio out, your backend in the middle. This demo deliberately does not do that. The browser opens a direct WebRTC connection to OpenAI and streams microphone audio straight there. The spoken reply comes back over the same connection. The server never touches the audio at all.
OpenAI’s realtime model (gpt-realtime-2) handles the full conversational loop over that one connection: listening for speech, transcribing it, reasoning about what was asked, and speaking the answer back. We are not going to re-create OpenAI’s own material on how that works. If you want the voice plumbing in depth, the WebRTC handshake and session events and so on, read OpenAI’s voice agents guide. What this post is about is the gap that guide leaves open: where does the model get something true to say, and how does that fit in.
So the server has just two jobs:
- Mint a short-lived session token, so the browser can open the voice connection without ever holding a long-lived key.
- Run a Seltz search when the model asks for one, and hand the results back. This is the job that makes the assistant worth using.
Notice what is missing from that list. No audio handling, no transcription, no streaming of speech through the server. The expensive, latency-sensitive media path is between the browser and OpenAI; the server only steps in for the two moments that need a secret.
03.Minting a short-lived token
Before the browser can open a voice session, it needs permission. It cannot use your real OpenAI key for that, because anything the browser holds is readable by anyone using the page. So the first endpoint exchanges your server-side key for a short-lived session secret and sends only that to the browser.
This is also the moment where the assistant’s personality and its single tool get defined. The session is created with a system prompt and a declaration that a search_news tool exists. The model is allowed to ask to call that tool. It is never given the ability to run it.
// app/api/realtime/token/route.ts (server, never reaches the browser)
export async function POST() {
const res = await fetch("https://api.openai.com/v1/realtime/client_secrets", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
session: {
type: "realtime",
model: "gpt-realtime-2",
instructions: SYSTEM_PROMPT,
// The model is told a "search_news" tool exists. It can ask to call
// it, but it cannot run it. That happens on our side.
tools: [
{
type: "function",
name: "search_news",
description: "Search for recent news articles on a topic.",
parameters: {
type: "object",
properties: {
query: { type: "string" },
// Time-bounded questions ("past week") become real date
// filters instead of time words in the query.
from_date: { type: "string" },
to_date: { type: "string" },
},
required: ["query"],
},
},
],
audio: { output: { voice: "marin" } },
},
}),
});
const data = await res.json();
// The browser gets back a short-lived secret, not our real key.
return Response.json({ value: data.value });
}The browser takes the value that comes back and uses it to open its WebRTC connection to OpenAI. That secret is scoped to a single session and expires quickly, so even if someone fished it out of network traffic, it would be close to useless and would not expose the key that minted it.
04.The Seltz search
Here is the endpoint that does the actual work, and the reason any of this is interesting. When the model decides it needs to know something, this is what answers. It is a few lines, because Seltz does the hard part.
// app/api/news/search/route.ts (server)
export async function GET(request: NextRequest) {
const q = request.nextUrl.searchParams.get("q")?.trim();
if (!q) return Response.json({ error: "Missing q param" }, { status: 400 });
const res = await fetch("https://api.seltz.ai/v1/search", {
method: "POST",
headers: {
"content-type": "application/json",
"x-api-key": process.env.SELTZ_API_KEY,
},
body: JSON.stringify({
query: q,
max_results: 5,
scope: "news",
// Passed through when the model set them ("past week" → from_date)
from_date: request.nextUrl.searchParams.get("from") ?? undefined,
to_date: request.nextUrl.searchParams.get("to") ?? undefined,
}),
});
const { documents } = await res.json();
// documents is an array of clean docs, each with a url, content and
// published_date. We reshape them into something small and predictable.
const articles = (documents ?? []).map((doc) => ({
title: extractTitle(doc.content ?? "", doc.url ?? ""),
url: doc.url ?? "",
source: hostnameFromUrl(doc.url ?? ""),
content: extractSnippet(doc.content ?? ""),
published_date: doc.published_date ?? "",
}));
return Response.json(articles);
}The call that matters is one POST to api.seltz.ai/v1/searchwith your server-side key. That asks Seltz for up to five recent news documents about the query — and when the user asked a time-bounded question (“what happened in the past week?”), the model's from_date/to_date arguments ride along as real filters, so recency is enforced by the search engine instead of hoped for via words in the query. What comes back is documents, an array where each entry has a url, a block of clean content and a published_date. This is the bit worth pausing on: it is not a pile of raw HTML for you to scrape and untangle. It is readable content with the source attached, which is exactly the shape a language model can reason over.
The route then trims each document down to a small, predictable object, { title, url, source, content, published_date }, pulling a title out of the content, deriving the source from the hostname, and cutting the body to a short snippet. The model does not need the whole article to summarize it; it needs enough to be accurate and a link to stand behind. Keeping the payload small also keeps the spoken reply fast. For the full set of parameters the search call accepts, the Seltz docs cover the search API.
That is the entire integration. There is no vector database to maintain, no crawler, no freshness pipeline. Seltz takes a question and hands back current, cited material.
05.The tool-calling loop
Now the two halves connect. The model cannot reach your search endpoint on its own, and that is by design. Instead the flow goes through the browser, which acts as the courier between OpenAI and your server. It plays out like this:
- The user asks a question out loud. OpenAI hears it and decides it needs to look something up.
- The model emits a
search_newsfunction call with a query string. It does not run anything; it just announces what it wants. - The browser catches that request and calls the
/api/news/searchendpoint, which runs the Seltz search. - The browser hands the results back to the model and asks it to continue. The model reads the documents and speaks a short summary.
The middle two steps are the courier code in the browser. When a function call arrives over the realtime data channel, this runs:
// in the browser: the model asked to search, so we run the tool for it
async function runSearch(callId: string, query: string) {
const res = await fetch(`/api/news/search?q=${encodeURIComponent(query)}`);
const output = JSON.stringify(await res.json());
// hand the results back to the model over the realtime data channel,
// then ask it to continue (this is the part the user hears spoken)
dataChannel.send(
JSON.stringify({
type: "conversation.item.create",
item: { type: "function_call_output", call_id: callId, output },
})
);
dataChannel.send(JSON.stringify({ type: "response.create" }));
}The model never sees the Seltz key, never sees the search endpoint, and cannot call it directly. It only knows how to ask, in words, for a search. The server decides whether and how to honor that. That indirection is not just a security nicety; it is what keeps control of the tool on the server side: what it actually does, how results are shaped, and what the model is allowed to act on.
06.The trust model
Step back and look at where the secrets live, because for anything you would put in front of real users this is the part that matters most. There are two long-lived keys in this system: your OpenAI key and your Seltz key. Both stay on the server, read from environment variables, and neither is ever sent to the browser.
What the browser receives is only ever the short-lived realtime token, scoped to one session and quick to expire. Audio flows directly between the browser and OpenAI using that token. The Seltz search runs entirely server-side, triggered by a request the browser relays but executed with a key the browser cannot see. If someone opens the network tab and reads everything the page sends and receives, they learn nothing they could reuse to run up a bill on your account.
That separation is what makes this pattern safe to ship. You get the low latency of a direct browser-to-model audio connection without exposing the credentials that pay for it, and the one capability that reaches the live web stays behind your own endpoint where you can rate-limit it, log it, and change it.
07.Conclusion
The voice experience here is genuinely OpenAI’s to own: it does the listening, transcribing, reasoning, and speaking over a single WebRTC connection, and their guide is the right place to go deep on it. What turns that smooth conversation into something you would actually trust about the news is one short server route and three lines of Seltz.
That is the takeaway worth keeping. Fresh, source-backed knowledge did not require a retrieval stack or a pile of infrastructure. It required asking Seltz a question and handing the answer to the model. If you want to try it, grab a key from the Seltz console and point a search call at whatever your assistant needs to know.