A one-second latency target for AI booking

A latency budget for conversational AI booking is an upper bound on user-perceived round-trip time, split across network, model call, enrichment APIs, and render. Typelessity designs the booking turn for one second — a target, not a measurement. What makes a budget like that reachable is architecture — one extraction call per turn, not one per field — rather than post-hoc optimization. Read the single-call architecture, and how it works for the four phases.

The AI-product latency death spiral: call a model, then call an enrichment API, then call another model to interpret the result, then re-render. Every hop adds a network round-trip and processing time. Three hops and the user starts wondering if the page is broken.

Typelessity sets a one-second target for the user-perceived round-trip from "send" to "first response visible". We have not measured it on the current model, so this post describes the target and the architecture, not a result.

What is a latency budget?

A latency budget is an upper bound on user-perceived round-trip time, allocated across the phases that contribute to it. The budget is the architectural constraint; the architecture must fit inside it. Typelessity's target allocation — a plan, not a measurement:

Target:                1000 ms
  Network round-trip:   100 ms (user ↔ API)
  Model call:           600 ms (OpenAI, currently gpt-5.4-mini)
  Enrichment APIs:      200 ms
  Render and paint:     100 ms
                       --------
                       1000 ms

Without a measurement, the target cannot yet tell us when a turn runs over it; measuring is the next step (see below).

Bottom line: pick the budget first; architect to fit. The order matters.

What does the current code do inside the budget?

One extraction call per turn. Every field the user mentioned is extracted in one model round-trip, not one per field. When the user taps an option card whose value matches exactly, no model call runs at all. Architecture detailed in Replacing a booking form with one GPT call.

Enrichment after the call, one API at a time. When the model returns a field that triggers an enrichment — service: "brakes" → GET /mechanics?service=brakes — the engine calls the business's API after the model call, one enrichment after another. Running them in parallel with the render is not built.

No streaming. The widget waits for the complete response and shows a typing indicator meanwhile. Streaming tokens to bring the first visible text forward is not built.

What is cut to stay inside?

One model call by default, not a chain. The first call has to do the work. A second call runs in one case: when an enrichment loads options and the user has already stated a preference, a second call picks among them. If the call still fails, the widget asks the user for the next field instead of chaining more model calls.

No retrieval step before the prompt. The prompt carries the configuration itself — the business's fields, options and rules — and no vector-database lookup runs before the call.

No third-party analytics SDKs in the widget. The widget that loads on a customer's website carries no PostHog, Mixpanel or similar script — and the engine has no analytics endpoint of its own either.

Bottom line: the things you remove to stay in budget tell you more about the architecture than the things you keep.

What is outside the budget?

Voice transcription. A separate call to OpenAI gpt-4o-mini-transcribe runs after the user stops recording and before the turn. The widget shows its typing indicator meanwhile. Not measured. Details in Whisper vs Web Speech API for booking voice.

Cascade corrections. When a user edits a field that others depend on, the widget first asks them to confirm which fields will be cleared, then clears them and re-runs the affected enrichments. Not measured. Mechanics in Cascade corrections in conversational booking.

Turns with a second model call — when an enrichment loads options and a preference already exists (see above).

These are documented exceptions, not violations. The design rule is not "every interaction must be under one second" — it is "the booking happy path is designed for one second, and every exception must be legible to the user."

How does Typelessity measure?

It does not yet. The engine logs how long each model call takes, but that number stays in the server log: no percentile is collected, stored or published. That is why this post quotes a target and no result. Until we measure, treat any latency figure for Typelessity — the one-second target included — as a design goal.

Direct comparison summary

Sources of latency in conversational AI booking, ranked by typical contribution:

• Model call → largest single phase
• Enrichment APIs → second largest if serial; small if parallel
• Network round-trip → depends on how close the server is to the user
• Render and paint → small if streaming, large if blocking
• Third-party SDKs → cumulative; our hypothesis is that a few of them can eat a share of the budget on their own, and we have not measured it
• Re-prompting / second model call → the hardest to fit in a one-second budget

Architectural choices that help hold a budget: single-call extraction, parallel enrichment, streaming UI, no vendor SDKs. Typelessity today has the first and the last; parallel enrichment and streaming are not built. Choices that break a budget: chained model calls, serial enrichment, blocking renders, vendor SDKs.

When the 1-second budget is the wrong target

• Long-form generation (drafting an email, writing a contract clause) tolerates round-trips of several seconds. Streaming is the user-facing affordance.
• Voice-only interfaces (Alexa, Google Home) can run on longer budgets because the user has nothing else to do during the wait.
• Agent flows with explicit "thinking" UI can run multi-second model calls if the affordance is loud enough.

For booking surfaces — where the user is at a customer-facing site and a slow widget reflects on the customer's brand — sub-second is the bar to design for. For chat with a financial advisor — multi-second is fine.

FAQ

What is a latency budget for an AI booking widget? An upper bound on the user-perceived round-trip, split across phases. Typelessity designs for one second — a target, not a measurement.

Why does Typelessity treat latency as architectural? Because two model calls in series cannot be optimized into one retroactively. The architecture must fit the budget at design time.

What does the widget show while the model is working? A typing indicator. It waits for the complete response; there is no token streaming and no acknowledgement copy.

What is outside the one-second target? Voice transcription, cascade corrections, and turns with a second model call. None is measured yet.

What is left out of the widget to keep it light? Third-party analytics SDKs — and there is no in-house analytics endpoint either.

For the single-call extraction architecture, see Replacing a booking form with one GPT call. For the cascade exception, see Cascade corrections in conversational booking. For the voice-input exception, see Whisper vs Web Speech API for booking voice. For the multilingual prompt, see 25 interface languages, one booking prompt.

— Alex Isa, founder of Typelessity. Also founder of Webappski and TypelessForm.