Training a specialized hotel booking model

01

Why current AI deployments fail

Most booking agents today are built on general-purpose LLMs like GPT-4o, which aren't designed for multi-turn transactional workflows. 60% of organizations report minimal revenue and cost gains from AI despite substantial investment, with only 5% achieving value at scale (BCG, 2025).

To understand why, consider what the task actually requires. A guest doesn't submit a fixed request. They engage in a multi-turn conversation where preferences change and constraints are added, updated, and corrected. The system must maintain an accurate, persistent state of those requirements in order to retrieve and confirm the correct inventory. This is called dialogue state tracking (DST), and it's where general-purpose models fail in production. Rather than maintaining structured state, they re-interpret the conversation at each turn, leading to dropped or overwritten constraints.

Hotels already know this problem under a different name: the average handle time for a mishandled booking requiring human rework is 3 to 5 minutes per interaction (Zendesk, 2025). Multiply that across tens of thousands of interactions and the cost of unreliable AI isn't abstract.

02

A concrete example of how errors propagate

This is how most general-purpose AI booking agents fail in production.

A guest opens a chat and asks for a room for two adults this weekend. The agent asks about bed type. The guest says a king if possible, but flexible. The agent asks about budget. The guest says mid-range, around $150 a night. The agent surfaces three options.

The guest then says "actually, can we do next weekend instead, my partner just checked and this weekend doesn't work".

The model updates the dates. But in re-querying availability for next weekend, it drops the $150 budget constraint captured two turns earlier. It returns options including a $240 room. The agent recommends it as the best fit based on bed type and availability. The guest, assuming the agent remembered their budget, confirms.

They find out the price at checkout. They abandon the booking, frustrated after ten minutes of back and forth that led nowhere, and book elsewhere.

A single dropped constraint cascades through the full pipeline into an incorrect query and a wrong confirmation. In production systems where agents query live inventory across multiple data sources, the same failure becomes harder to catch and harder to reverse.

03

The Kairos pipeline

Kairos built a specialized model trained specifically on hotel booking conversation workflows. The approach combines domain-specific fine-tuning with a preference optimization stage to produce a model that maintains accurate dialogue state across the full conversation. The model is approximately 30x smaller than GPT-4o.

We evaluated this against a basic GPT-4o integration and a GPT-4o integration given full scope and high context of the task at hand.

Average Daily Rate (ADR). The average revenue a hotel earns per occupied room per night. The US hotel industry ADR hit $162 in 2025, combined with an average 2.3 night stay for business travelers (Prostay, 2026). These are the figures behind our revenue estimate.

Slot Extraction F1. Measures how accurately the model identifies individual booking attributes (location, dates, price range, amenities) from a guest message at a given turn in the conversation.

Joint Goal Accuracy (JGA). Measures whether the model's complete accumulated booking state is fully correct at every point in the conversation. Every slot must be simultaneously correct; a single dropped or overwritten constraint counts as a failure.

Tokens Used. Total units of text processed by the model. Directly determines API cost and latency per interaction.

Cost (USD). Total API spend for the benchmark run. Kairos cost reflects infrastructure only; no per-token API charges.

BERT Score. Measures semantic similarity between the model's response and the ideal response. Captures meaning-level accuracy rather than exact word matching. The Kairos Pipeline scores 0.898 versus 0.861 for basic GPT-4o, reflecting stronger semantic alignment with ideal customer service outputs.

BLEU Score. Measures how closely the model's guest-facing response matches gold-standard customer service outputs. Higher score means more natural, on-brand language.

04

Results

Benchmarked against GPT-4o on multi-turn hotel booking dialogues:

Kairos pipeline cost reflects infrastructure only, optimized per client.

Joint Goal Accuracy is the metric that matters most for booking reliability. GPT-4o's basic integration achieves a JGA of 2.6%. The high-context engineered prompt raises this to 21.5%, but at nearly 2.5x the token cost ($6.42 versus $3.47). The Kairos Pipeline reaches 37.9% JGA while consuming a similar amount of tokens to the basic GPT-4o integration. The BLEU score improvement from 0.02 to 0.143 reflects guest-facing response quality meaningfully closer to what a hotel brand would want delivered.

05

What these numbers mean for your business

Over half of travelers already abandon hotel booking flows before completing a reservation (SiteMinder, Changing Traveller Report 2025). Every percentage point of JGA improvement is a direct reduction in sessions that end in a wrong confirmation or an abandoned conversation. At the scale hotels operate, that translates into three concrete outcomes. The figures below assume 10,000 monthly interactions.

Revenue. Recovering just 5% of failed bookings at the US average daily rate translates to over $180,000 in monthly recovered revenue.

Time saved. Every mishandled booking requires 3 to 5 minutes of staff rework. Resolving those correctly the first time recovers up to 833 staff hours per month.

Cost. The pipeline runs on a model 30x smaller than GPT-4o, consuming fewer tokens than even a basic GPT-4o integration. For hotels already paying per API call, the difference compounds fast. The same pipeline run on GPT-4o would cost roughly $0.021 per interaction. The Kairos architecture costs $0.00066 per interaction, a reduction of over 70% versus current systems.

06

This is a baseline

This benchmark was run without client-specific operational data. With real booking history, property-specific inventory context, and operator feedback loops applied to model outputs, accuracy improves further. The closer the training data is to your actual workflows, the closer performance gets to deterministic reliability.

Academic state-of-the-art models achieve higher benchmark scores by optimizing purely for test set performance. Our results are benchmarked against the actual production baseline enterprises deploy today, and our approach optimizes for the full deployment stack: training cost, latency, model size, and operational reliability.

07

Beyond hotels

The failure pattern documented here isn't unique to hospitality. Insurance claims intake, healthcare prior authorization, legal client intake, financial services onboarding: every industry deploying AI agents for customer-facing transactional workflows faces the same problem. A general-purpose model gets dropped into a multi-turn conversation it was never trained for, extracts the wrong information from one of the turns, and the error quietly propagates through the rest of the workflow. By the time a human notices and steps in to correct it, the ROI that AI automation promised has already evaporated into rework.

In each case the workflow is identical: a conversation, a structured extraction, a database query, a critical action. In each case the fix is the same: a specialized small model trained on domain-specific dialogue, paired with evaluation infrastructure that verifies outputs before they become decisions.