Better Tools for Agents (Or None at All)

We run AI agents inside company chats. When a teammate @-mentions an agent, the message triggers a model, and the agent replies in the thread.

Why require an explicit @-mention instead of letting the agent monitor every message? Because unconstrained monitoring turns an agent into a firehose of context bloat, token burn, and unsolicited interruptions. Requiring an explicit @-mention keeps operations disciplined and costs low.

The catch: the agent lands blind. The previous message might have arrived two minutes ago or eleven days ago, while teammates argued over invoices, ordered lunch, or posted stickers.

We evaluated four interface architectures by replaying production chat traffic through our evaluation harness.

You're the agent

A user @-mentions you:

"What did I miss? What is the latest on the vendor review?"

Over the last eleven days, teammates sent dozens of messages: they debated schedules, discussed invoices, and swapped stickers. Buried in that thread, someone rescheduled the review from Tuesday Aug 11 to Thursday Aug 13.

If you answer "Tuesday", your team acts on stale information.

You have four candidate architectures:

Op = Literal["recent", "unread", "range", "search", "by_sender", "participants"]

# A. dispatch — One tool, six operations.
def chat_log(*, op: Op, n=20, text=None, sender=None, since=None, until=None) -> str: ...

# B. flat optional — Single signature, all parameters optional, no mode enum.
def chat_log(*, n=None, text=None, sender=None, since=None, until=None, unread=False): ...

# C. split — One tool per operation. Six separate schemas in the prompt.
def chat_recent(*, n=20): ...
def chat_search(*, text: str, n=50): ...
def chat_unread(*, n=100): ...          # + by_sender, range, participants

# D. ambient context — Zero tools. Timestamped history injected directly in the prompt.

The scoreboard

We benchmarked the candidate architectures under identical prompt conditions:

Interface	Tool Slots	Valid Call Rate	Avg Turns	Accuracy	Prompt Tokens
A dispatch	1	52.2%	3.00	90.0%	2,076
B flat optional	2	78.3%	2.43	100.0%	1,405
C split tools	6	55.0%	2.97	96.7%	2,823
D ambient context	0	—	1.00	100.0%	379 (p50)

Interface B (flat optional parameters) won decisively among tool designs: 100% accuracy, zero failures, and ~32% fewer prompt tokens than dispatch. When we removed the required op enum, the model no longer hesitated or mismatched arguments.

Interface C (split tools)—the "clean" design with one crisp function per operation—wasted tokens. Declaring six distinct schemas consumed 2,823 prompt tokens on every single request, whether the agent needed history or not.

Architecture D (ambient context) injected timestamped history directly into the prompt. It answered every question in a single generation turn (1.00 turn), made zero tool calls, and used only 379 tokens at p50.

Which raises the obvious question: why not use Architecture D for everything?

The ambient context baseline

Architecture D eliminates tool calling entirely. When someone @-mentions an agent, the gateway injects recent channel history directly into the prompt alongside timestamps and elapsed-time markers:

----- BEGIN EPHEMERAL LINE GROUP CONTEXT -----
# 16 unread since 2026-06-18 10:46 (8d 8h ago)
[2026-06-19 16:09] Teammate A (U1234): ...
[2026-06-20 10:01] Teammate B (U5678): ...
[2026-06-22 10:51] Teammate B (U5678): ...
----- END EPHEMERAL LINE GROUP CONTEXT -----
----- BEGIN CURRENT LINE REQUEST -----
[Teammate A/U1234] What did I miss? What is the latest on the vendor review?
----- END CURRENT LINE REQUEST -----

showing newest 15. For the rest: chat_log {"op":"unread"}

The model reads a clear chronological timeline with zero schema overhead, zero round trips, and zero syntax errors. It reads the context just as a human reads a chat thread.

In benchmark runs, the model answered every question within this ambient window in a single generation turn with zero tool calls. It identified the rescheduled review date without querying the database.

"You just moved the data into the prompt."

Right, and that is the core insight: the cheapest tool call is the one you eliminate by placing the answer in the prompt. When recent history contains the answer, ambient context wins decisively on latency, accuracy, and total cost.

"You can't dump six months of chat into every turn."

Also true. And that is the tradeoff: Always-Pay vs. Pay-On-Demand.

We spend 379 baseline tokens ($p50$) to inject 15 recent messages into the prompt on every turn. If the user asks a recent question, Architecture D answers immediately for 379 tokens with zero tool calls.

If the answer sits 11 days in the past—beyond the prompt buffer—Architecture D cannot answer alone. The agent must read the backlog header and trigger an on-demand tool call, paying the 379 baseline prompt tokens plus tool execution overhead.

Ambient context forms the first tier of a hybrid system rather than a complete replacement for tools:

• Tier 1 (Ambient Context): Answers frequent recency queries (over 80% of production mentions) in a single turn without tool round-trips.
• Tier 2 (On-Demand Retrieval): Runs only when conversations exceed the prompt buffer and the agent must search older history.

How models actually query: Semantic entities over database cursors

When conversations outgrow the prompt window, agents still need on-demand tools. To design those tools, we must understand how models query.

Database engineers think in storage mechanics: offsets, pagination tokens, and sliding time windows ("fetch everything since timestamp T").

Language models think in semantic targets: topics ("vendor review"), people ("Jordan"), and calendar anchors ("last Thursday"). When an agent searches for information, it does not calculate time deltas. It filters by sender (by_sender), selects date ranges (range), or runs keyword searches (search). Even on straightforward retrieval queries, real-model execution exposes subtle behavioral patterns. For example, in our evaluation runs, a model asked to look up recent history emitted:

{"chatId": "line-ops-planning", "limit": 100}

This single invocation highlights three runtime dynamics:

1. Omitted operation: It selected none of the explicit operation modes. A tolerant fallback (op="recent") saved the turn; an enforced enum validator would have crashed.
2. Hallucinated parameter: It inferred a chatId parameter that was never in the schema.
3. Aliased parameter: It passed limit instead of n, succeeding only because our runtime layer defensively normalized aliases.

Our defensive runtime handler prevented an immediate crash. Models do not follow schema names with perfect precision. Your runtime harness must handle minor variations gracefully.

When schema validation creates false claims

Rigid schema validators magnify the flaws of poor tool design.

In our tests, permissive clients ignored unexpected arguments like chatId, but an upstream JSON Schema validator crashed on an oversized argument:

{"op": "range", "since": 0, "until": 1786992000000, "limit": 1000}

Invalid args for chat_log: Validation failed:
  - limit: must be <= 200

The tool schema declared maximum: 200. We wrote defensive clamping into our backend handler (limit = min(limit, 200)), but the upstream validator rejected the request before our handler ever ran.

When the validator failed, the agent did not retry with a lower limit. Instead, it treated the error as empty data and made a false claim:

"I checked the full available chat history … the history contains only your two requests."

Weak reasoning did not cause this hallucination. A rigid schema boundary misled the agent.

The rule: Do not set rigid schema constraints on parameters you can clamp, sanitize, or default in handler code. Let the handler absorb edge cases, and save strict schema validation for unrecoverable syntax errors.