Grounding an LLM agent with search data

Most teams wire a search API into an agent the same way: call the search endpoint, take organic_results, JSON.stringify the top ten, paste them into the prompt, ship it. It works in the demo. Then the agent starts citing the wrong page, contradicting itself between turns, or quietly eating 8,000 tokens per step, and nobody can say which of the three problems came first.

The root cause is not the model and usually not the search provider. It is that a SERP response and a grounding context are different data structures, and the conversion between them is real work that most pipelines skip.

This article is about that conversion. The technique applies whatever your search provider is — the last section shows the shortcut, but the first four are worth implementing yourself if you would rather own them.

Why raw search results are the wrong shape #

A search response is optimised for a human scanning a page. A grounding context is optimised for a model that will read every token you send it and charge you for the privilege. Four concrete mismatches:

It is full of envelope. A single organic result carries position, displayed URL, sitelinks, rich snippet fragments, tracking parameters, and often a duplicate of the description under another key. The model needs roughly three fields — title, text, URL — and pays attention tax on the rest. Serialised JSON is also a hostile format for an LLM to read: braces and quotes are tokens too.

It is redundant. Query anything mildly newsworthy and results three, five and nine are the same wire story on three domains. You pay for those tokens three times, and worse, the model reads three sources agreeing and treats the claim as well-corroborated when it has one source behind it. Duplication does not just cost money, it manufactures false confidence.

It is ranked for the wrong question. Search ranking answers "what is a good page for this query string". Your agent has a question, and the query string is a lossy compression of it. The page ranked first for postgres connection pooling is a good landing page for that phrase; the block that answers "why does my pooler drop connections under load after an upgrade" may be on result seven.

It has no budget. Ten results might be 1,200 tokens or 6,000 depending on snippet lengths, and you find out after you have already sent them. Anything with a context limit and a per-step cost needs a number it can plan against, not a distribution.

The five steps that fix it #

1. Over-retrieve, then throw away #

Ask for more results than you intend to send — twenty when you plan to use five. Retrieval is cheap relative to context tokens, and selection is only as good as the pool it selects from. If you retrieve exactly what you send, you have no selection step at all, you have a rename.

2. Deduplicate on content, not on URL #

URL deduplication catches almost nothing: syndicated copies live on different domains, and the same page appears as ?utm_source=…, amp/, and the canonical. Two things that work:

• Near-duplicate text detection. Normalise (lowercase, strip punctuation, collapse whitespace), then compare with MinHash/SimHash over shingles, or cosine similarity over embeddings. Anything above a similarity threshold collapses to the highest-ranked member.
• Claim-level dedup, if you can afford it: split into sentences and drop sentences already covered. Better results, more moving parts.

Keep the dropped URLs attached to the survivor. Three independent sources for a claim is genuinely useful signal for the model — just do not pay for the text three times.

3. Rerank against the question #

This is the step with the largest effect and the one skipped most often. A cross-encoder reranker scores each candidate jointly with the query, instead of comparing two independently-computed vectors, which is why it catches relevance that embedding similarity misses.

You have three options, in increasing order of how much you own:

# Option A — self-hosted cross-encoder. You own the weights and the GPU.
from sentence_transformers import CrossEncoder

reranker = CrossEncoder("BAAI/bge-reranker-v2-m3")
pairs = [(question, c["text"]) for c in candidates]
for candidate, score in zip(candidates, reranker.predict(pairs)):
    candidate["score"] = float(score)
candidates.sort(key=lambda c: c["score"], reverse=True)

Option B is a hosted reranking endpoint — same idea, someone else's hardware. Option C is asking your main LLM to rank the candidates, which works but costs a full model call and is not reproducible between runs.

Whichever you pick, rerank against the user's actual question, not the search query you derived from it. The query was for the search engine. The question is what the answer has to satisfy.

4. Trim to a measured token budget #

Not an estimated one. len(text) / 4 is a rule of thumb that breaks on code, URLs, non-Latin scripts and long compound words — exactly the content search results are full of. Count with the real tokenizer:

import tiktoken

enc = tiktoken.encoding_for_model("gpt-4o")

def pack(blocks, budget=4000):
    """Greedily fill a token budget with the highest-scoring blocks."""
    out, used = [], 0
    for b in blocks:                      # already sorted by rerank score
        n = len(enc.encode(b["text"]))
        if used + n > budget:
            continue                      # skip, do not stop: a smaller block may still fit
        out.append({**b, "tokens": n})
        used += n
    return out, used

Note continue rather than break. Stopping at the first block that does not fit wastes the tail of the budget on nothing; skipping it lets a shorter, still-relevant block use the space.

Pick the budget deliberately. It is the one number that directly sets your per-step cost, and in a multi-turn agent it is re-read on every single turn — a context pack is not paid for once, it is paid for once per turn it stays in the window.

5. Keep the citation welded to the text #

Every block travels with its URL from retrieval to prompt. Do not plan to "add citations later" — after the model has written a paragraph, attributing it back to a source is guesswork, and guessed citations are worse than none because they look verified.

The prompt-side format that costs least and works well:

[https://example.com/postgres-pooling] PgBouncer in transaction mode releases the
server connection at commit, so session-level state such as prepared statements or
SET commands does not survive between transactions.

[https://other.example/upgrade-notes] Version 1.21 changed the default pool_mode …

Then instruct the model to cite the bracketed URL for any claim it takes from a block. You now have an answer you can audit: every sentence either points at a URL you fetched or is the model's own reasoning, and the two are distinguishable.

Doing all five in one call #

Steps 2 through 4 mean owning an embedding model, a reranker and a tokenizer, plus the infrastructure to run them. That is a reasonable thing to own. If it is not the thing you want to own this quarter, POST /v1/context does the whole conversion and returns blocks that are already deduplicated, reranked and trimmed:

curl -X POST https://api.dataswap.io/v1/context \
  -H "Authorization: Bearer $DATASWAP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "eu ai act compliance deadlines",
    "token_budget": 2000,
    "depth": 20
  }'

// npm i dataswap
import { Dataswap } from 'dataswap';

const dataswap = new Dataswap({ apiKey: process.env.DATASWAP_API_KEY });

const pack = await dataswap.context({
  query: 'eu ai act compliance deadlines',
  token_budget: 2000,
  depth: 20,
});

const grounding = pack.blocks
  .map((b) => `[${b.url}] ${b.text}`)
  .join('\n\n');

console.log(pack.total_tokens, '<=', 2000);

# pip install dataswap
import os
from dataswap import Dataswap

client = Dataswap(api_key=os.environ["DATASWAP_API_KEY"])

pack = client.context("eu ai act compliance deadlines", token_budget=2000, depth=20)
grounding = "\n\n".join(f"[{b['url']}] {b['text']}" for b in pack["blocks"])

Each block comes back as { text, url, title, score, tokens }. score is the reranker's, so you can apply your own floor and drop weak blocks. tokens is counted, not estimated, and total_tokens is guaranteed not to exceed the budget you asked for — which is what makes it a number you can plan against. depth (1–20) controls how many candidates are considered before reranking; token_budget accepts 256–32000 and defaults to 4000.

The response also carries request_id, credits_used for that exact call, and a signed provenance receipt describing how the data was obtained. Intelligence-layer calls reserve a ceiling while they run and seal at the real cost, so credits_used is the number that was actually charged — read it rather than assuming a list price.

If you only want the reranking step and already have your own candidates, POST /v1/ai/rerank takes a query and up to 512 documents and returns them ordered.

Limits and failure modes #

Snippets are not pages. /v1/context builds blocks from what the search surface returns — title, description, and with mode: "deep" the extended snippet and breadcrumb when the upstream provider supplies them. Neither mode fetches the body of the pages. If your question needs the third paragraph of a document, retrieval-from-snippets will not find it, and no amount of reranking fixes that. Fetch and parse the page, or use an extraction endpoint that is documented to read page content.

Reranking cannot invent relevance. If the right document is not in the retrieved pool, the reranker will confidently order a set of wrong answers. Symptoms of an under-retrieval problem look identical to a ranking problem from the outside. When quality is bad, raise depth before you touch anything else.

A token budget is a hard constraint, not a quality setting. Cutting the budget cuts blocks, and past a point you are removing the source that had the answer. Watch how many blocks survive, not just whether you stayed under budget.

Freshness is a property of the index, not of your call. Search surfaces do not have every page, and pages they have may be stale. For anything time-sensitive, treat the retrieval timestamp as part of the evidence — a grounded answer with an old source is still a wrong answer, just a traceable one.

Dedup thresholds are a judgement call. Set them too tight and you keep near-identical wire copies; too loose and you drop a source that genuinely said something different. There is no universally correct threshold; there is only the one you validated on your own queries.

What to take away #

Grounding is not "put search results in the prompt". It is a conversion with five distinct steps — over-retrieve, deduplicate on content, rerank against the question, trim to a measured budget, carry citations — and each one is a place where a working demo turns into an unreliable product.

Build it yourself if the retrieval pipeline is your differentiator. Use the grounding API if it is not. Either way, measure the token count and keep the URL attached to the text; those two habits catch most of what goes wrong.

If your agent speaks MCP rather than HTTP, the same context step is available as a tool — see MCP search tools for agents. Full request and response schemas for every route are in the API reference.