Reading Google AI Overviews programmatically

There is no official Google API for AI Overviews. Not a restricted one, not a waitlist — the block is generated in the search result page and no Google developer programme exposes it. The Custom Search JSON API returns classic web results and has never included it.

That leaves everyone who needs the data in the same position: read it from the rendered result page, directly or through a provider that does it for you. This article covers what the block actually is, how it differs from AI Mode, what a response contains, and — the part most teams get wrong — why a single sample of an AI Overview is close to meaningless.

AI Overview and AI Mode are not the same surface #

They get used interchangeably and they behave differently, which matters if you are measuring anything.

AI Overview is a block inside the ordinary results page. The user searched normally; Google decided this query deserved a generated summary and put one above the organic results, with links out to the sources it drew on. The organic ranking still exists underneath it. So for one query you can observe both surfaces at once and compare them — which is the whole reason the block is interesting.

AI Mode is a separate conversational surface the user has to enter. It answers at greater length, follows up, and does not sit above a classic ranking. It is closer to a chat assistant that happens to be inside Google.

Two consequences:

1. 1.A domain can win one and lose the other. Ranking third organically while being absent from the AI Overview above it is completely normal, and is the single most useful diagnostic these endpoints give you.
2. 2.AI Mode costs more to measure and triggers less. Treat it as an addition to a measurement programme, not the default.

Pulling the block #

POST /v1/serp/ai-overview runs the organic SERP and extracts the AI Overview block from it. The request is keyword-first, with explicit location and language rather than inherited defaults:

curl -X POST https://api.dataswap.io/v1/serp/ai-overview \
  -H "Authorization: Bearer $DATASWAP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "keyword": "best crm for startups",
    "location_name": "London,England,United Kingdom",
    "language_code": "en",
    "device": "desktop"
  }'

// npm i dataswap — no dedicated method yet, so use the escape hatch
import { Dataswap } from 'dataswap';

const dataswap = new Dataswap({ apiKey: process.env.DATASWAP_API_KEY });

const res = await dataswap.request('POST', '/v1/serp/ai-overview', {
  keyword: 'best crm for startups',
  location_name: 'London,England,United Kingdom',
  language_code: 'en',
  device: 'desktop',
});

if (!res.ai_overview) {
  console.log('no AI Overview for this query/location/device');
} else {
  for (const ref of res.ai_overview.references) {
    console.log(ref.domain, '→', ref.url);
  }
}

# pip install dataswap
import os
from dataswap import Dataswap

client = Dataswap(api_key=os.environ["DATASWAP_API_KEY"])

res = client.request("POST", "/v1/serp/ai-overview", {
    "keyword": "best crm for startups",
    "location_name": "London,England,United Kingdom",
    "language_code": "en",
})

overview = res.get("ai_overview")
print([r["domain"] for r in overview["references"]] if overview else "not triggered")

The response has four top-level parts:

{
  "search_metadata":   { "id": "…", "engine": "…", "credits_used": 1, "cached": false },
  "search_parameters": { "keyword": "best crm for startups", "device": "desktop" },
  "ai_overview": {
    "markdown":   "…the generated block, as markdown…",
    "text":       "…the same content as flat text…",
    "references": [
      { "title": "…", "url": "https://…", "domain": "example.com", "source": "…", "text": "…" }
    ]
  },
  "organic_results": [
    { "position": 1, "title": "…", "url": "https://…", "domain": "example.com", "snippet": "…" }
  ]
}

ai_overview is null when the SERP did not carry a block. That is not an error and it is not rare — see the pitfalls below. organic_results is returned alongside, which is what lets you compare the two surfaces from a single call.

For AI Mode, POST /v1/serp/ai-mode takes the same keyword/location/language shape and returns ai_mode with markdown, text and references — no organic block, because there is no ranking to compare against.

Base cost is 1 credit for AI Overview and 2 for AI Mode; a credit is $0.002. The exact amount charged comes back in the response and in the X-Credits-Used header, so you never have to infer it.

The comparison worth building #

The valuable output is not the summary text. It is the set difference between who ranks and who gets cited:

const cited = new Set(
  (res.ai_overview?.references ?? [])
    .map((r) => r.domain)
    .filter((d): d is string => Boolean(d)),
);
const ranked = res.organic_results
  .slice(0, 10)
  .map((r) => r.domain)
  .filter((d): d is string => Boolean(d));

const rankedNotCited = ranked.filter((d) => !cited.has(d));
const citedNotRanked = [...cited].filter((d) => !ranked.includes(d));

rankedNotCited is your list of pages that won the classic SERP and lost the block above it. citedNotRanked is more interesting still: domains the generator trusted enough to cite without them ranking on page one. Both lists are the raw material of any serious answer-engine strategy.

Third-party research suggests the overlap between classic ranking and AI citation is far smaller than people assume — Digital Applied reports that only about 2.1% of pages in Google's top ten also appear in ChatGPT's citations. That is their measurement, not ours, and worth re-deriving on your own keyword set — which is exactly what the two lists above let you do.

Limits and failure modes #

Triggering is not stable, and this is the big one. Whether a query produces an AI Overview varies by query, location, language, device, signed-in state and time. The same keyword can return a block in one call and null in the next, with nothing wrong on either end. Any code that assumes ai_overview is present will crash in production; any measurement that assumes it is present will be wrong.

The practical consequence: a single sample tells you nothing. If you want "does this keyword have an AI Overview", you need repeated sampling over days and a trigger rate, not a boolean. If you want "are we cited", you need the same. Report n alongside every percentage you publish internally, and treat any keyword sampled once as unmeasured.

Generated text is not stable either. Two calls minutes apart can return differently-worded summaries citing an overlapping-but-different set of sources. Diff the citation set, not the prose. Anything that alerts on wording changes will alert constantly and get muted within a week.

Location is not a nice-to-have. AI Overview content and triggering are geographically sensitive. Sending no location gives you some location, not a neutral one. Pin location_name (or location_code) explicitly and keep it constant across a measurement series, or you are comparing two different populations and calling the difference a trend.

Device changes the answer. Measure the device your audience actually uses, and do not mix desktop and mobile samples in one average.

References are what the block links to, not a bibliography. A domain can be described in the summary without being linked, and a link is not proof the claim came from that page. Treat references as attribution signal, not ground truth.

Caching cuts both ways. A cached response is cheaper and faster, and useless for measuring volatility. search_metadata.cached tells you which one you got — check it before you record a data point in a time series.

Reading the block is not permission to republish it. The summary is generated content on a search results page. Pulling it for analysis, monitoring and internal reporting is an ordinary competitive intelligence use; republishing the text as your own content is a different act with different legal exposure. If you plan to display any of it publicly, ask your legal team first. This is not legal advice.

A measurement loop that survives contact with reality #

1. 1.Fix the variables. One location, one language, one device per series.
2. 2.Sample on a schedule, not on demand. Daily is enough for most keyword sets; hourly only for things you genuinely react to within the hour.
3. 3.Store the raw response, not just your derived metrics. When a number looks wrong in three months, the only way to find out why is the payload you kept.
4. 4.Record the trigger rate as a first-class metric next to citation share. A drop in citations caused by the block no longer appearing is a completely different problem from being replaced by a competitor, and only the trigger rate distinguishes them.
5. 5.Compare sets, not strings. Citation domains in and out over time is the signal; wording is noise.

Where to go next #

If what you want is not one surface but the whole picture — organic, AI Overview, AI Mode and several LLM assistants answering the same question, with share-of-answer computed across all of them — that is a composition, and building it call-by-call gets fiddly fast. The next article covers the methodology: what GEO is and how to measure it.

The full response shape for both routes is the one shown above. The docs cover authentication, the credit envelope and idempotency, the API reference carries the schemas for the rest of the surface, and AI search visibility is the product page for measuring these surfaces continuously.