News

Rank tracking by API, without fooling yourself

Build rank tracking that reports something true: pin location and device, diff sets instead of averaging positions, and know why keyword volumes never agree.

7 min read
On this page

Rank tracking looks like the easiest thing in SEO to automate. Search a keyword, find your domain, store the number, chart it. Every team that builds it discovers the same three problems in the same order: the number moves when nothing changed, the average across keywords means nothing, and two tools report different volumes for the same word.

None of those are bugs. They are properties of what is being measured, and a tracker that does not account for them produces charts that are worse than no charts — because people act on them.

This article is how to build one that reports something true. The API calls are ours; the methodology is not specific to any provider.

Position is a sample, not a fact #

A ranking is not a stored value you look up. It is generated per request, influenced by at minimum:

  • Location, down to city level for many query types
  • Language
  • Device — desktop and mobile are different result sets, not a formatting difference
  • Personalisation and history, which you cannot reproduce
  • Time, including ordinary index churn with no cause you will ever identify

So "we rank 4th" is shorthand for "in one sample, with one configuration, at one moment, this URL was 4th". Every design decision below follows from taking that literally.

Pin every variable you can, and hold them constant for the life of the series. Location, language, device, depth. A tracker that sends no location does not get a neutral result — it gets some location, chosen for it, possibly a different one next week. When your chart moves, you will not be able to tell whether the ranking changed or the sample did.

Absolute rank is not the number you think #

Two different quantities get called "position", and mixing them is a classic source of phantom movement:

  • Organic-only position — count the ordinary results, ignore everything else. You are 4th among blue links.
  • Absolute position — count every block on the page: ads, the AI Overview, People Also Ask, local packs, video carousels. You are 9th on the page.

They diverge, and the divergence changes when Google adds or removes a feature from the SERP — which means your "ranking" can move while your organic position is identical. Whichever you use, use one, and label your charts with which.

POST /v1/serp-diff compares the organic block only, keyed on rank_absolute within those results. That is a deliberate narrowing: it makes the series stable against SERP-feature churn, at the cost of not telling you how far down the page you actually are. If page geography is what you care about, capture the full SERP with /v1/search and compute your own.

Diff, do not average #

The single worst rank-tracking metric is the average position across a keyword set. It hides everything that matters: a keyword falling off the first page entirely and another climbing two spots can leave the average unchanged, and one keyword you do not care about can swing the whole line.

Track movements as a set: what entered, what left, what moved and by how much. That is what /v1/serp-diff returns:

curl -X POST https://api.dataswap.io/v1/serp-diff \
  -H "Authorization: Bearer $DATASWAP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "keyword": "postgres connection pooling",
    "location_name": "London,England,United Kingdom",
    "language_code": "en",
    "device": "desktop",
    "depth": 100
  }'
import { Dataswap } from 'dataswap';

const dataswap = new Dataswap({ apiKey: process.env.DATASWAP_API_KEY });

// No dedicated SDK method for this route yet, so use the escape hatch. `request<T>` types the
// response; declare the fields you actually read.
type SerpDiff = {
  baseline: boolean;
  captured_at: string;
  previous_captured_at: string | null;
  changes: {
    added: { url: string }[];
    removed: { url: string }[];
    moved_up: { url: string }[];
    moved_down: { url: string }[];
    unchanged: number;
  };
};

const diff = await dataswap.request<SerpDiff>('POST', '/v1/serp-diff', {
  keyword: 'postgres connection pooling',
  location_name: 'London,England,United Kingdom',
  language_code: 'en',
  device: 'desktop',
  depth: 100,
});

if (diff.baseline) {
  // First capture for this keyword+config. There is nothing to compare against, so every result
  // is reported under `added` — branch here rather than reading the diff as real movement.
  console.log('baseline stored at', diff.captured_at);
} else {
  const { added, removed, moved_up, moved_down, unchanged } = diff.changes;
  console.log(
    `since ${diff.previous_captured_at}: +${added.length} -${removed.length} ` +
    `↑${moved_up.length} ↓${moved_down.length} =${unchanged}`,
  );
}

The endpoint is stateful: it captures the current ranking and compares it against your previous capture for the same keyword and configuration. The first call returns baseline: true, and because there is no earlier capture to compare against, every result comes back under added — that is not a page of new entrants, it is the baseline being stored. Branch on baseline before you read changes, or your first data point will show a hundred arrivals that never arrived. previous_captured_at tells you what the comparison window actually was, which matters when a scheduled run was missed and today's "daily" diff spans three days.

current carries the full captured ranking, so you can store the raw list alongside the diff.

Cadence: match it to the decision #

Sampling more often does not make the number more true, it makes it more expensive and noisier. Choose by what you would actually do with the alert:

  • Daily is right for most tracking. Enough to see real movement, cheap enough for a large set.
  • Weekly is right for large sets that inform quarterly content decisions.
  • Hourly is right for a handful of keywords during a launch, a migration, or a suspected penalty — situations where you would genuinely act within the hour.

Cost scales with keywords × frequency. The freshness parameter — cached at half price, live at 1.5× — applies to /v1/search, /v1/news and /v1/maps, not to the endpoints in this article: /v1/serp-diff always captures fresh, which is what you want when you are measuring movement. Sending freshness to it returns a 400.

The keyword set: what to track #

Tracking keywords you chose by intuition is how most sets go stale. Two calls fix it.

What you already rank for, which is usually broader than the list you were tracking:

import os
from dataswap import Dataswap

client = Dataswap(api_key=os.environ["DATASWAP_API_KEY"])

ranked = client.request("POST", "/v1/labs/ranked-keywords", {
    "target": "example.com",
    "gl": "us",
    "limit": 200,
})

What those keywords are worth, for up to 100 at a time:

volumes = client.request("POST", "/v1/keywords", {
    "keywords": ["postgres connection pooling", "pgbouncer vs pgpool"],
    "gl": "us",
    "hl": "en",
})

Together: discover what you rank for, size it, track the subset that matters, and revisit quarterly. /v1/labs/keyword-suggestions and /v1/labs/related-keywords extend the set; /v1/labs/search-intent classifies intent for up to a thousand keywords, which is how you avoid tracking informational queries with a commercial page.

Base costs: serp-diff 2 credits, keywords 3, labs/ranked-keywords 4. A credit is $0.002, and the exact charge comes back on every response.

Limits and failure modes #

Search volumes are estimates, and they disagree wildly between tools. This is not a minor discrepancy. Practical Ecommerce reported the same keyword at 880,000 in Google Ads, 590,000 in Semrush and 100,000 in Ahrefs. Their measurement, not ours — and the lesson is structural: volumes come from different models over different panels. Never mix sources in one table, never compare a volume from one tool against a volume from another, and treat volume as an order of magnitude for prioritisation rather than a forecast.

The diff is keyed on URL. If a page changes URL — a redirect, a slug edit, a trailing-slash change, http to https — it appears as one removed and one added, not as a move. Before investigating a "lost ranking", check whether the URL simply changed. Normalising URLs on your side before storing helps; it does not eliminate this.

Only organic results are compared. An AI Overview appearing above you does not register as a change here, even though it may change your traffic substantially. Track that surface separately — see reading AI Overviews programmatically.

A missed run silently widens the window. If your scheduler skips a day, the next diff covers two. Read previous_captured_at rather than assuming your interval held, and label charts with real intervals.

Position is not traffic. Click-through rate varies enormously by query type and by how much of the page sits above you. Rising from 5 to 3 on a SERP that gained an AI Overview and a local pack can lose you clicks. If traffic is the goal, join rankings against Search Console rather than inferring.

Depth costs and truncates. depth: 100 captures more and costs more than depth: 10; a URL below your depth is indistinguishable from a URL that is gone. Keep depth constant across a series, or your "removed" counts are an artefact of your own configuration.

Zero-click is not measured here. Ranking first for a question answered on the results page is a ranking success and a traffic non-event. Rank data cannot see that; only your analytics can.

A tracker that holds up #

  1. Pin location, language, device and depth. Never change them mid-series — start a new series instead.
  2. Store the raw capture, not just derived metrics.
  3. Report movements as sets: entered, left, moved. Never a single average.
  4. Record cached and previous_captured_at with every point.
  5. Refresh the keyword set from what you actually rank for, quarterly.
  6. Treat volume as prioritisation input, from one source, never mixed.
  7. Track AI surfaces separately, because organic rank no longer describes the page.

Do that and the chart means something. The docs cover the credit envelope, freshness and idempotency; the API reference carries the published schemas; and AI search visibility covers the surfaces that classic rank tracking cannot see.

More reading

3 min read

AI crawlers read your HTML, not your JavaScript

Before you measure whether an AI assistant cites you, check whether it can read you at all. The test is one curl command, and the result is often uncomfortable.

  • ai-visibility
  • crawling
  • geo
8 min read

Grounding an LLM agent with search data

Search results are the wrong shape for a prompt. How to turn a SERP into grounding context an agent can use: dedupe, rerank, budget tokens, keep citations.

  • grounding
  • rag
  • llm-agents
  • search-api
  • context-window