Skip to content

How to check on-page SEO when the reader and the crawler disagree

The tab shows what a person sees. A fetch shows what a crawler gets, and only one of them decides indexing.

2 min read

Checking on-page SEO in a browser tab is the habit most content teams already have, and it works well for one question: does this page look right to a reader. It works less well for a different question: will a search engine be able to use it. Keeping those two questions separate is most of the skill.

The reader's view

Open the page, look at the heading, the title in the tab, the description in the source, the images and the internal links. Someone with a little practice spots most problems in a minute.

That pass is worth doing before publishing, and it does not need an extension at all. A browser's own developer tools show the title, the meta tags and the heading structure.

The crawler's view

The crawler's view differs in three ways that a browser tab hides.

Response headers. X-Robots-Tag: noindex and canonical HTTP headers never appear in the rendered page. A page can look perfect and be excluded at the server.

What the server sent. A template that injects the meta description with JavaScript may leave the crawler with none. Google's canonical guidance is explicit that the source is the reliable place.

What is missing after hydration fails. If a script errors, a person sees a broken page and a crawler sees whatever arrived first. Those can be very different documents.

A cheap way to see the difference

Fetch the page as a crawler would, with a plain HTTP client, and search the result for the title and description. If the text you expected is absent, the page depends on JavaScript for something a crawler may not run.

curl -s https://example.com/page | grep -i "<title"

That one line answers a question no extension can.

Where a service fits

When the check has to be repeated or recorded, a URL-based checker is the right tool. seogeoaeo.ai's on-page checker takes a URL and reports the signals it found, including the ones that live in the response, and it can be called from a script.

For a content team, that means one sample per template before a launch, with the output saved next to the release notes. A month later, the record answers "did we check this" without anyone relying on memory.

What to check for each page type

Page type The signal that matters most
Article One clear headline and an accurate date
Product Price and availability in the source
Category Canonical that does not hide the listing
Landing page A description that is specific to the offer

Each is a template concern, which is why checking one page per template beats checking ten pages from the same one.

FAQ

Do I need an extension at all?

No. Developer tools show the same markup, and a URL fetch shows more. An extension mainly saves clicks.

Why does the page look fine but rank poorly?

Rendering and indexing are different steps. A page can render perfectly and still be excluded by a header or a robots rule.

How often should a content team check?

At launch for each template, and after any theme or plugin update. Between those, spot checks are enough.

The habit worth building

Check the rendered page before you publish, and check the fetched page before you call it done. The second takes one command and prevents the most common kind of silent failure.

FAQ

Do I need an extension at all?

No. Developer tools show the same markup, and a URL fetch shows more. An extension mainly saves clicks.

Why does the page look fine but rank poorly?

Rendering and indexing are different steps. A page can render perfectly and still be excluded by a header or a robots rule.

How often should a content team check?

At launch for each template, and after any theme or plugin update. Between those, spot checks are enough.

Originally published on claudegpt.si