Indexing: sitemap watching and IndexNow

How sitemap checks spot new, changed and removed pages, how to verify your IndexNow key and what each response means.

Lesson
7/16
Reading time
5 min
Sections
5
Through the course
44%

Indexing reads your sitemaps on a schedule and records every page with its last-modified date. Pages without one get a fingerprint of their visible text, so an edit still counts as a change.

Set it up

  • Switch Indexing on for the website. Your sitemaps are found from robots.txt or the usual locations, or you add them yourself in Settings.
  • Download the key file from Settings and upload it to the root of your site, so it opens at your domain followed by the key and .txt.
  • Press Verify key. Nothing is sent to search engines until the key checks out.

What gets sent

Your opening check is the baseline and isn't sent. After that, pages that appear in the sitemap or change are marked pending and go out in batches through IndexNow, the free protocol participating search engines share. You can also tick any pages in the table and send them straight away.

Check your understanding

A batch comes back with a 403 response. What does it mean?

Reading responses

  • 200 or 202: accepted. 202 means the key is still being checked on their side.
  • 400 or 422: rejected, usually because a page isn't on the verified host.
  • 403: the key wasn't valid; sending pauses until you verify it again.
  • 429: too many requests; the pages stay pending and are retried later.

Never-indexed pages

Connect Search Console in Integrations and link the website to its property. Pages that sit in your sitemap for a set number of days (14 by default), that Search Console reports as not indexed and that had no impressions in 28 days are flagged as never indexed. The Index column shows each page's status and the reason.

Lesson 7 of 16

Indexing: sitemap watching and IndexNow