A sitemap can be technically valid and still give you a very incomplete picture of a website. I learned this properly while auditing a large travel marketplace. The sitemap files loaded. The XML was valid. Nothing looked dramatically broken.
But when I compared the sitemap inventory with the pages the business actually had, more than 40,000 pages were missing.
They included useful pages for tours, attractions, destinations, and guide led experiences. The problem was not that someone had intentionally excluded forty thousand pages. The problem was that the sitemap generation rules no longer represented the website the company had become.
That experience changed the first question I ask during a sitemap audit. I no longer begin with, “Is this sitemap valid?” I begin with, “Does this sitemap represent the business?”
The company mistake was completely understandable
Most companies do not ignore their sitemaps. They set them up, submit them to Google Search Console, see a successful status, and move on to work that feels more urgent.
Meanwhile, the website changes. New templates appear. New markets launch. Old systems and new systems run beside each other. A content type gets added without being added to the sitemap rules. Nobody receives an alert because the sitemap itself still works.
This is why “valid” can create false confidence. Validation checks whether a file follows the required format. It does not check whether the file contains everything the business expects search engines to discover.
I stopped treating sitemap audits as XML checks
My process now starts outside the sitemap.
I build or request a view of the real canonical, indexable page inventory. I separate it by template, market, language, and business purpose. Only then do I compare that inventory with the sitemap files.
The most useful gaps are rarely random URLs. They usually form a pattern:
- One template is missing across every market.
- A newer section never entered the original generation rules.
- A language version is included inconsistently.
- Canonical pages exist but only parameter variations appear in the sitemap.
- Valuable pages are present in the sitemap but have almost no internal links.
Patterns make the problem easier to explain and easier to fix. “Forty thousand URLs are missing” sounds frightening. “This template is absent from the sitemap generator” gives engineering a place to start.
Discovery was only the first part of the problem
Adding a missing page to a sitemap does not automatically make it valuable or indexable. It simply gives the page a clearer route to discovery.
After finding the coverage gap, I still needed to ask whether those pages were canonical, internally linked, sufficiently distinct, useful to travellers, and supported by the right entity information. Some needed better descriptions. Some needed stronger links from destination and attraction pages. Some needed technical fixes before they belonged in a sitemap at all.
That distinction matters. The goal is not to make the sitemap as large as possible. The goal is to make it accurate.
Why I built a Sitemap Auditor
The app came after the recurring problem. I wanted a faster way to compare crawled pages, sitemap entries, and URL taxonomy without manually rebuilding the same spreadsheet each time.
The Sitemap Auditor is based on the questions I now use in real audits: What exists? What is declared? What is missing? Which template or section does each gap belong to? Which findings actually deserve action?
It does not replace technical judgment. It makes the mismatch visible enough for that judgment to be used.
What I would check on Monday
- Export the real canonical page inventory independently of the sitemap.
- Group the pages by template, market, and content type.
- Compare each group with the current sitemap files.
- Check whether important sitemap pages also receive internal links.
- Confirm that exclusions are deliberate rather than inherited accidents.
- Give each gap an owner and a reason, not only a URL count.
Questions about sitemap pages missing
Can a sitemap be valid while thousands of pages are missing?
Yes. XML validation checks formatting and protocol requirements. It does not know how many canonical, indexable pages your business intended to include.
Should every website URL appear in a sitemap?
No. A sitemap should contain canonical URLs that you genuinely want search engines to discover and index. Parameters, duplicates, utility pages, and low value combinations may need exclusion.
Will adding missing pages guarantee indexation?
No. It improves discovery, but Google still evaluates page quality, duplication, internal linking, canonical signals, and usefulness before indexing a page.
Why compare by template instead of checking URLs one by one?
Template patterns reveal the underlying rule that created the gap. Fixing that rule is more reliable than repairing thousands of URLs individually.
