An orphan pages checker cannot be a normal crawler alone. A crawler finds pages by following links from its starting URLs; a true orphan page has no crawlable internal path from the pages being crawled. To discover candidates, compare the crawler’s reachable URL set with independent sources that can reveal URLs from outside the internal link graph: XML sitemaps, analytics landing pages, Google Search Console, backlink data, CMS or database exports, and server logs.
The practical workflow is a set comparison: URLs known from other sources minus URLs reached through internal links = orphan candidates. Then validate each candidate. Some will be redirects, non-canonical URLs, old campaign pages, utility URLs, or pages that were intentionally removed from navigation. The parent orphan pages guide covers how to evaluate and fix those outcomes; this page focuses on finding them reliably.
Why a crawler by itself misses orphan pages
Google says crawlable links help it find other pages, but URLs can also be discovered through sitemaps and other sources. That means “not found in my crawl” and “unknown to Google” are different statements.
Suppose a site has 10,000 URLs in its CMS. An HTML crawl from the homepage reaches 8,400. The remaining 1,600 are not automatically orphans. Some might be redirected, blocked, non-indexable, unpublished, API endpoints, or duplicates. The job is to reconcile datasets until each URL has an intended state.
The data sources an orphan-page checker should combine
| Source | What it can reveal | Key limitation |
|---|---|---|
| HTML crawler | URLs reachable through the internal link graph, crawl depth, inlinks | Cannot discover a true orphan unless seeded from another source |
| XML sitemap | Canonical URLs the site wants search engines to know about | Sitemap inclusion does not prove a useful internal path; stale URLs may remain |
| Analytics | Landing pages that users visited, including pages reached from campaigns, bookmarks, or external links | Only pages with tracked visits appear; tracking can be missing |
| Search Console Performance | Pages that received Google Search impressions or clicks | Data is sampled/truncated in some contexts; not a full URL inventory |
| Search Console Page indexing / URL Inspection | URLs Google knows about and page-level index information | Reports answer Google-specific questions, not site architecture by themselves |
| Backlink data | Externally linked URLs that may not be internally linked | No backlink source is perfectly complete |
| Server logs | URLs actually requested by users and bots | Requires access, parsing, bot verification, and an appropriate time window |
| CMS/database export | Published URL inventory and content state | May contain drafts, legacy routes, aliases, or non-canonical records |
Workflow A: use a crawler with external URL-source integrations
Some crawlers can import URLs from sitemaps, analytics, and Search Console, then flag URLs that were discovered externally but had no inlinks in the main crawl. Screaming Frog documents this exact approach in its orphan pages tutorial: combine an internal crawl with XML sitemap, Google Analytics, and Search Console sources, run crawl analysis, and review orphan-URL reports.
Configuration sequence
- Define the canonical host and crawl scope. Decide whether subdomains, parameters, staging areas, language directories, or private sections belong in the analysis.
- Crawl from the normal site entry points. Use homepage/navigation paths rather than seeding every known URL, otherwise you lose the distinction between linked and externally discovered pages.
- Import XML sitemap URLs. Keep source attribution so you know a candidate came from the sitemap.
- Connect analytics landing-page data. Use a window long enough to capture low-frequency but real pages; document the date range.
- Connect Search Console page data. Pull pages with impressions/clicks and preserve the property/date range.
- Optionally import CMS and backlink exports. These can expose additional live URLs.
- Run post-crawl analysis. Separate URLs reached through internal links from URLs discovered only through imported sources.
- Export candidates with source labels. A page found in GSC + sitemap + logs has a different evidence profile from a page found only in an old CMS export.
Workflow B: a manual orphan-page checker in a spreadsheet or database
You can reproduce the logic without a crawler integration. The important part is preserving one canonical URL key and source flags.
1. Build the reachable set
Export every internal HTML URL reached by the crawler from your chosen start points. Include at least:
- Final normalized URL.
- Status code.
- Canonical URL.
- Crawl depth.
- Number of inlinks.
- Indexability.
2. Build the known-URL set
Append URLs from independent sources. Add one Boolean/source column for each:
- In sitemap?
- In analytics?
- In Search Console Performance?
- Known in Page indexing or URL Inspection sample?
- Has external backlink?
- Seen in server logs?
- Published in CMS?
3. Normalize URLs before comparing
False orphan reports often come from URL-format differences. Normalize carefully:
- Preferred protocol and hostname.
- Trailing slash policy.
- Case rules where the platform treats case consistently.
- Tracking parameters that should not define a separate canonical page.
- Known redirect aliases.
Do not strip parameters blindly. On some sites a parameter creates a genuinely different page state or canonical URL.
4. Create the candidate rule
A simple candidate is:
Published/known URL = yes AND reached by internal crawl = no.
Then add confidence tiers:
| Candidate tier | Evidence pattern | Interpretation |
|---|---|---|
| High-confidence orphan | 200 + canonical/self + sitemap or CMS + analytics/GSC/log evidence + zero crawl inlinks | Live page is used or known but has no discovered internal path |
| Medium-confidence orphan | 200 + sitemap/CMS + zero crawl inlinks | Likely orphan; verify page purpose and rendering |
| Legacy candidate | Redirect/404 + old analytics/backlink/log evidence | Not a live orphan page; evaluate redirects and cleanup |
| Intentional unlinked utility | 200 + noindex or private/transactional purpose + zero inlinks | May be correct; do not “fix” for SEO automatically |
| False positive | Page has link in rendered navigation but crawler missed it due scope/rendering/auth | Fix crawl configuration or rendering diagnosis first |
Method 1: find orphan candidates from XML sitemaps
Google’s sitemap guidance says sitemaps should include the canonical URLs you want to appear in search results, while submitting a sitemap is a hint rather than a guarantee of crawling or indexing. This makes the sitemap an excellent comparison source.
- Export all sitemap URLs.
- Compare them to the crawler’s internally reached URLs.
- Filter for sitemap URL = yes and crawler inlinks = 0 / not reached.
- Check status, canonical, indexability, and page purpose.
A sitemap-only URL may be a valuable orphan, but it may also be a stale sitemap entry. Fix the source of truth, not only the report.
Method 2: find orphan candidates from analytics
Analytics can reveal pages users reached from ads, email, bookmarks, direct links, external links, or old search results even when those pages are not linked internally.
- Export landing pages for a documented date range.
- Normalize them to canonical URL format.
- Left-join against the crawl set.
- Investigate landing pages with real activity but no internal crawl path.
Check tracking quality before drawing conclusions. A page with no analytics data is not proof that nobody uses it.
Method 3: use Search Console Performance and Page indexing
The Search Console Performance report can show pages that received impressions or clicks from Google Search. If such a page is missing from the internal crawl, it deserves review because Google and users can reach a URL the site architecture does not expose.
The Page indexing report provides a broader view of URLs Google knows about and their indexing state. Google notes that if it does not know about a URL, that URL will not appear in the indexed or non-indexed totals. For specific URLs, use URL Inspection.
Do not treat Search Console as a perfect URL inventory. Google documents data limitations, and the Links report is also a sample rather than a complete list. Combine sources.
Method 4: use backlinks to uncover externally discoverable pages
A page can lose all internal links but continue to receive visits and crawler discovery through external links. Search Console’s Links report can show externally linked pages, although Google states that the report is not a comprehensive list of every link.
- Export top externally linked pages or use a trusted backlink dataset.
- Compare target URLs with the crawl set.
- Prioritize live 200 pages that still receive relevant external links.
- Decide whether to restore internal links, consolidate to a successor, or redirect if the content is retired.
Method 5: use server logs to find requested but unlinked URLs
Server logs answer a different question: which URLs were actually requested? They can reveal Googlebot requests, old campaign URLs, parameter variants, and externally discovered pages that your internal crawl does not reach.
- Choose a representative log window; include longer windows for low-frequency sections.
- Normalize requested URLs carefully.
- Verify Googlebot or other bots if bot-specific conclusions matter.
- Separate status codes and user agents.
- Compare live 200 URLs with the internal crawl set.
- Review repeated bot or user requests to unlinked pages as high-priority candidates.
Logs do not tell you whether a page should be linked. They prove requests happened. The editorial/technical decision still requires page purpose and business context.
False positives to rule out before fixing anything
JavaScript or crawler configuration hid the link
If a link appears only after rendering, behind interaction, or outside the crawler’s configured scope, the page may not be truly orphaned. Compare rendered HTML and crawl settings. Google’s link guidance says crawlable links are normally anchor elements with an href.
The URL is a redirect or non-canonical alias
An old URL with backlinks may correctly redirect to a current page. Do not add internal links back to the old address. Normalize to the preferred destination and update important external references when practical.
The page is intentionally unlinked
Some confirmation, account, testing, paid-campaign, or utility pages should not sit in public navigation. Check indexability, user need, and privacy/security requirements before “repairing” them.
The page is intentionally retired
A 404 or 410 can be the correct outcome when content has no relevant successor. The task is cleanup: remove it from sitemaps and stale internal references, and decide how to handle external links.
What to do with confirmed orphan pages
| Page state | Recommended action |
|---|---|
| Valuable, current, distinct page | Add useful internal links from relevant hubs, navigation, or contextual source pages |
| Useful but overlaps a stronger page | Merge content and redirect if one page can fully replace the other |
| Outdated with a clear successor | Redirect to the successor and update sitemap/internal references |
| Obsolete with no replacement | Retire with appropriate status and remove from sitemap |
| Utility/non-public page | Keep intentionally unlinked and control indexation/access as required |
| False positive caused by rendering/crawl scope | Fix the diagnostic setup or the crawlable-link implementation |
When adding links, use pages where the relationship helps a reader. The site’s internal linking strategy guide covers crawlable links, useful source pages, and architecture decisions. The Internal Linking Audit Template provides a workbook for tracking URL inventory, orphan candidates, anchors, and fixes.
Validate after remediation
- Re-crawl from normal site entry points without seeding the repaired URLs.
- Confirm the page now has one or more useful crawlable inlinks.
- Verify the source page and destination are both canonical/current URLs.
- Check that sitemap membership matches the page’s intended indexation role.
- Use URL Inspection for important Google-specific cases.
- Monitor Search Console, analytics, and logs over an appropriate period without claiming that the link alone caused later ranking or traffic changes.
Minimal data model for an orphan-page checker
| Field | Purpose |
|---|---|
| URL | Normalized candidate key |
| Status | 200, redirect, 404/410, error |
| Canonical | Preferred URL relationship |
| Indexability | Whether indexation is intended/allowed |
| Internal inlinks | Evidence from crawl graph |
| Crawl depth | Whether/where the crawler reached it |
| Sitemap | Independent discovery source |
| Analytics | User landing-page evidence |
| GSC | Google Search visibility/knowledge evidence |
| Backlinks | External discovery evidence |
| Server logs | Actual request evidence |
| CMS | Publishing/source-of-truth evidence |
| Classification | Link / merge / redirect / retire / intentional / false positive |
| Owner | Team responsible for action |
| Validation | How the final state will be checked |
The key rule
Do not ask one tool to “find all orphan pages.” Build two sets: what the internal graph can reach and what the organization, users, search engines, or external sites know exists. The gap between those sets is where orphan investigation starts.
Sources and verification
- Google Search Central: Link best practices
- Google Search Central: Build and submit a sitemap
- Google Search Console: Performance report
- Google Search Console: Page indexing report
- Google Search Console: Links report
- Screaming Frog: How to find orphan pages
How this page was prepared
Reviewed by SEO Strategy Editorial Team. Claims, terminology, and time-sensitive details were checked against the sources listed below and the page was last updated September 3, 2026.
AI-assisted tools supported research organization or drafting; editorial review remained responsible for source selection and the published conclusions.
Download "Internal Linking Audit Template" and adapt it to your project.