SerpCrest
New

Site Crawl & Internal Links

Crawl a site to see how its pages connect: click depth, orphan pages nothing links to, broken internal links and duplicate titles, with AI link suggestions.

Site Audit checks one page. This follows every internal link across the whole site and shows how the pages connect: what is buried too deep, what nothing links to, and what is broken.

150
Pages per crawl
13
Site-wide checks
Free
No API credits
AI
Link suggestions
Site crawlerFree crawl
Pages
up to 40

How it works

1

We start at your homepage

Every internal link is followed, page by page, exactly like a search engine. Your robots.txt is obeyed: pages you disallow are listed, never fetched.

2

Then we read your sitemap

Pages listed in the sitemap but never linked from anywhere are crawled too. Those are your orphan pages.

3

We map how the pages connect

Click depth from the homepage, how many internal links point at each page, broken links, redirect chains, duplicate titles and pages with no way out.

4

AI reads the structure

The AI sees only the page list (URL, depth, link counts, title), not your content. It names the fixes worth doing first and suggests internal links to add, using only pages that exist.

Questions

Up to 40 pages for visitors and 150 when you are signed in, and it stops after about 75 seconds either way. Big sites get their most important pages, since the crawl follows links outward from the homepage.

A page no other page on your site links to. Visitors cannot find it by browsing, and search engines struggle to reach it, so it often gets ignored even when it is in your sitemap.

Pages more than three clicks from the homepage get crawled less often and pass less link value. Moving an important page closer usually helps it.

Yes. Disallowed pages are reported so you can see what is blocked, but we never download them. The crawler identifies itself as SerpCrest-SiteCrawl.

No. It only receives the list of URLs with their depth, link counts, word counts and titles. Your page text never leaves your site.

No. Site Audit checks one page in depth, with speed scores and keyword positions. This looks at how your whole site fits together. Use both.

Search engines find and weigh pages through links, so a page reached only through the sitemap gets less attention. Our guide to internal linking covers the fix. For a full technical check with a fix plan, run the site audit.

Related tools

All 31 tools →

Run this on every link you own.

SerpCrest re-checks the placements in your projects every week and tells you when one breaks or disappears.