DeadLinkPatrol › Blog

Broken Link Checking for Documentation Sites: Build-Time Checks Plus Live Monitoring

· 7 min read

Documentation sites are hard on links. They're full of cross-references between pages, anchors to specific sections, links to API references, code repositories, changelogs and third-party docs, and all of those move. Pages get renamed when the product changes, versions get archived, and the external projects you link to restructure their own docs. A reader who follows a "see Configuration" link and lands on a 404 in the middle of a task is a frustrated reader, and often becomes a support ticket.

The good news is that docs teams have better tools for this than most websites, because docs are usually built from source files. The catch is that build-time checks only catch some problems. This guide covers both halves: what to check during the build, and what you can only catch by checking the live site.

It helps to separate them, because different tools catch each kind.

  1. Internal page links. A link from one doc page to another that doesn't exist (renamed file, deleted page, typo in a relative path). Your static site generator can catch these at build time, because it knows every page that exists.
  2. Anchor links. A link to /config#retry-options when the heading is now "Retry settings", so the anchor is #retry-settings. The page loads, so nothing returns a 404, but the reader lands at the top of a long page instead of the section they wanted. Some generators can check these. Many link checkers ignore them.
  3. External links. Links to GitHub, package registries, other projects' docs, blog posts and standards. Your build can't know whether these work, so you need a checker that makes real HTTP requests, and those links can break at any time after you deploy.

There's a fourth, sneakier kind: links that work in the build but break in production. Server-level redirects, a base path that's different in production, a CDN rule, or a versioned-docs setup where old versions point at pages that only exist in the latest version. Only a check against the live site finds these.

Layer 1: Turn on your generator's own checks

Start here. It's free and takes a few minutes. Exact option names vary between versions, so check your generator's docs for the version you run.

Docusaurus

Docusaurus checks internal links when you run a production build, and the onBrokenLinks option in docusaurus.config.js controls what happens when it finds one. In current versions the default is 'throw', which fails the build. Keep it that way. There are related options for broken Markdown file links and for broken anchors (onBrokenAnchors, which defaults to a warning in the versions that have it). If you can, set anchors to throw as well, so renamed headings fail the build instead of quietly sending readers to the top of the page.

Note that these checks happen during docusaurus build, not in the dev server, so make sure CI runs a real build on every pull request.

MkDocs

Run mkdocs build --strict in CI. Strict mode turns warnings, including warnings about links to documentation files that don't exist, into a failed build. Recent MkDocs versions also have a validation section in mkdocs.yml that controls how strictly different link problems are treated, including anchors. MkDocs validates links between your Markdown files. It doesn't check external URLs.

Sphinx

Sphinx warns about broken cross-references during a normal build (-W turns warnings into errors, and -n "nitpicky" mode reports missing references more aggressively). For external URLs it ships a dedicated builder: sphinx-build -b linkcheck (or make linkcheck) requests every external link and writes a report. It's slow on large sites, so most teams run it on a schedule rather than on every commit.

Hugo

Hugo's ref and relref shortcodes fail or warn (depending on refLinksErrorLevel) when they point at a page that doesn't exist. Plain Markdown links aren't checked, so teams that care use ref/relref for internal links and an external checker for everything else.

Everything else (GitBook, Mintlify, ReadMe, custom setups)

Hosted docs platforms vary a lot. Some flag broken internal links in their editor, some don't, and few check external links on a schedule. If yours doesn't, you'll rely more on Layers 2 and 3.

Build checks handle internal links. For external links, add a dedicated link checker to your CI pipeline. Common choices:

A few lessons that save a lot of frustration:

Don't block pull requests on external links. An external site going down for ten minutes shouldn't stop someone merging a typo fix. Block PRs on internal link failures (Layer 1). Run external checks on a schedule or as a non-blocking job, and open an issue when something fails.

Expect bot-blocking. Some sites answer automated requests with 403, 429 or other non-200 codes while working fine in a browser. Social networks and some large platforms are known for it. Keep an exclusion list for known false positives, and review it every few months, because "excluded" can quietly turn into "never checked."

Retry before failing. Use your checker's retry settings, or have it check twice before reporting. A single timeout isn't a broken link.

Use caching on a schedule. Checking every external link in a large docs repo on every run is slow and can get you rate-limited. Caching results for a day or so is a reasonable compromise.

Layer 3: Monitor the live site

Layers 1 and 2 check your source. Your readers use the deployed site, and the two differ more than you'd think:

A scheduled crawl of the live site catches all of these. It works the way a reader does: it starts at your docs homepage, follows links, and records every link and image that returns an error, along with the page it was found on. Weekly is enough for most docs sites. If you deploy docs many times a day, run it daily and alert only on new breakage, so the alerts don't become noise.

DeadLinkPatrol does this kind of crawl. It checks every internal and external link on a schedule, records the page each broken link was found on, and emails you (or posts a signed webhook to Slack or your issue tracker) only when something new breaks. Every paid account also gets an API key for adding monitors and pulling results into your own tooling. Plans start at $9/mo for 3 sites with weekly scans, with daily scans on higher plans (see pricing).

Docs-specific habits that prevent breakage

Redirect when you rename. Most docs platforms and hosts support redirect maps. Make "add a redirect" part of the checklist for renaming or moving any page, the same way you'd update the sidebar. Old docs URLs get bookmarked, linked from forum answers and Stack Overflow, and cited in other projects' docs, so they need to keep working.

Link to stable URLs. For external references, prefer permalinks: tagged release URLs instead of main branch file links (which break when files move), versioned API docs instead of "latest" where the content matters, and DOIs or archived copies for papers and specs.

Link to headings deliberately. If your generator lets you set explicit heading IDs, use them for sections that get linked often. Then changing the heading text doesn't change the anchor.

Keep an eye on archived versions. Versioned docs multiply your link count. Decide how far back you'll maintain links. For versions you've frozen, you can exclude external links from checks and add a banner pointing readers to the current version.

Give link reports an owner. Route scheduled link failures to a real place, such as a docs issue label, a Slack channel or a weekly triage, with a named owner. A report nobody owns gets ignored.

A setup that covers all three layers

  1. Every pull request: production build with the generator's broken-link checks set to fail (internal pages, plus anchors if supported).
  2. Nightly or weekly in CI: lychee or similar against the source for external links, non-blocking, opening an issue when something fails.
  3. Weekly (daily if you deploy often): a crawl of the live docs site, alerting only on new breakage, with the "found on" page included.
  4. Always: redirects for moved pages, stable external URLs, and explicit IDs for frequently linked headings.

Want to see what a live-site crawl turns up on your docs today? Run the free DeadLinkPatrol checker on your docs URL. It scans up to 25 pages with no signup and lists each broken link with the page it's on.

Check your own site for broken links

Run a free scan of up to 25 pages, no signup. Or let DeadLinkPatrol watch your sites on a schedule and email you when a link breaks.

Monitoring: Starter $9/mo (3 sites, weekly) · Agency $79/mo (50 sites, daily, white-label reports). See plans

More from the blog

More free tools from Swift Labs