Detecting and fixing broken links (404s) on a website with n8n and AI

A broken link doesn't break anything visible: the page keeps rendering fine for everyone except the one person who clicks that specific link and lands on a 404. That's exactly what makes it dangerous — nobody notices until a visitor, or worse, Google's crawler, stumbles on it. This guide builds an n8n pipeline that walks a site's XML sitemap, checks every URL without stopping the workflow at the first error code, then has an LLM suggest the best redirect target — before a weekly report a human validates, never an automatic fix in production.

A measured phenomenon, not a copywriter's worry

The idea that links "rot" over time isn't unique to the general web — it has been studied in depth in a context where citation accuracy matters especially: law and scientific research. Zittrain, Albert and Lessig, in their study Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations (2014), found that more than 70% of the links cited in major US law reviews, and nearly half of those cited in Supreme Court decisions, no longer point to the original content. Klein et al., in Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot (PLOS ONE, 2014), found the same mechanism across more than a million links extracted from scientific articles: one in five articles loses access to the web context it cites, a share that climbs to seven in ten among articles that actually cite web resources. A blog that cross-links its articles as heavily as this one does isn't immune to that mechanic: every slug rename, every removed product or outdated post potentially leaves a trail of broken links somewhere else on the site.

What the workflow needs to do

Four steps: list every URL on the site from the sitemap, check its HTTP response code without crashing the workflow on the first 404 it hits, have an LLM suggest a replacement page among the still-valid URLs, then deliver a weekly report — never touching the site's actual configuration automatically.

Step 1 — Pull every URL from the XML sitemap

A Schedule Trigger fires, once a week, an HTTP Request node in GET on https://yourdomain.com/sitemap.xml. The XML response goes through the XML node to become usable JSON, then a Split Out node turns each <url><loc> into an individual item — the same mechanic covered in our Split Out and Aggregate guide. At this point you have one n8n item per URL published on the site: the reference list of "live" pages, used both to check links and to suggest redirects later on.

Step 2 — Check every URL without stopping the workflow

This is the classic trap: by default, n8n's HTTP Request node treats any response outside the 2xx range as an error and halts execution — the exact opposite of what's needed here, where a 404 is data to collect, not an incident. Two settings under the node's Options change that behavior:

• Never Error: the request is treated as successful regardless of the returned HTTP code.
• Include Response Headers and Status: the status code becomes available in the output (statusCode), instead of simply being swallowed.

This second HTTP Request node runs in HEAD mode (lighter than a full GET, since only the status code matters here) inside a Loop Over Items, a few URLs per batch with a short delay between batches — so as not to hammer the checked server with hundreds of simultaneous requests. Our HTTP Request guide: retry, timeout and reliable API calls covers the tolerance settings for network slowness, useful here since a simple timeout shouldn't be mistaken for a genuinely broken link. An IF node downstream routes URLs by code range: statusCode >= 400 goes to the diagnostic branch, everything else is simply logged for the record.

Step 3 — Let the AI suggest the best redirect

Once a broken URL is identified, the question isn't just "is it dead?" but "redirect it to what?". An AI Agent node receives the broken URL, the anchor text or context of the page that linked to it, and the list of still-valid URLs collected in step 1. Its job: identify the closest semantic match among live pages — a renamed article, a product page replaced by a new reference — rather than a generic redirect to the homepage, which dilutes both the user experience and the inbound link's SEO value.

Force a structured output with the Structured Output Parser: { "brokenUrl": "...", "suggestedTarget": "...", "confidence": 0.8, "reason": "..." }. Below a confidence threshold (0.6, say), the suggestedTarget field stays empty rather than forcing an unreliable guess — a report that says "no clear match" beats a redirect suggested at random.

Step 4 — Report, never fix automatically

The workflow deliberately stops short of pushing the redirect to production. The weekly report — sent by email or to a dedicated Slack channel — lists every broken URL, the source text that linked to it, the AI's suggested redirect, and its confidence score. It's the exact same safeguard principle covered in our guide to human approval with the Wait node and Slack: the AI proposes, a human validates before anything changes in the site's actual configuration.

This weekly cadence isn't arbitrary: the studies cited above show that link rot is an ongoing process, not a one-off event an annual audit could cover. A recurring check, even a lightweight one, catches the problem while it's still a handful of pages rather than a backlog of hundreds of links. The topic is related, but distinct, from what we cover in our guide to content decay and SEO traffic decline: there, a page loses traffic without disappearing; here, it has simply ceased to exist somewhere in your own internal linking — and our weekly Search Console SEO report guide naturally complements this check by watching the impact on rankings and clicks instead.

Going further

This workflow reuses the same logging mechanics — every decision traced, every action requiring human validation — covered in the Compliance & Audit Pack (€149), directly reusable here to keep a history of fixed links over time. If your immediate priority is an overflowing inbox rather than link maintenance, the Inbox AI Pack (€79) applies the same triage-and-alert logic to your emails; and the Full FlowKit Bundle (€269 instead of €347) brings all three packs together on a shared foundation if you plan to chain several automations like this one.