How Do 404 Errors Work? Causes, SEO Impact and Fixes

What a 404 error means and how it works

The web operates through requests and responses. A browser, search crawler or application requests a URL. The server processes that request and responds with headers, a numerical HTTP status code and, in many cases, a response body. The HTTP specification in RFC 9110 defines 404 Not Found as a client error response indicating that the origin server did not find a current representation for the target resource, or is unwilling to disclose that one exists.

The sequence is usually straightforward:

1. A user or bot requests a URL such as /products/blue-widget.
2. Domain name and network systems route the request to the appropriate server or content delivery network.
3. The application, web server or edge rule attempts to match the path to a file, database record or route.
4. No matching resource is found, so the server returns HTTP 404.
5. The client displays the response body, which may be a plain error message or a designed custom page.

The status code and displayed content are separate. A helpful branded error page can still be a technically correct 404. Conversely, a page can say “not found” while returning HTTP 200 OK. That mismatch is commonly called a soft 404.

404, soft 404, 410 and redirects compared

The correct response depends on what happened to the resource and whether an equivalent destination exists. The following matrix provides a practical decision rule.

A 404 does not specify whether the condition is temporary or permanent. MDN identifies 410 Gone as the more specific choice when permanent removal is known. In practice, both 404 and 410 communicate that a URL should not remain indexed when Google repeatedly confirms the response.

Why 404 errors occur

Some 404s reflect ordinary web activity. People mistype addresses, external sites link incorrectly and automated crawlers invent URL combinations. Others expose defects in publishing, migration or application logic.

• Deleted content: A product, article, profile, location or media file was removed without a replacement plan.
• Changed URLs: Slugs, folders, domains, protocols or platform routes changed without complete redirects.
• Broken internal links: Navigation, body copy, canonicals, hreflang annotations or structured feeds reference an obsolete path.
• Case and character differences: A server treats uppercase and lowercase paths differently, or encoded characters resolve incorrectly.
• Deployment defects: Files, routes or database records are absent from the production environment.
• JavaScript routing failures: A single page application displays a not found state without sending the corresponding HTTP response.
• Invalid dynamic URLs: Filters, tracking parameters, calendar paths or faceted navigation generate combinations with no content.
• CDN or cache inconsistency: Edge rules, stale caches or origin configuration produce different responses for the same URL.

Broken links are widespread rather than exceptional. A 2024 study examining 88,000 homepages from the Majestic Million found that 35.2 percent had at least one broken link. That is prevalence evidence from a defined dataset, not a universal benchmark for every website.

How 404 errors affect SEO and AI search visibility

A 404 response is not a sitewide penalty. If a URL has no useful content and no replacement, returning 404 is correct maintenance. Google can remove that URL from its index after discovering the response. Other valid pages can continue ranking.

The SEO cost comes from context. A deleted page may have rankings, referral visits, editorial backlinks or strong internal links. Removing it without restoring or redirecting the resource can eliminate that page’s search visibility and strand external link value. Active internal 404s also waste user attention and make site architecture less reliable. Sitemap URLs returning 404 send contradictory discovery signals because a sitemap should list canonical, indexable URLs that the site wants crawled.

Soft 404s are more troublesome at scale. Google’s crawl budget documentation, updated July 22, 2026, describes 404 as a strong signal not to recrawl, while soft 404 responses may continue consuming crawling resources. A 2025 Web Science redirection study identified 62,000 custom 404 URIs in its dataset and classified nearly half as soft 404s, illustrating how frequently presentation and protocol signals can diverge.

For Google AI Overviews and AI Mode, the foundational requirements remain those of Search. A supporting page must be indexed and eligible to appear with a snippet. Bing, Copilot and ChatGPT also depend on accessible, discoverable source material through their respective search and retrieval systems. A removed URL cannot remain a dependable citation target. Preserve durable URLs for original research, statistics and expert resources whenever possible, and redirect them only when a true successor exists.

The 404 decision framework: restore, redirect, remove or repair

Do not make the decision from the status code alone. Evaluate the missing URL’s intended content, replacement relevance, demand and link value.

1. Confirm the response. Test the exact URL and inspect the HTTP status, redirect path, rendered content and canonical signals.
2. Identify the intended resource. Determine whether the URL was valid, mistyped, generated automatically or removed deliberately.
3. Find demand signals. Check organic landing page history, conversions, backlinks, referral traffic, internal inlinks and search impressions.
4. Assess replacement equivalence. A successor should satisfy substantially the same intent, not merely belong to the same website.
5. Choose an action. Restore valuable content when it should still exist. Apply a direct 301 when an equivalent permanent successor exists. Use 302 for a genuine temporary move. Return 404 or 410 when no appropriate resource exists.
6. Repair references. Update internal links, navigation, sitemap entries, canonicals, hreflang and feeds rather than relying indefinitely on redirects.
7. Validate after release. Recrawl the affected URLs and monitor logs, indexation, traffic and conversion outcomes.

For example, a discontinued shoe model might redirect to its direct replacement if specifications and user intent remain comparable. Redirecting it to a generic footwear homepage is less defensible. Google’s site move guidance warns that mass redirects to an irrelevant destination can confuse users and may be treated as soft 404s.

How to find and diagnose 404 errors

Use several data sources because each reveals a different part of the problem. A crawler finds reachable internal links, but it cannot identify every orphan URL, historical backlink or request made by users and bots.

Recommended diagnostic sequence

1. Crawl the website. Filter for 4xx responses and export each missing URL with its source pages, link location, anchor text and crawl depth. Screaming Frog documents this workflow in its SEO Spider tutorials.
2. Inspect Search Console. Review not found and soft 404 classifications, then test representative URLs. Treat the reports as discovery data, not a command to redirect every listed address.
3. Analyze server or CDN logs. Count requests by URL, bot, referrer and date. Logs expose orphan 404s and repeated crawler demand that a link crawl may miss.
4. Check analytics. Find 404 templates receiving sessions and determine their previous page, landing URL, device and conversion impact.
5. Audit XML sitemaps. Every submitted URL should resolve directly to the intended canonical status, normally 200, without a redirect chain.
6. Intersect backlink data. Prioritize missing URLs with authoritative external links, referral visits or unlinked references that can be corrected through outreach.
7. Test rendering and variants. Compare desktop, mobile, JavaScript enabled, JavaScript disabled, uppercase, trailing slash and parameter versions where relevant.

For a small site, a manual crawl and Search Console review may suffice. Large commerce, marketplace and publishing sites usually need scheduled crawls, log processing, analytics events and route level dashboards. When selecting software or an agency, require source page exports, redirect chain detection, rendered JavaScript testing, log integration and before and after validation. A raw count of 404s is not an adequate technical audit.

How to implement fixes without creating new problems

Start with high impact defects: broken navigation, paid campaign destinations, converting landing pages, URLs with strong backlinks, recently migrated content and missing URLs in XML sitemaps. Repair the cause before clearing reports.

For a 301 redirect, send the request directly to the final equivalent URL. Avoid chains such as old URL to interim URL to final URL, and eliminate loops. Update internal references to the final destination. Redirect maps should be tested for status, destination relevance, case handling, query strings and fragment behavior before a migration launches.

When no replacement exists, serve a true 404 or 410 and remove the URL from sitemaps and internal links. Do not use 404 or 403 as a rate limiting mechanism. Google notes that most 4xx responses, except 429, can lead to content removal from Search. Temporary server problems require an appropriate 5xx response, while excessive requests should use 429.

Single page applications need special care. A client-rendered error view may initially return 200 because the server serves the same application shell for every route. Google’s JavaScript SEO guidance recommends meaningful status handling. Solutions include server-side rendering, edge routing, dynamic rendering architecture that is not deceptive, or redirecting an invalid client route to a server endpoint that returns 404. Test the response seen before and after rendering.

What a useful custom 404 page should contain

A custom 404 page cannot recover every lost visit, but it can help a person continue their task. It must preserve the true 404 status while providing concise, visible recovery options.

• A plain explanation that the requested page could not be found
• A link to the homepage and major task-oriented sections
• Site search when the search function produces reliable results
• Links to popular products, resources or support pathways
• Consistent navigation and accessible design
• A way to report a broken link when user reports can be processed

Avoid automatically redirecting visitors after a timer. Do not overload the page with every category, and do not disguise the error as a normal article. Track the template through analytics without changing its status code. Useful measurements include exits, search usage, recovery clicks and sessions that later convert.

Also prevent the 404 template itself from producing broken asset requests. Missing style sheets, scripts, fonts or images can make the recovery page unusable and create additional log noise.

Prioritization, monitoring and advanced content operations

A useful 404 program measures business and discovery impact rather than chasing a zero count. Bots will always request malformed and obsolete URLs. Segment known valid pages from random requests and prioritize controllable problems.

Track the number of active internal links to 404s, 404 requests with human sessions, crawler requests to soft 404s, sitemap error rate, redirect chain rate, recovery click rate and time to resolve critical defects. Use seven day and thirty day trends rather than a one-time total.

Integrate this work with content consolidation and decay remediation. Before deleting an outdated article, determine whether it supports a topic cluster, attracts links or answers a query that a stronger hub page can absorb. Update internal hub-and-spoke links after consolidation. Durable research, statistics and comparison assets deserve stable URLs because they can generate natural link demand and become citation sources. During digital PR or unlinked mention outreach, ask publishers to update broken destinations only when a live, genuinely equivalent resource exists.

Riskier tactics include redirecting expired domains, large batches of unrelated URLs or every 404 to a commercial page. Any short term benefit is outweighed by relevance loss, poor user experience and the possibility that search systems will disregard the redirects. Do not use cloaking, deceptive redirects or structured data that conflicts with visible content.

What is proven, what practitioners agree on and what remains uncertain

Proven by standards or official documentation

HTTP 404 means the origin server cannot find a current representation of the requested resource, or will not disclose that one exists. It does not establish permanence. A 410 is more explicit when permanent removal is known. Google recommends a permanent redirect for a permanent move, a temporary redirect for a temporary move, and 404 or 410 when no relevant replacement exists.

Strong practitioner consensus

Technical SEO practitioners generally prioritize missing URLs that have internal links, backlinks, traffic, conversions or sitemap inclusion. Ahrefs recommends redirecting only to a genuinely relevant replacement, while crawler workflows from Screaming Frog emphasize exporting source pages and inlinks. Current Reddit discussions similarly distinguish harmless historical 404s from active internal failures and valuable missing pages. Those community reports are anecdotal, not evidence of a ranking mechanism.

Still contextual or uncertain

There is no universal threshold at which a raw number of 404s harms an entire site’s rankings. Crawl effects depend on site scale, update frequency, server health, linking and the proportion of wasteful URLs. Search engines also do not publish every timing rule for recrawling, deindexation or redirect signal processing. Teams should therefore base decisions on URL value, direct observations and trend data rather than promises that every removed URL must be redirected.