What Are 404 Errors? Complete Guide

What a 404 error actually means

404 Not Found is a client error response in the HTTP 4xx class. Under RFC 9110, it means the origin server did not find a current representation for the target resource, or is not willing to disclose that one exists. It does not prove that the resource was never available.

The code also does not specify whether the condition is temporary or permanent. If the publisher knows that a resource has been removed permanently and no replacement exists, 410 Gone can communicate that distinction. In ordinary publishing systems, however, 404 remains the standard response for unavailable URLs.

A 404 is not the same as a server outage. A 404 indicates that the server responded but could not supply the requested resource. A 500-class response indicates a server-side failure. DNS failures and timeouts occur before a valid HTTP response is delivered.

Why 404 errors happen

Most 404s come from routine website change rather than a catastrophic technical failure. Pages are deleted, products expire, URL structures change, editorial slugs are rewritten, or links contain typing and encoding errors. External sites can also link to malformed or obsolete URLs that the destination site never controlled.

Common causes

• A page was removed without an appropriate redirect.
• A page moved to a new URL, but internal links still point to the old address.
• A visitor or publisher mistyped the path.
• A migration changed folders, parameters, domains, protocols, or trailing-slash rules.
• A content management system generated invalid category, pagination, filter, or media URLs.
• A single-page application displayed a not-found screen without setting the correct status.
• Uppercase rules, URL encoding, locale handling, or routing logic produced an unmatched path.
• A linked external resource was deleted, creating link rot.

Broken links are common across the web. A 2024 study examining 88,000 homepages from the Majestic Million found that 35.2 percent had at least one broken link. That is useful prevalence evidence, but it should not be treated as a universal benchmark for every site or for conditions in 2026.

404, soft 404, 410, 301, or 302: the decision matrix

The correct treatment depends on what happened to the resource and whether an equivalent destination exists. The objective is not to force every old URL to return 200. It is to provide an honest response that helps users and search systems understand the change.

A relevant category page may occasionally be the best replacement, but only when it fulfills substantially the same intent. Redirecting an article about a discontinued feature to a generic blog archive is normally less useful than returning an honest 404 with navigation options.

How 404 errors affect SEO, crawling, and AI search

Legitimate 404s are a normal part of the web and are not an automatic sitewide ranking penalty. Google can remove a URL returning a 4xx response from its index, but that does not mean unrelated pages lose rankings. The practical damage occurs when a valuable page vanishes, internal links lead users into dead ends, or external authority points to an unresolved URL.

Google’s crawl-budget guidance says a 404 is a strong signal not to recrawl a URL. Soft 404s are less efficient because crawlers may continue requesting pages that appear successful at the protocol level but contain no useful resource. This becomes more consequential on large ecommerce, marketplace, news, and faceted-navigation sites.

For Google AI Overviews and AI Mode, supporting pages must satisfy the same foundational technical requirements as Search, including index eligibility and snippet eligibility. A removed or soft-404 page is therefore a weak retrieval candidate. Similar practical logic applies to Bing, Copilot, ChatGPT, and other systems that depend on accessible indexed pages or crawlable source documents. A 404 does not poison an entire brand, but it removes that URL as a dependable answer source.

Do not use 404 or 403 responses as makeshift rate limiting. Google specifically distinguishes these from 429 Too Many Requests. Incorrect rate-limiting responses can cause content to be removed from search rather than simply slowing requests.

A diagnostic workflow for finding and verifying 404s

No single report contains every important 404. Search Console shows URLs encountered by Google, a crawler exposes internal link paths, analytics reveals user landings, backlink tools identify external equity, and server logs show what bots and visitors actually request.

1. Collect candidate URLs. Combine crawler exports, Google Search Console, XML sitemap validation, analytics landing-page reports, backlink data, server logs, monitoring alerts, and migration mappings.
2. Verify the final HTTP response. Test the exact URL and follow any redirect chain. Inspect both headers and rendered content.
3. Locate the source. Export every internal page, anchor, canonical, hreflang annotation, structured-data reference, or sitemap entry pointing to the URL.
4. Measure value. Check organic landings, conversions, backlink quality, impressions, historical content purpose, and current demand.
5. Choose the treatment. Restore the page, correct the link, redirect to an equivalent resource, or retain a true 404 or 410.
6. Retest. Confirm the final status, destination relevance, canonical, indexability, rendering, and absence of loops or chains.
7. Monitor recurrence. Track newly discovered internal 404s and top requested missing URLs after every release or migration.

Screaming Frog’s documented workflow is particularly useful for exporting response codes together with linking pages, anchors, and inlink counts. Log-file analysis adds an important distinction: a URL requested thousands of times by users or search crawlers deserves different attention from an old path seen once by a third-party bot.

How to prioritize thousands of 404 URLs

Large sites should not assign equal urgency to every missing URL. Historical links, random bot probes, malformed parameters, and typographical paths can create enormous reports without equivalent business impact.

A practical score can combine internal inlinks, qualified referring domains, 90-day organic landings, conversions, sitemap presence, crawl frequency, and strategic content value. Revenue and user harm should outweigh raw URL volume. Ahrefs similarly recommends prioritizing 404s with meaningful internal links or backlinks and redirecting only when a relevant replacement exists.

Implementation patterns and common failure modes

A custom 404 page can retain branding, navigation, search, and useful recovery links while still returning the correct HTTP 404 status. Keep it lightweight and make clear that the requested resource was not found. Suggested destinations should be contextual, not a disguised redirect.

Frequent implementation failures

• HTTP 200 with a not-found message: Return 404 or 410 at the server, edge, or application-routing layer.
• Homepage redirect: Replace broad fallback rules with one-to-one mappings based on destination relevance.
• Redirect chains: Update links and rules so the old URL reaches the final destination in one step.
• Redirect loops: Test protocol, hostname, locale, slash, and canonicalization rules together.
• 404s in XML sitemaps: Remove unavailable URLs and include only canonical, indexable destinations.
• Broken canonicals or hreflang: Point annotations to valid final URLs that return 200.
• Deleted assets: Repair missing images, scripts, fonts, and API endpoints when they affect rendering or usability.
• Misleading custom pages: Do not make a missing resource appear available merely to preserve engagement.

For JavaScript applications, changing visible text is not enough. Google warns that single-page applications need meaningful status handling because a client-rendered not-found view can still return 200. Where infrastructure cannot change the initial status, use an architecture that lets the server or edge recognize invalid routes, or redirect to a URL that returns a true 404. Test the raw response and rendered result separately.

Using 404 data for content consolidation and authority recovery

A 404 inventory can reveal more than technical defects. It can expose obsolete topic clusters, content decay, failed migrations, unmaintained statistics pages, and external demand for resources that no longer exist.

Start with missing URLs that still have qualified backlinks, impressions, branded mentions, or recurring requests. Compare each with the current content graph. If a page has a direct successor, update internal links and apply a 301. If several outdated pages covered the same intent, consolidate their useful material into one stronger canonical resource before redirecting. If the old topic is no longer relevant, retain 404 or 410 rather than forcing an unrelated destination.

For hub-and-spoke structures, audit whether deleted spokes leave important questions unanswered. Rebuild only where current search demand, customer questions, or product strategy supports the topic. Useful replacement assets can include maintained statistics pages, migration references, comparison pages, original datasets, or expert-supported technical guides. These assets create more natural link demand than redirecting old URLs to generic commercial pages.

Link-intersect analysis can identify publishers still citing a missing resource. After restoring or replacing it, contact those publishers with the accurate destination. Unlinked brand mentions may offer similar opportunities, but outreach should be factual and selective. Never fabricate evidence, impersonate contributors, or use deceptive redirects to manufacture authority.

Monitoring, KPIs, and release controls

Track the health of user journeys and valuable URLs, not an unrealistic target of zero 404s. Useful KPIs include internally linked 404s, sessions landing on 404 pages, conversion journeys interrupted, backlink-bearing 404s, 404 URLs in sitemaps, crawler requests to soft 404s, average redirect hops, and the recurrence rate after releases.

Segment dashboards by template, directory, locale, device route, and discovery source. A sudden increase in product 404s after a deployment is different from gradual requests to decade-old blog slugs. Maintain release checks for navigation links, canonical tags, hreflang, structured data, XML sitemaps, redirect maps, and application routes.

After a migration, compare old and new URL inventories, validate high-value mappings, inspect logs daily, and sample rendered pages. Use strategic refresh cycles for consolidated resources so redirected authority does not terminate at another stale or removed page. Controlled title testing may improve the replacement page’s search performance, but it should not substitute for destination equivalence or correct status handling.

What is proven, accepted, and still uncertain

Proven by standards or official documentation

• 404 means the origin server cannot find a current representation or declines to disclose one.
• 404 does not state whether absence is temporary or permanent.
• 410 is available when permanent removal is known.
• Pages displaying not-found content while returning 200 can be classified as soft 404s.
• Unrelated mass redirects can confuse users and may be treated as soft 404s.

Strong practitioner consensus

• Fix active internal links before chasing every externally generated or historical URL.
• Prioritize missing URLs with traffic, conversions, backlinks, sitemap inclusion, or strategic value.
• Redirect only where the destination substantially satisfies the original intent.
• Use crawls, backlink data, analytics, Search Console, and logs together.

Still contextual or uncertain

• The exact time required for every 404 or 410 URL to disappear from each search index varies by crawl frequency and other signals.
• No universal number of 404s defines poor site quality because site size, URL generation, demand, and linking patterns differ.
• Community reports about traffic changes after mass redirects are anecdotal unless supported by controlled data.

A 2025 WebSci redirection study identified 62,000 custom 404 URIs in its dataset and classified nearly half as soft 404s. This supports the scale of implementation problems, but it does not establish that the same percentage applies to all websites. Current Reddit discussions similarly distinguish harmless historical 404s from active internal breakage, while remaining anecdotal rather than causal evidence.