Published on 2025-08-07T06:18:08Z

deadlinkchecker bot

The deadlinkchecker bot is a web crawler for the Dead Link Checker service, operated by DLC Websites. Its purpose is to scan websites to identify broken or 'dead' links, which helps webmasters with site maintenance and SEO optimization. Its presence on your site is almost always the result of someone—either you, your SEO team, or a third party—intentionally initiating a scan to check your site's link health.

What is deadlinkchecker?

Dead Link Checker is a specialized auditing crawler from DLC Websites that scans websites to find broken links. It is a tool designed to help webmasters and SEO professionals maintain a site's link integrity. The crawler systematically navigates through a website, checking each hyperlink to verify that it leads to a valid destination. It identifies itself with user-agent strings like www.deadlinkchecker.com Mozilla/5.0 (...) or Mozilla/5.0 (compatible; Dead Link Checker; http://www.dead-link-checker.com/). As a well-behaved crawler, it respects robots.txt directives, including Crawl-delay instructions.

Why is deadlinkchecker crawling my site?

The deadlinkchecker bot is visiting your website because someone has initiated a scan using the Dead Link Checker service. This could be you, a member of your team, an SEO professional you've hired, or even a third party analyzing your site's health. The crawler's visit is not random; it is a direct result of a user-initiated request to find and report on broken links. The frequency of its visits depends on whether it is a one-time scan or a regularly scheduled check.

What is the purpose of deadlinkchecker?

The primary purpose of the Dead Link Checker service is to support website maintenance and SEO optimization. Broken links can harm user experience and negatively impact a site's search engine rankings. By systematically identifying these dead links, the tool helps website owners improve site quality, enhance user experience by eliminating 404 errors, and support their technical SEO efforts. The service is valuable for regular site health monitoring, especially for large websites with many internal and external links.

How do I block deadlinkchecker?

While blocking this bot is generally not necessary as its scans are user-initiated for a beneficial purpose, you can prevent it from accessing your site by adding rules to your robots.txt file.

To block the Dead Link Checker bot, you can use one of its known user-agent strings. Add the following lines to your robots.txt file:

User-agent: Dead Link Checker
Disallow: /

User-agent: www.deadlinkchecker.com
Disallow: /

How to verify the authenticity of the user-agent operated by DLC Websites?

Reverse IP lookup technique

To verify user-agent authenticity, you can use host linux command two times with the IP address of the requester.

```
> host IPAddressOfRequest
```
This command returns the reverse lookup hostname (e.g., 4.4.8.8.in-addr.arpa.).

> host ReverseDNSFromTheOutputOfFirstRequest

If the output matches the original IP address and the domain is associated with a trusted operator (e.g., DLC Websites), the user-agent can be considered legitimate.

IP list lookup technique

Some operators provide a public list of IP addresses used by their crawlers. This list can be cross-referenced to verify a user-agent's authenticity. However, both operators and website owners may find it challenging to maintain an up-to-date list, so use this method with caution and in conjunction with other verification techniques.