dotbot

DotBot is the web crawler for Moz, a prominent SEO software company. It functions as an indexing and discovery bot that systematically visits websites to collect data that powers Moz's suite of SEO tools and analytics. The crawler identifies itself in server logs with the user-agent string DotBot. It is designed to analyze website structure, content quality, and link profiles, behaving similarly to other legitimate search engine bots by respecting standard web protocols.

DotBot is crawling your site to gather data about its structure, content, and, most importantly, its link relationships. It is primarily interested in discovering the links between websites to build the Moz link index, which is used to calculate their proprietary Domain Authority and Page Authority scores. The frequency of its visits depends on factors like your site's size, popularity, and update schedule. The crawling is generally considered authorized as it is part of a legitimate SEO analysis service and respects robots.txt directives.

The core purpose of DotBot is to collect the web data that powers Moz's SEO analytics platform. Its main function is to map the web's link structure to calculate metrics like Domain Authority, which helps SEO professionals predict a site's ranking potential. The data also supports analysis of on-page elements and site structure. For website owners using Moz's tools, the bot provides valuable data about their online presence. For others, it contributes to the broader SEO ecosystem that helps professionals understand and improve website performance.

To prevent DotBot from accessing your website, you can add a rule to your robots.txt file. This is the standard method for instructing web crawlers not to visit your site.

Add the following lines to your robots.txt file to block DotBot:

User-agent: DotBot
Disallow: /