Mastodon bot

The Mastodon bot is a web fetcher for the open-source, decentralized social network Mastodon. It is not a traditional web crawler; rather, it retrieves content on behalf of a specific Mastodon server (or 'instance') when a user shares a link. It identifies itself with a user-agent string that includes both the Mastodon software version and the domain of the instance making the request, for example: Mastodon/4.3.2 (+https://mastodon.social/). Because Mastodon is a federated network of independent servers, there is no single operator; each server has its own administrators.

A Mastodon bot is visiting your website because a user on a Mastodon instance has shared a link to one of your pages in a post. The bot is dispatched on-demand to gather metadata—such as the page title, description, and a thumbnail image—to generate a rich link preview. The frequency of visits is directly tied to how often users share your links across the Mastodon network. This is a standard and authorized practice for social media platforms to enhance the user experience.

The primary purpose of the Mastodon bot is to improve the user experience by providing rich previews for links shared within the platform. When a link is posted, the bot fetches metadata to display a preview card with an image, title, and description. This benefits both users, by providing context, and website owners. For site owners, these previews can increase visibility and click-through rates when content is shared across the Mastodon network. The data collected is used solely for generating these previews and not for other purposes like AI training or search indexing.

If you wish to prevent Mastodon from generating link previews for your site, you can block its fetcher using your robots.txt file. Note that this will cause links to your site to appear as plain text URLs.

To block the bot, add the following lines to your robots.txt file:

User-agent: Mastodon
Disallow: /