Published on 2025-08-07T06:18:08Z

Slackbot-LinkExpanding

Slackbot-LinkExpanding is the specialized web crawler from the communication platform Slack. It is not a general-purpose crawler but an on-demand fetcher that visits a web page only when a user shares a link to it in a Slack conversation. Its purpose is to retrieve metadata like the title, description, and thumbnail image to generate the rich link preview that appears in the chat. For website owners, this is beneficial as it makes your content more engaging when shared.

What is Slackbot-LinkExpanding?

Slackbot-LinkExpanding is the web crawler from Slack that fetches metadata from URLs shared in Slack conversations. When a user shares a link, this bot visits the URL to extract information to create a rich link preview. It identifies itself in server logs with the user-agent string Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots). It is a lightweight and efficient crawler that does not execute JavaScript or store cookies; it primarily looks for Open Graph metadata to generate its previews.

Why is Slackbot-LinkExpanding crawling my site?

Slackbot-LinkExpanding is visiting your website because a user has shared a link to one of your pages in a Slack conversation. The bot is not proactively crawling your site; its visit is triggered by a user sharing your URL. The frequency of visits is directly tied to how often your links are shared in Slack. This is a standard and authorized activity for a social platform and is a sign of user interest in your content.

What is the purpose of Slackbot-LinkExpanding?

The purpose of Slackbot-LinkExpanding is to enhance communication in Slack by providing visual context for shared links. Instead of plain URLs, Slack shows rich previews with titles, descriptions, and images. This improves the user experience by making conversations more engaging and informative. For website owners, these previews are beneficial as they can increase visibility and engagement when your content is shared on Slack. Well-crafted metadata ensures your content is presented attractively.

How do I block Slackbot-LinkExpanding?

If you wish to prevent Slack from generating link previews for your site, you can block the Slackbot-LinkExpanding crawler. This will cause links to your site to appear as plain text URLs in Slack. You can add the following rule to your robots.txt file.

To block this bot, use this directive:

User-agent: Slackbot-LinkExpanding
Disallow: /

How to verify the authenticity of the user-agent operated by Slack?

Reverse IP lookup technique

To verify user-agent authenticity, you can use host linux command two times with the IP address of the requester.
  1. > host IPAddressOfRequest
    This command returns the reverse lookup hostname (e.g., 4.4.8.8.in-addr.arpa.). 
  2. > host ReverseDNSFromTheOutputOfFirstRequest
If the output matches the original IP address and the domain is associated with a trusted operator (e.g., Slack), the user-agent can be considered legitimate.

IP list lookup technique

Some operators provide a public list of IP addresses used by their crawlers. This list can be cross-referenced to verify a user-agent's authenticity. However, both operators and website owners may find it challenging to maintain an up-to-date list, so use this method with caution and in conjunction with other verification techniques.