Published on 2025-08-07T06:18:08Z

PerplexityBot

PerplexityBot is the official web crawler for Perplexity, the AI-powered search and answer engine. Its purpose is to index public web content to power Perplexity's search results. Unlike some other AI crawlers, Perplexity explicitly states that this bot is not used to crawl content for training its AI models. For website owners, being indexed by this bot can drive traffic through citations in Perplexity's search answers.

What is PerplexityBot?

PerplexityBot is the web crawler for the AI-powered search engine Perplexity. It functions as an AI search crawler, indexing web content to be surfaced and linked in Perplexity's search results. The bot identifies itself in server logs with the user-agent string Mozilla/5.0 (...) PerplexityBot/1.0; +https://perplexity.ai/perplexitybot. Perplexity makes a clear distinction between this bot and other AI crawlers, stating that its purpose is search indexing, not collecting content to train AI foundation models.

Why is PerplexityBot crawling my site?

PerplexityBot is crawling your website to gather and index information that will help it deliver relevant search results to its users. It is looking for content that can be referenced when users ask questions through the Perplexity interface. The frequency of its visits is not on a fixed schedule but varies based on factors like your site's popularity, content freshness, and relevance to common queries. This crawling is a standard practice for a search engine and is considered authorized as long as the bot respects your robots.txt directives.

What is the purpose of PerplexityBot?

The purpose of PerplexityBot is to support Perplexity's AI-powered search service by building an index of web content. A key feature of Perplexity is that it provides answers with inline citations that link back to the source websites. For website owners, this can be a valuable source of traffic. As AI-powered search becomes more common, being properly indexed by services like Perplexity may become increasingly important for online visibility.

How do I block PerplexityBot?

If you wish to prevent PerplexityBot from indexing your site for its search results, you can add a disallow rule to your robots.txt file. This is the standard method for managing crawler access.

To block this bot, add the following lines to your robots.txt file:

User-agent: PerplexityBot
Disallow: /

How to verify the authenticity of the user-agent operated by Perplexity?

Reverse IP lookup technique

To verify user-agent authenticity, you can use host linux command two times with the IP address of the requester.
  1. > host IPAddressOfRequest
    This command returns the reverse lookup hostname (e.g., 4.4.8.8.in-addr.arpa.). 
  2. > host ReverseDNSFromTheOutputOfFirstRequest
If the output matches the original IP address and the domain is associated with a trusted operator (e.g., Perplexity), the user-agent can be considered legitimate.

IP list lookup technique

Some operators provide a public list of IP addresses used by their crawlers. This list can be cross-referenced to verify a user-agent's authenticity. However, both operators and website owners may find it challenging to maintain an up-to-date list, so use this method with caution and in conjunction with other verification techniques.