Published on 2025-08-07T06:18:08Z

MojeekBot

MojeekBot is the official web crawler for Mojeek, a UK-based independent search engine. Its purpose is to discover and index public web content to build Mojeek's own search database, without relying on data from other major search engines. For website owners, being indexed by MojeekBot offers another channel for discovery, particularly for reaching a privacy-conscious audience.

What is MojeekBot?

MojeekBot is the web crawler for the independent search engine Mojeek. It functions as a traditional search engine bot, systematically browsing the internet to discover, analyze, and index web content for Mojeek's search platform. The bot identifies itself in server logs with a user-agent string like Mozilla/5.0 (compatible; MojeekBot/0.11; +https://www.mojeek.com/bot.html). A key characteristic of Mojeek is its commitment to building its own independent search index, which makes this bot essential to its operations.

Why is MojeekBot crawling my site?

MojeekBot is crawling your site to discover and index your public content for inclusion in its search results. It is looking for text, images, and links to help build its search database. The frequency of its visits is determined by factors like your site's size, popularity, and how often you update your content. This is a standard and legitimate activity for a search engine crawler, and the bot is designed to respect standard protocols.

What is the purpose of MojeekBot?

The primary purpose of MojeekBot is to build and maintain the search index for Mojeek. As a privacy-focused alternative to larger search engines, Mojeek emphasizes that it does not track users. The data collected by the bot is used exclusively to power its search results. For website owners, having your content indexed by Mojeek provides another channel for discovery, allowing you to reach an audience that values privacy and independent search.

How do I block MojeekBot?

If you wish to prevent MojeekBot from accessing your website, you can add a disallow rule to your robots.txt file. This will prevent your pages from appearing in Mojeek search results.

To block MojeekBot, add the following lines to your robots.txt file:

User-agent: MojeekBot
Disallow: /

How to verify the authenticity of the user-agent operated by Mojeek?

Reverse IP lookup technique

To verify user-agent authenticity, you can use host linux command two times with the IP address of the requester.
  1. > host IPAddressOfRequest
    This command returns the reverse lookup hostname (e.g., 4.4.8.8.in-addr.arpa.). 
  2. > host ReverseDNSFromTheOutputOfFirstRequest
If the output matches the original IP address and the domain is associated with a trusted operator (e.g., Mojeek), the user-agent can be considered legitimate.

IP list lookup technique

Some operators provide a public list of IP addresses used by their crawlers. This list can be cross-referenced to verify a user-agent's authenticity. However, both operators and website owners may find it challenging to maintain an up-to-date list, so use this method with caution and in conjunction with other verification techniques.