Published on 2025-08-07T06:18:08Z

Mediatoolkitbot

Mediatoolkitbot is the official web crawler for Determ, a company providing online media monitoring and analytics services. Its purpose is to scan public web pages to find mentions of specific keywords, brands, or topics that Determ's clients are tracking. The bot is well-behaved and respects robots.txt directives. Its presence indicates your content is relevant to ongoing brand or market monitoring.

What is Mediatoolkitbot?

Mediatoolkitbot is an intelligence-gathering web crawler operated by Determ, a media monitoring and analytics company. The bot systematically browses websites to collect publicly available content for Determ's monitoring platform. It identifies itself in server logs with the user-agent string Mediatoolkitbot. The crawler scans over 100 million web sources to identify content that matches the specific phrases or topics being monitored by Determ's users. It is designed to be a well-behaved crawler that respects robots.txt directives.

Why is Mediatoolkitbot crawling my site?

Mediatoolkitbot is visiting your website to search for content that may be relevant to Determ's clients, who are typically marketing and PR professionals monitoring online brand mentions. The bot is specifically looking for content that matches the keywords that Determ users have set up for tracking. The frequency of its visits depends on how often your site publishes content that is relevant to these monitored topics. The crawler only accesses publicly available content.

What is the purpose of Mediatoolkitbot?

The purpose of Mediatoolkitbot is to support Determ's media monitoring platform, which helps businesses track their online presence, monitor competitors, and measure campaign performance. The data collected by the bot allows Determ to alert its users when their monitored keywords appear in new online content. While website owners do not directly benefit from being crawled, the visibility that comes from being included in media monitoring reports can sometimes lead to new business or partnership opportunities.

How do I block Mediatoolkitbot?

To prevent Mediatoolkitbot from accessing your website, you can add a disallow rule for it in your robots.txt file. This is the standard method for managing access for legitimate web crawlers.

Add the following lines to your robots.txt file to block Mediatoolkitbot:

User-agent: Mediatoolkitbot
Disallow: /

How to verify the authenticity of the user-agent operated by Determ?

Reverse IP lookup technique

To verify user-agent authenticity, you can use host linux command two times with the IP address of the requester.
  1. > host IPAddressOfRequest
    This command returns the reverse lookup hostname (e.g., 4.4.8.8.in-addr.arpa.).
  2. > host ReverseDNSFromTheOutputOfFirstRequest
If the output matches the original IP address and the domain is associated with a trusted operator (e.g., Determ), the user-agent can be considered legitimate.

IP list lookup technique

Some operators provide a public list of IP addresses used by their crawlers. This list can be cross-referenced to verify a user-agent's authenticity. However, both operators and website owners may find it challenging to maintain an up-to-date list, so use this method with caution and in conjunction with other verification techniques.