Twingly Recon

Twingly Recon is a web crawler from the Swedish blog and social media monitoring company Twingly. It is a data collection bot that systematically browses the web to gather information for Twingly's blog and content discovery services. The crawler identifies itself with user-agent strings like Twingly Recon. It operates from various European IP addresses, primarily in Sweden, and is designed to find and index blog posts and news articles.

Twingly Recon is crawling your site to discover and index its content for Twingly's blog search and monitoring services. If your site contains blogs or news articles, the bot may visit to keep its index current with your latest publications. The frequency of visits depends on how often you update your content. This is a legitimate web activity as part of a content discovery service.

The purpose of Twingly Recon is to power Twingly's blog search engine and media monitoring services. The company offers products that help businesses track mentions of specific topics or brands across blogs and other online publications. For website owners, particularly bloggers, being indexed by Twingly can provide additional visibility, as your content may appear in Twingly's search results and monitoring feeds. This can drive traffic to your site from users of Twingly's services.

To prevent Twingly Recon from accessing your website, you can add a disallow rule to your robots.txt file. There are several user-agents for the service, so you may need to block them individually.

To block the main crawler, add the following lines to your robots.txt file:

User-agent: Twingly Recon
Disallow: /