Published on 2025-08-07T06:18:08Z

Yahoo Link Preview

Yahoo Link Preview is the web fetcher for Yahoo Mail. It is not a general-purpose crawler but an on-demand bot that visits a web page only when a user includes a link to it in a Yahoo Mail message. Its purpose is to retrieve metadata like the title and thumbnail image to generate the rich link preview that appears in the email. For website owners, this is beneficial as it makes your content more engaging when shared.

What is Yahoo Link Preview?

Yahoo Link Preview is an automated web service from Yahoo that generates visual link previews for URLs shared in Yahoo Mail. It is a specialized web client that fetches and analyzes web pages to extract metadata when a user pastes a link into an email. The service identifies itself with the user-agent string Mozilla/5.0 (compatible; Yahoo Link Preview; ...). It primarily relies on static HTML analysis and Open Graph tags to efficiently extract the metadata needed for a preview.

Why is Yahoo Link Preview crawling my site?

Yahoo Link Preview is visiting your site because someone has included a link to your content in a Yahoo Mail message. Its visits are triggered by this user action, not by a systematic crawl. The service looks for metadata that helps create a rich link preview, such as titles, descriptions, and featured images. The visits are typically a one-time event per URL, as Yahoo uses caching to reduce duplicate requests.

What is the purpose of Yahoo Link Preview?

The purpose of Yahoo Link Preview is to enhance the user experience in Yahoo Mail by transforming plain text URLs into visually informative previews. This benefits both the sender, by creating more engaging messages, and the recipient, by providing visual verification of a link's destination. The system also contributes to Yahoo's anti-phishing efforts by analyzing domain reputation as part of the preview generation process. For website owners, well-optimized previews can improve click-through rates.

How do I block Yahoo Link Preview?

To prevent Yahoo Link Preview from generating previews of your content, you can add a disallow rule to your robots.txt file. This will cause links to your site to appear as plain text in Yahoo Mail.

To block this bot, add the following lines to your robots.txt file:

User-agent: Yahoo Link Preview
Disallow: /

How to verify the authenticity of the user-agent operated by Yahoo?

Reverse IP lookup technique

To verify user-agent authenticity, you can use host linux command two times with the IP address of the requester.

```
> host IPAddressOfRequest
```
This command returns the reverse lookup hostname (e.g., 4.4.8.8.in-addr.arpa.).

> host ReverseDNSFromTheOutputOfFirstRequest

If the output matches the original IP address and the domain is associated with a trusted operator (e.g., Yahoo), the user-agent can be considered legitimate.

IP list lookup technique

Some operators provide a public list of IP addresses used by their crawlers. This list can be cross-referenced to verify a user-agent's authenticity. However, both operators and website owners may find it challenging to maintain an up-to-date list, so use this method with caution and in conjunction with other verification techniques.