Published on 2025-08-07T06:18:08Z

YandexVideoParser

YandexVideoParser is a specialized web crawler from the Russian search engine Yandex that discovers, analyzes, and indexes video content for its video search results. It focuses exclusively on video files and their associated metadata to help power a rich video search experience for users. For website owners, its activity can drive traffic from users searching for video content on Yandex.

What is YandexVideoParser?

YandexVideoParser is a web crawler from Yandex designed specifically to discover and index video content for its video search results. It is a specialized part of Yandex's crawler suite, focusing exclusively on video files and their metadata. The bot identifies itself in server logs with the user-agent string Mozilla/5.0 (compatible; YandexVideoParser/1.0; +http://yandex.com/bots). It has targeted behavior, focusing on pages with embedded videos and video sitemaps, and it is particularly interested in parsing HTML for video metadata.

Why is YandexVideoParser crawling my site?

YandexVideoParser is visiting your website because it contains video content that Yandex wants to include in its video search results. The bot is looking for video files, embedded players, and associated metadata. Sites that regularly publish new videos will typically see more frequent visits. This is a standard and authorized part of the web indexing process for a major search engine.

What is the purpose of YandexVideoParser?

The purpose of YandexVideoParser is to power Yandex's video search feature. By extracting detailed information about videos, it allows Yandex to offer users rich video search results with thumbnails, durations, and other relevant metadata. For website owners, having your videos properly indexed by this bot can drive additional traffic from Yandex's video search. This is especially valuable for sites targeting Russian-speaking audiences or operating in regions where Yandex has a significant market share.

How do I block YandexVideoParser?

To prevent YandexVideoParser from accessing your website, you can add a specific disallow rule to your robots.txt file. This will prevent your videos from appearing in Yandex's video search results.

To block this bot, add the following lines to your robots.txt file:

User-agent: YandexVideoParser
Disallow: /

How to verify the authenticity of the user-agent operated by Yandex?

Reverse IP lookup technique

To verify user-agent authenticity, you can use host linux command two times with the IP address of the requester.

```
> host IPAddressOfRequest
```
This command returns the reverse lookup hostname (e.g., 4.4.8.8.in-addr.arpa.).

> host ReverseDNSFromTheOutputOfFirstRequest

If the output matches the original IP address and the domain is associated with a trusted operator (e.g., Yandex), the user-agent can be considered legitimate.

IP list lookup technique

Some operators provide a public list of IP addresses used by their crawlers. This list can be cross-referenced to verify a user-agent's authenticity. However, both operators and website owners may find it challenging to maintain an up-to-date list, so use this method with caution and in conjunction with other verification techniques.