YandexSitelinks bot

The YandexSitelinks bot is a web crawler from Yandex designed specifically to validate and maintain the integrity of the sitelinks that are displayed in its search results. Sitelinks are the secondary links that appear beneath a website's main search result. The bot identifies itself in server logs with the user-agent string YandexSitelinksBot. It operates independently of the main Yandex crawler and focuses exclusively on monitoring the URLs that have been designated as potential sitelinks.

The YandexSitelinks bot is visiting your site to verify the availability and relevance of pages that Yandex has identified as potential sitelinks for your domain. It periodically checks these pages to ensure they are still accessible and serve their intended purpose. The bot focuses on important navigational pages like 'About Us' or 'Contact' pages. This is an authorized and standard activity for a major search engine.

The purpose of the YandexSitelinks bot is to enhance the quality of Yandex search results by ensuring that sitelinks are functional and relevant. This helps Yandex maintain user satisfaction by preventing broken links from appearing in search results. For website owners, having sitelinks displayed in Yandex search can significantly improve click-through rates by allowing users to navigate directly to important sections of your site. The bot's work benefits both search users and website owners.

To prevent the YandexSitelinks bot from accessing your website, you can add a specific disallow rule to your robots.txt file. This is the standard method for managing crawler access.

To block this bot, add the following lines to your robots.txt file:

User-agent: YandexSitelinksBot
Disallow: /