Published on 2025-08-07T06:18:08Z

YisouSpider

YisouSpider is the web crawler for the Chinese mobile-focused search engine Shenma (SM.CN). It scans and indexes web content to power Shenma's search results. For website owners, being indexed can increase visibility among mobile users in the Chinese market, which can be a valuable source of diverse traffic.

What is YisouSpider?

YisouSpider is the web crawler for the Chinese search engine Shenma (SM.CN). It is a search engine crawler that systematically browses the web to discover and index content for its search results. The bot identifies itself in server logs with user-agent strings like YisouSpider or more detailed variants like Mozilla/5.0 (...) YisouSpider/5.0 Safari/537.36. The crawler operates from IP addresses primarily based in China.

Why is YisouSpider crawling my site?

YisouSpider is crawling your website to evaluate its content for inclusion in Shenma's search index. It is interested in all types of content that would be relevant to search queries from its users. The frequency of visits depends on how often your content changes and how valuable Shenma considers it to be. This is a standard and generally authorized activity for a search engine.

What is the purpose of YisouSpider?

The purpose of YisouSpider is to support the Shenma search engine, which is popular for mobile search in China. The data it collects is used to determine page rankings and provide search results. For website owners, having your content indexed by YisouSpider can increase your visibility to Chinese audiences using the Shenma search engine. Being indexed by regional search engines like Shenma can diversify your traffic sources and increase your reach in specific markets.

How do I block YisouSpider?

To prevent YisouSpider from accessing your website, you can add a specific disallow rule to your robots.txt file. This will prevent your pages from appearing in Shenma's search results.

To block this bot, add the following lines to your robots.txt file:

User-agent: YisouSpider
Disallow: /

How to verify the authenticity of the user-agent operated by Sm.cm?

Reverse IP lookup technique

To verify user-agent authenticity, you can use host linux command two times with the IP address of the requester.
  1. > host IPAddressOfRequest
    This command returns the reverse lookup hostname (e.g., 4.4.8.8.in-addr.arpa.). 
  2. > host ReverseDNSFromTheOutputOfFirstRequest
If the output matches the original IP address and the domain is associated with a trusted operator (e.g., Sm.cm), the user-agent can be considered legitimate.

IP list lookup technique

Some operators provide a public list of IP addresses used by their crawlers. This list can be cross-referenced to verify a user-agent's authenticity. However, both operators and website owners may find it challenging to maintain an up-to-date list, so use this method with caution and in conjunction with other verification techniques.