CrawlerCheck

CrawlerCheck is a web-based tool that allows SEO professionals and webmasters to test website crawlability for major search engine bots (such as Googlebot and Bingbot) and AI crawlers (including ChatGPT, Claude, and Perplexity). The platform analyzes robots.txt files, meta tags, and HTTP headers to identify access restrictions, helping users prevent accidental blocking of important crawers or verify intentional blocks against AI data scraping.

LLM mention score

Normalized 0–100 · last 8 weeks

DataForSEO API

Get LLM mention data of any company via DataForSEO API

Get access to the structured data on keyword, brand, and website mentions in LLMs, including metrics like AI search volume, impressions, and mentions count.

Get Free API Credits API Documentation

How to get LLM mention data →

// Fetch CrawlerCheck mentions
POST v3/ai_optimization/llm_mentions/search/live
[
    {
        "target": [
            {
                "keyword": "CrawlerCheck",
                "search_scope": ["any"]
            }
        ],
        "platform": "chat_gpt",
        "order_by" : ["ai_search_volume,desc"]
    }
]