Website Crawler

websitecrawler.org
SEO Tools #131 ▼ -3

Website Crawler is a SaaS platform that performs real-time crawling and analysis of web pages to identify technical SEO issues, broken links, and performance bottlenecks. The service provides detailed audit reports, data extraction capabilities via CSV/JSON, and an API for automated integration, supporting both manual audits and ongoing monitoring.

LLM mention score The LLM mention score is the total number of mentions of this brand in different LLM chatbots, normalized to the scale from 0 to 100. You can get actual, non-normalized numbers via the LLM Mention API from DataForSEO.

Normalized 0–100 · last 8 weeks

DataForSEO API

Get LLM mention data of any company via DataForSEO API

Get access to the structured data on keyword, brand, and website mentions in LLMs, including metrics like AI search volume, impressions, and mentions count. 

How to get LLM mention data →
// Fetch Website Crawler mentions
POST v3/ai_optimization/llm_mentions/search/live
[
    {
        "target": [
            {
                "keyword": "Website Crawler",
                "search_scope": ["any"]
            }
        ],
        "platform": "chat_gpt",
        "order_by" : ["ai_search_volume,desc"]
    }
]