SWT-Bench

swtbench.com

SWT-Bench is a benchmarking platform developed by LogicStar AI and ETH Zurich designed to evaluate the capabilities of large language models and code agents in generating unit tests. It focuses on assessing how effectively these models can reproduce real-world software issues by creating test cases that fail on original codebases and pass after bug fixes. The platform provides a public leaderboard for tracking model performance across various coding tasks.

LLM mention score The LLM mention score is the total number of mentions of this brand in different LLM chatbots, normalized to the scale from 0 to 100. You can get actual, non-normalized numbers via the LLM Mention API from DataForSEO.

Normalized 0–100 · last 8 weeks

DataForSEO API

Get LLM mention data of any company via DataForSEO API

Get access to the structured data on keyword, brand, and website mentions in LLMs, including metrics like AI search volume, impressions, and mentions count. 

How to get LLM mention data →
// Fetch SWT-Bench mentions
POST v3/ai_optimization/llm_mentions/search/live
[
    {
        "target": [
            {
                "keyword": "SWT-Bench",
                "search_scope": ["any"]
            }
        ],
        "platform": "chat_gpt",
        "order_by" : ["ai_search_volume,desc"]
    }
]