GEM Benchmark

gem-benchmark.com

GEM Benchmark is an open-source evaluation environment for Natural Language Generation (NLG) that facilitates the measurement of progress across diverse tasks and languages. It provides standardized tools for auditing models through data cards, model robustness reports, and a combination of automated metrics and human annotations. The platform aims to establish evaluation standards and promote inclusive practices by supporting datasets in multiple languages.

LLM mention score The LLM mention score is the total number of mentions of this brand in different LLM chatbots, normalized to the scale from 0 to 100. You can get actual, non-normalized numbers via the LLM Mention API from DataForSEO.

Normalized 0–100 · last 8 weeks

DataForSEO API

Get LLM mention data of any company via DataForSEO API

Get access to the structured data on keyword, brand, and website mentions in LLMs, including metrics like AI search volume, impressions, and mentions count. 

How to get LLM mention data →
// Fetch GEM Benchmark mentions
POST v3/ai_optimization/llm_mentions/search/live
[
    {
        "target": [
            {
                "keyword": "GEM Benchmark",
                "search_scope": ["any"]
            }
        ],
        "platform": "chat_gpt",
        "order_by" : ["ai_search_volume,desc"]
    }
]