MCP Evals

mcpevals.io

MCP Evals is a developer tool designed for evaluating Model Context Protocol (MCP) tool implementations. It provides a Node.js package and GitHub Action that leverage LLM-based scoring to assess tools based on metrics such as accuracy, completeness, and relevance. The solution integrates directly into CI/CD pipelines to automate testing and provide actionable feedback on MCP server performance.

LLM mention score The LLM mention score is the total number of mentions of this brand in different LLM chatbots, normalized to the scale from 0 to 100. You can get actual, non-normalized numbers via the LLM Mention API from DataForSEO.

Normalized 0–100 · last 8 weeks

DataForSEO API

Get LLM mention data of any company via DataForSEO API

Get access to the structured data on keyword, brand, and website mentions in LLMs, including metrics like AI search volume, impressions, and mentions count. 

How to get LLM mention data →
// Fetch MCP Evals mentions
POST v3/ai_optimization/llm_mentions/search/live
[
    {
        "target": [
            {
                "keyword": "MCP Evals",
                "search_scope": ["any"]
            }
        ],
        "platform": "chat_gpt",
        "order_by" : ["ai_search_volume,desc"]
    }
]