mcp-evaluator

Model-agnostic harness that measures how well LLMs call the tools of any MCP server

What does this entry provide?

mcp-evaluator is an MCP server entry in ChatHome Skills, sourced from GitHub MCP Search and scored 44/100. Model-agnostic harness that measures how well LLMs call the tools of any MCP server. As a Model Context Protocol server, it exposes tools that MCP-compatible LLM clients such as Claude, Cursor, and Copilot can call directly, extending an agent's reach beyond its built-in capabilities. ChatHome Skills discovered this entry through its daily ingest of official registries and GitHub discovery queries, and re-verifies repository facts, freshness signals, and install metadata on every run. The quality score weighs official-registry presence, a working install command, license clarity, GitHub star trajectory, and repository freshness — the full formula is on the Sources page. The install command above is ready to copy. Review the last-pushed date and repository status before adding mcp-evaluator to a production agent workflow; the snapshot facts below are point-in-time values from the most recent ingest.

Snapshot facts

Source: GitHub MCP Search
Quality: 44/100
Install method: source code
Repo: ajimae/mcp-evaluator
Repo updated: Sep 1, 2026

Source checkout

git clone https://github.com/ajimae/mcp-evaluator.git

Install method: source code

Review the upstream source before running this command.

GitHub project Homepage

Quality

Repo stars

Repo updated

Sep 1, 2026

How is this score calculated?

Weighted across six dimensions, scored 0-100.
Provenance (25): listed in an official or curated registry.
Installability (25): a runnable install command beats a declared transport, which beats a source checkout.
Maintenance (20): recency of the last push; archived entries score zero here.
Adoption (20): repository stars on a log curve.
Documentation (10) and description quality (10): parsed manifest or SKILL.md frontmatter, and a real description rather than a placeholder.

Security signals

Scan coverage: Install metadata and repository README.

Automated signals from install metadata and a bounded skill body — not a full code security audit and not a recursive source or dependency review.

MCP manifest

{
  "fullName": "ajimae/mcp-evaluator"
}

Discovery note

Model-agnostic harness that measures how well LLMs call the tools of any MCP server

Report an issue with this listing