Loading…
Run a golden set and gate LLM model swaps on accuracy, with latency and cost per request.
LLM Eval Harness is an MCP server entry in ChatHome Skills, sourced from Official MCP Registry and scored 78/100. As a Model Context Protocol server, it exposes tools that MCP-compatible LLM clients such as Claude, Cursor, and Copilot can call directly, extending an agent's reach beyond its built-in capabilities. ChatHome Skills discovered this entry through its daily ingest of official registries and GitHub discovery queries, and re-verifies repository facts, freshness signals, and install metadata on every run. The quality score weighs official-registry presence, a working install command, license clarity, GitHub star trajectory, and repository freshness — the full formula is on the Sources page. The install command above is ready to copy. Review the last-pushed date and repository status before adding LLM Eval Harness to a production agent workflow; the snapshot facts below are point-in-time values from the most recent ingest.
docker run -i --rm ghcr.io/razilzul/llm-eval-harness:1.0.0Review the upstream source before running this command.
{
"_meta": {
"io.modelcontextprotocol.registry/official": {
"status": "active",
"isLatest": true,
"updatedAt": "2026-10-18T13:22:46.283666Z",
"publishedAt": "2026-10-18T13:22:46.283666Z",
"statusChangedAt": "2026-10-18T13:22:46.283666Z"
}
},
"server": {
"name": "io.github.razilzul/llm-eval-harness",
"title": "LLM Eval Harness",
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
"version": "1.0.0",
"packages": [
{
"transport": {
"type": "stdio"
},
"identifier": "ghcr.io/razilzul/llm-eval-harness:1.0.0",
"registryType": "oci",
"environmentVariables": [
{
"name": "ANTHROPIC_API_KEY",
"format": "string",
"isSecret": true,
"description": "Only needed to evaluate anthropic:* models. Not needed for mock models."
},
{
"name": "GEMINI_API_KEY",
"format": "string",
"isSecret": true,
"description": "Only needed to evaluate gemini:* models. Not needed for mock models."
}
]
}
],
"repository": {
"url": "https://github.com/razilzul/llm-eval-harness",
"source": "github"
},
"description": "Run a golden set and gate LLM model swaps on accuracy, with latency and cost per request."
}
}