← Experts

Measure and improve AI quality

Build eval suites that catch regressions before users do. Write LLM-as-judge evaluators, create golden datasets, run CI evals on every model change, and track quality metrics over time. Turn 'it feels worse' into a reproducible test.

1

Expert Install

Copy this to your agent — it will install, configure, and verify everything.

Say to your agent
Read https://clawhub.md/expert/eval-expert.md and set me up as AI Eval Engineer

Works on OpenClaw, Claude Code, Telegram, Feishu, and any agent interface. Your agent reads eval-expert.md and follows the setup steps inside.

2

Talk and handle it

After setup, say these to your agent:

Use Claude as a judge —… Official

  • › "Write a Claude-as-judge evaluator for my chatbot"
  • › "Compare my current prompt vs a new version on 50 test cases"
  • › "Build an eval that detects hallucinations in my RAG pipeline"

Scaffold eval harnesses, generate test datasets,… Official

  • › "Scaffold a pytest eval suite for my LLM app"
  • › "Generate 100 diverse test cases from my production logs"
  • › "Wire evals into my GitHub Actions CI"

Track eval results across PRs —… Official

  • › "Post eval score diff as a PR comment"
  • › "Block merge if eval score drops below threshold"
  • › "List all eval regressions since the last release"
▶ View setup file eval-expert.md

AI Eval Engineer

Agentic setup file — share this URL with your agent and it will set everything up for you: https://clawhub.md/expert/eval-expert.md

Goal: Measure and improve AI quality

What you'll have: Build eval suites that catch regressions before users do. Write LLM-as-judge evaluators, create golden datasets, run CI evals on every model change, and track quality metrics over time. Turn 'it feels worse' into a reproducible test.

Step 1: Install

clawhub install anthropics/claude-api openclaw/coding-agent openclaw/github

Step 2: Configure

Each skill may need credentials or auth before it can act on your behalf.

openclaw/coding-agent

Scaffold eval harnesses, generate test datasets, and wire up CI pipelines — so evals run automatically on every commit.

  • Ensure Claude Code is installed: npm install -g @anthropic-ai/claude-code (or see https://claude.ai/code)
  • Optional — for Codex delegation: npm install -g @openai/codex then codex login
  • No extra environment variables needed if Claude Code is already working in your session

openclaw/github

Track eval results across PRs — comment score diffs on pull requests, block merges when quality drops, and keep a history of model quality over time.

  • Authenticate with the GitHub CLI: run gh auth login and follow the prompts
  • Choose HTTPS or SSH, then log in via browser — no extra tokens needed
  • The skill uses gh under the hood, so all your existing GitHub access applies

Step 3: Try it

After setup, say these to your agent to verify everything works:

anthropics/claude-api

  • "Write a Claude-as-judge evaluator for my chatbot"
  • "Compare my current prompt vs a new version on 50 test cases"
  • "Build an eval that detects hallucinations in my RAG pipeline"

openclaw/coding-agent

  • "Scaffold a pytest eval suite for my LLM app"
  • "Generate 100 diverse test cases from my production logs"
  • "Wire evals into my GitHub Actions CI"

openclaw/github

  • "Post eval score diff as a PR comment"
  • "Block merge if eval score drops below threshold"
  • "List all eval regressions since the last release"

AI Eval Engineer · clawhub.md/expert/eval-expert

← Experts Browse all skills →