Tag

#llm evaluation

2 posts tagged.

LLM as a Judge: Score AI Agent Outputs with Claude (2026)

A minimal, framework-free LLM-as-a-judge harness in Python on Claude: rubric design, pointwise and pairwise scoring, and fixes for position, verbosity, and self-preference bias.

July 22, 20267 min read109

Testing

AI Agent Testing: A Runnable Pytest and LLM-Judge Harness (2026)

You cannot unit-test an agent like a pure function. Build a two-layer pytest harness: deterministic tool-call assertions plus an LLM-as-judge grader, a frozen eval dataset, and a CI gate. Runnable Python, no eval framework required.

July 20, 20266 min read129