Explain evaluation quality beyond a single score
Not a dashboard — an audit journal. Each conclusion carries real data and the records behind it. Produce a team-detailed version, a management brief, or a data-supplier delivery report, with AI findings, human-confirmed items, and open issues clearly separated.
Evaluation Reliability
Open records →
Overall agreement after rubric v13, weighted by criterion. Confirmed by expert review on contested pockets.
Reviewer Differences
Open records →
Concentration of disagreement by reviewer pair and criterion. Accuracy drives most contested ratings.
Rubric Ambiguity
Open records →
Rules flagged by the Ambiguity Lens with the disagreement each one explains.
Edge Case Coverage
Open records →
Coverage of known failure modes by the current evaluation set and gold references.
Gold Set Status
Open records →
Size, version, and requirements met by the current trusted reference dataset.
Recommended Actions
Open records →
Prioritized next steps — separated into AI-suggested, human-confirmed, and still-open.
Find what makes your AI evaluations unreliable.
Point AnnotRift at your evaluation set and get a traceable quality report in minutes — every finding backed by data and rules, every decision confirmed by a human.