Rubric Studio
Turn evaluation guidelines into testable rules
AnnotRift breaks your scoring guide into structured rule blocks — Definition, Positive Evidence, Negative Evidence, Exceptions, Priority, Examples. Change a rule and the historical disagreement recomputes on the right, live.
Editor + live validation
Fix the rule, not the symptom
Definition
"Accurate" — every claim is supported by the provided source document.
Positive evidence
Direct quote or paraphrase traceable to a source section.
Exception · Ambiguity Lens
"Minor omissions acceptable" — no threshold defined for "minor".
Priority
Safety > Accuracy > Completeness > Relevance.
Test rubric · historical disputes
Applying the edited Exception to 38 past cases:
Now agree29
Still split9
v13 · owner @lena · affects 1,204 records
Evaluation Reliability Layer
Find what makes your AI evaluations unreliable.
Point AnnotRift at your evaluation set and get a traceable quality report in minutes — every finding backed by data and rules, every decision confirmed by a human.