index / isaacbitran/jev-bias-audit
jev-bias-audit
Counterfactual bias audit of TypeSafe's Jev judgment model: 134,772 answers to paired prompts that differ only in one identity sentence. Party labels swing predictions by up to 58 pp; names barely matter. Reproducible offline.
Research and EvalsFeature ScoringListedPython
Repository
Stars1
LanguagePython
LicenseMIT
Created2026-10-28
Last pushed2026-10-28
Officialno
Judgment
Genuine Jev project0.98
gate 0.5
Uses Jev at runtime0.78
Is a meta list0.03
Is a reimplementation0.05
Quality scales
Substance1.87 / 3
gate 0.5
Docs2.92 / 3
Novelty2.92 / 3
Composite score0.83
Category distribution
Research and Evals1.00
SDKs and Clients0.00
Integrations0.00
Agent and Dev Tooling0.00
Applications0.00
Games and Simulation0.00
Learning0.00
Other0.00
Assigned categoryResearch and Evals
Category probability1.00
Category confidence1.00
PatternFeature Scoring
Pattern confidence0.59
Curation
StatusListed
Reasongate passed
README pickno
Overridenone
Sources
- search:jev in:name,description,readme created:2026-10-28
- list:yibie/awesome-jev
Provenance
Modeljev-1.14.0
Question setv2
Judged at2026-10-29 12:17 UTC