321 AI Labs is the research arm of 321ai LLC. We study the AI systems we and others actually ship and how they fail.
Our working rule: register the study before we run it, pin every artifact to a hash, and publish what we find. That includes the nulls. A result we can't defend to a skeptic doesn't leave the building as a claim, so we try to make sure it never has to.
judge-bias-battery — a pre-registered 741-call battery measuring bias in LLM-as-judge panels. All three audited judges flipped verdicts based on answer order and preferred padded answers above chance. The repo ships the harness, the raw data, and a correction model gated on significance tests, so the numbers can be checked rather than trusted.
Across the lab's studies to date: 7 pre-registered studies, a 105-record failure census from production deployment work, classifier agreement of κ 0.965 on our verification pipeline, and null results published as measured. The full study index and rigor stack live at 321ai.xyz/labs.
- Site: 321ai.xyz
- Research: 321ai.xyz/labs
- Email: hello@321ai.xyz