April 2026
Accuracy Is the Floor, Not the Ceiling
Why 93% accuracy still leaves the failure boundary unknown, and what calibration, constraint adherence, and consequential error rate actually measure.
Matt Nash
AI systems fail in predictable ways. I build evaluation frameworks and constraint architectures so the system earns trust before it reaches users, not after.
- Adversarial red teaming that finds what benchmarks miss
- Constraint architecture that holds under real-world conditions
- Welfare measurement frameworks for responsible AI deployment
I started in product leadership, shipping systems that had to work in the real world. Over time, I became less interested in feature velocity and more focused on a harder question: how do you know this system is safe, reliable, and ready to deploy? Not whether it clears a benchmark. Whether it works in the context where it will be used, under real-world conditions, against adversarial inputs, and for the people who will depend on it.
That question defines the practice. Evaluation criteria are set before system design begins. Failure modes are mapped as architectural inputs, not post-launch discoveries. Constraints, what the system must not do, are specified as rigorously as capabilities. The hardest problems in AI are not purely technical. They are epistemic: knowing when a system is not ready, and having the discipline to hold that line.
My undergraduate training is in social sciences: anthropology, sociology, and psychology. That background informs how I approach AI welfare research and behavioral red teaming, where the hardest problems are behavioral and measurement problems before they are engineering problems. How do you operationalize a construct like "model welfare" into something measurable? What does construct validity mean when applied to consciousness indicators in a system with no agreed phenomenology? These are social science questions, and the field is early enough that methodology from adjacent disciplines carries real weight.
I am building a research foundation in AI welfare measurement, applying measurement theory and construct validity frameworks from the social sciences to welfare indicators that currently lack agreed evaluation standards. The work builds toward an original measurement framework paper in 2027.
Latest writing
-
-
March 2026
The Valence Gap: What AI Welfare Frameworks Cannot Measure
Current AI welfare frameworks are built to detect access consciousness, not valenced experience. That is the wrong target. Here is what a better framework would require.
-
March 2026
The Requirement Is Not the Design
Every AI governance framework says human oversight is required for irreversible actions. That requirement, on its own, does nothing. Here is the difference between policy and architecture.