Oak Ridge National Laboratory · HUMAN-AI EVALUATION
How might we know when AI is ready to be trusted?
I've studied how people over- and under-trust AI since my PhD, and my rule for AI is the same at every scale: check it against human judgment before trusting it, and put guardrails on what it claims, not how it sounds. At Oak Ridge National Laboratory, I've applied that rule at rising scale and stakes, from one experiment to a release gate a development team now runs without me.
BY THE NUMBERS
- 90,000+AI assistant user messages analyzed with a human-validated method
- 50+studies formatted via AI & published in the research repository
- 8accuracy risks caught before a high-risk model's release
- 2models benchmarked against human coding
- DATE
- 2024 -
- ORGANIZATION
- Oak Ridge National Laboratory
- ROLE
- AI evaluationExperimental designLLM-assisted analysisAI product designAI safety consulting
- DELIVERABLES
- IRB-approved trust experiment and model benchmark, severity-ranked rubric adopted as internal guidance, human-validated LLM coding method, analysis of ~90,000 AI assistant prompts, grounded conversion agent for a research repository, release-gating evaluation framework
What did I do?
- 01
The trigger: people trusted AI more than humans, even when it was wrong
I designed a two-part, IRB-approved study of AI in research reporting. First, I benchmarked Claude and GPT-4o against human-coded findings using a severity-ranked rubric for hallucination, omission, and oversimplification. Claude matched human coding most closely; GPT-4o was the most confident and most often wrong. Then a source-blind experiment showed participants trusted AI summaries more than human ones, including the wrong ones. The rubric now governs internal guidance for AI use in sensitive research reporting.
- 02
Into practice, then at scale: AI as a checked second coder
I now use Claude as a second coder on large qualitative datasets, checking its codes against human coding before relying on them. That method lets me do things like quickly analyzing ~90,000 prompts from the org's internal AI assistant, triangulated with 127 user feedback submissions. It showed the tool's director how staff actually use it: writing is the top use, and about half of conversations are iterative. Where logs couldn't answer a question, I added follow-ups to the AI Awareness survey.
- 03
Built into tools: AI that flags gaps instead of guessing
When building my team's research repository, which has published over 50 studies so far, I designed a conversion agent that turns findings decks into searchable records, so researchers publish without engineering help. Where information is missing, it writes a 'TO-DO' instead of guessing. A verifier rejects any AI-quoted finding that doesn't match the source word for word.
- 04
High stakes: a release gate for a deployed model
The developers of EAGLE-I, a power-outage monitoring platform, fine-tuned an LLM to summarize its dashboards, a high-risk use. I audited outputs and surfaced 8 accuracy risks before release. I then delivered guardrails, design recommendations, fine-tuning training data, and a release-gating evaluation framework: release-blocking checks, repeated sampling, independent dual raters, and prompt-injection tests. The development team now runs it on its own.
Each step used the same check, AI measured against human judgment with guardrails on what it claims, at larger scale and higher stakes. The last step matters most: enabling teams to run high-stakes AI evaluations without me.