Stanford deployed LLM vancomycin dosing agents on 403 real orders; the higher-autonomy version halved the error mixedPreprintSingle source
- The authors say they "silently deployed PharmaAgent, LLM-based vancomycin dosing agentic systems, at Stanford Hospital for two months," comparing high- and low-autonomy configurations. "Across 403 dosing events in 316 patients, mean absolute percentage error against the treating pharmacist's order was 21.8% with high autonomy versus 37.8% with low autonomy."
- Recommendations "were within 25% of the order in 73.4% versus 50.9% of events (P < 0.001)," against a human benchmark the paper also reports: "a second pharmacist was within 25% in 90.2%." So the better agent still trailed a second pharmacist by a wide margin.
- The counter-intuitive finding is that tighter engineering performed worse: pharmacist review of 196 out-of-tolerance recommendations "found a safety concern in 65.8%, more often with low autonomy (odds ratio 2.37)." The authors conclude "A higher-autonomy agent that self-debugged its intermediate steps proved more robust than stricter engineering."
- The authors state plainly that "neither configuration is ready to dose patients without pharmacist oversight." The deployment was silent — recommendations did not reach patients — and this is a preprint, posted 29 October 2026, not peer reviewed; corresponding author Jonathan H. Chen, with Andrew Y. Ng among the co-authors.