Judge-dependent safety gains and model-specific helpfulness costs of evidence-sufficiency prompting in clinical LLMs

За 30 секунд

Что произошло

A prompt technique reduced unsafe overconfidence in clinical AI models from 49% to 25%, but the safety gain heavily depends on which AI evaluates the results.

Почему это важно

Engineers building clinical AI systems need to know that safety metrics from AI judges may not reflect real behavior change and require human validation.

На что обратить внимание

The measured safety improvement vanished by half when a different AI judge scored the same answers, and helpfulness dropped severely for some models but not others.