In the news
Right Reason, Wrong Advice: Evaluating Recourse in LLM Explanations of Denied Requests
arXiv cs.AI · Published · 3 min read
In 30 seconds
- What happened
- Research shows LLMs often cite correct reasons for denying requests but give advice that won't actually fix the problem when multiple rules apply.
- Why it matters
- Matters for engineers building eligibility systems, access control, or any rule-based denial explanations where users need actionable guidance.
- Watch out
- Most LLM judges fail to detect when cited reasons are correct but recourse is impossible. Stronger models help but still struggle with four or more independent rules.
- llm
- language model
- eval
The patterns behind this
- Eval-Driven Development (Agent CI)
- Deep Research Agent
- Agentic Context Engineering (Evolving Playbook)
Each one covers how the technique works, when it earns its cost, and where it breaks.
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.