Prompt Optimizer
Give the optimizer a prompt and a handful of labelled examples. It runs a DSPy optimization loop, returns an improved prompt, and shows the before, the after, and the scores that justify the change.
One real run
What the optimizer actually returns
This is a single run captured from the service, not an illustration. The interesting part is how the starting prompt fails: it does not pick the wrong queue, it ignores the routing job and answers the ticket as a support agent would.
- Task:
- Route a support ticket to one of four queues. The rule is deliberately not guessable from the label names: sales means money has not moved yet, billing means it has, and incident means many users are affected rather than one.
| Ticket | Before | After | Correct queue |
|---|---|---|---|
| You billed my card after I cancelled. | I sincerely apologize for this billing error. I understand how frustrating it is to be charged after cancelling… | billing | billing |
| Every request from our region is timing out right now. | Thank you for reporting this critical issue. I understand that all requests from your region are currently tim… | incident | incident |
| Is there a nonprofit rate before we sign up? | Thank you for your interest in our services for your nonprofit organization! Yes, we do offer special pricing… | sales | sales |
| My saved filter disappeared after I renamed it. | Thank you for reporting this issue. I understand how frustrating it is to lose a saved filter, especially afte… | support | support |
The starting prompt replies to the customer instead of routing.
The optimized prompt returns the queue and nothing else.
Route the support ticket to one queue: billing, sales, incident, or support.
Ticket: {ticket}
Queue:Examples:
Example 1:
Ticket: I cannot change my avatar in settings.
Reasoning: This ticket describes a user experiencing an issue with a specific feature (changing avatar in settings). This is a localized problem affecting an individual user's ability to use a particular functionality. It's not a widespread outage or system-wide issue that would constitute an incident. Instead, it's a support request where the user needs help troubleshooting or resolving their specific problem with the avatar change feature.
Response: support
Example 2:
Ticket: Can we get a trial extension before we commit?
Reasoning: This ticket is asking about a trial extension before making a commitment/purchase. This is a sales-related inquiry about extending a trial period, not a technical support issue or a system incident. The customer is in the evaluation/decision-making phase and needs to discuss commercial terms with the sales team.
Response: sales
---
Let me think step by step to solve this task.
Ticket: {ticket}
Reasoning: Let me analyze this step by step.
Response:12 labelled tickets in, scored on 8 the optimizer never saw.
8 of 8 held-out tickets routed correctly after optimizing.
Captured from a live run. Re-measured whenever the optimizer changes. anthropic/claude-sonnet-4.5 · DSPy 3.2.1 · BootstrapFewShot