The challenge
Support tickets arrived in English, Hindi, and a mix of the two, and had to be routed to one of 34 queues. The prompt-based router misread product names and had no idea of the team's internal reason codes.
Financial ServicesIllustrative
Routing 40k support tickets a month with an 8B model
A digital lender. A frontier-API prompt pipeline was accurate enough to demo but too slow and too expensive at production volume.
Results
Base model versus shipped model
Measured on the same held-out test set.
Routing accuracy
Before: 71%After: 93%
Cost per 1K tickets
Before: $4.80After: $0.60
p95 latency
Before: 2.4sAfter: 0.6s
The engagement
What we did
We mined 18 months of resolved tickets, had senior agents review 2,000 edge cases, and compared three open-weight 7–8B models. The smallest one that beat the target went through a two-week shadow deployment before cut-over.
The outcome
Agents now start from the correct queue almost every time, and uncertain tickets go to a person with the model's reason attached.
Under the hood
Stack used
- Base model
- Llama 3.1 8B Instruct
- Method
- LoRA SFT on 18k labelled tickets
- Deployment
- vLLM on the client's AWS VPC
Have a similar problem?
Tell us about the task and the data you have. We'll tell you if the same approach applies.
We reply within one business day (IST).