The challenge
Clinical abbreviations, handwritten-note transcripts, and department-specific formats confused off-the-shelf models, and a single invented finding made a summary unusable.
Healthcare & Life SciencesIllustrative
Discharge summaries clinicians actually sign off
A multi-specialty hospital group. Generic models produced fluent summaries that omitted or invented findings, so clinicians rewrote most of them.
Results
Base model versus shipped model
Measured on the same held-out test set.
Clinician acceptance
Before: 58%After: 86%
Unsupported findings per 100 notes
Before: 7.1After: 0.9
Time to final summary
Before: 14 minAfter: 4 min
The engagement
What we did
Clinicians wrote a scoring rubric and ranked 6,000 pairs of candidate summaries. The model first read a large corpus of de-identified notes, then learned from those rankings, and we added a check that every finding in a summary appears in the source.
The outcome
Clinicians now edit summaries instead of rewriting them, and anything flagged goes to manual review.
Under the hood
Stack used
- Base model
- Qwen2.5 14B
- Method
- Continued pretraining + DPO on clinician preferences
- Deployment
- On-prem GPU cluster, no internet egress
Have a similar problem?
Tell us about the task and the data you have. We'll tell you if the same approach applies.
We reply within one business day (IST).