Case study
Case Intelligence Backend
Async AI backend for a case-management platform — summarization, transcription, a booking assistant, and a constraint-solver day planner.
- Role
- Primary developer and top committer
- Domain
- Generative AI & LLMs · Automation & Agents · Backend & APIs · Cloud & Production
- Stack
- Python · Flask · RabbitMQ · OpenAI · tiktoken
- Result
- ~160s case summarization
Overview
The AI backend behind a case-management platform used by caseworkers. It summarises long case records, transcribes and analyses visit recordings, answers questions and books appointments through a conversational assistant, and plans each worker's day with a constraint solver.
The problem
A caseworker's day goes on paperwork around the work: reading a long case history before a meeting, writing up a visit afterwards, booking the next appointment, and deciding which of a growing backlog to tackle first. Each is slow, repetitive and important to get right.
The platform underneath
Every expensive job runs off the request path. The API validates the request, puts it on a message queue and acknowledges immediately; dedicated workers do the work and report back by webhook. That architecture predates my work on it — the capabilities below were built on top of it, which is why each one is asynchronous.
Four capabilities
Case summarisation
A long, deeply nested case record becomes a readable summary.
- Record flattened to text, then sized in tokens before anything runs
- Size and complexity choose the model and one-shot vs map-reduce
- Cost estimated up front, and available on its own endpoint
- Output shaped by audience and template, not one fixed format
Booking assistant
Staff ask about a case or book an appointment in plain language.
- Tool calling against the live platform API — look up, then book
- The caller's own credentials passed through per request
- Checks the service belongs to the client before booking
- Detects conflicts and asks before double-booking
Day planner
A worker's backlog becomes a scheduled day in their free time.
- Nine kinds of outstanding task normalised into one model
- Scored, then placed by a constraint solver
- Respects calendar, work hours and personal focus preferences
- Falls back to a greedy scheduler rather than returning nothing
Visit recordings
A recorded meeting becomes a transcript, a summary and a filled form.
- Transcription via an external speech-to-text provider
- Separate summary and sentiment passes
- Arbitrary form fields extracted from the transcript by type
- Each stage its own queue, so one can fail without the others
Inside the day planner
The one capability that is optimisation rather than prompting. Given a worker's backlog and calendar, it has to decide what to do today and when.
Normalise
Nine task types — draft notes, unsigned forms, appointments missing a write-up — become one shape, each carrying its age.
Score
Task type weighs most, then urgency from age, with a small nudge towards work on the same case. Priority and recently active cases get a boost.
Find the free time
Working hours minus calendar events and breaks, cut into slots.
Solve
A CP-SAT model places tasks: each task at most once, each slot at most once, longer tasks in contiguous slots. The objective rewards priority, doing important work early, matching the worker's energy and focus preferences, and batching one case's tasks together.
Fall back
The solver's time limit scales with the size of the backlog. If it times out or returns something degenerate, a greedy scheduler produces a plan instead — a worker always gets a day, even when the optimum is out of reach.
My Role
Role
Primary developer and top committer
Contribution
- Built a Flask API fronting a RabbitMQ worker fleet so expensive AI jobs run asynchronously rather than blocking requests
- Implemented cost-aware model selection — token-counting picks the cheapest model and one-shot vs map-reduce strategy that will still handle the case
- Built audio transcription with summarization and sentiment passes, and auto-filled forms from transcripts
- Built a function-calling assistant that books appointments against the live platform API with per-request dynamic auth, service-to-client validation, conflict detection and anti-hallucination guards
- Built a daily task planner on OR-Tools CP-SAT with multi-factor priority scoring, energy and focus preferences, a greedy fallback and adaptive timeouts
- Shipped as Dockerized cloud services through CI/CD
Team Context
Primary developer on the AI backend, within a multi-contributor product repository.
Decisions
Size the job before running it.
Case records vary enormously. Counting tokens first lets a small case take the cheap path and a huge one take the strategy that will actually fit — and makes cost a number that exists before the money is spent, not after.
- Instead of
- One model for every case
- Always map-reduce
A constraint solver, with a greedy fallback.
A heuristic cannot weigh six competing preferences at once; a solver can, but offers no guarantee on time. Pairing them gets the optimum when it is reachable and a sensible plan when it is not — and the time limit scaling with the backlog keeps small days instant.
- Instead of
- A heuristic alone
- The solver alone
Validate in code around the assistant, not only in the prompt.
Booking an appointment is a commitment to a real person. Whether a service belongs to a client, or a slot conflicts, is checked by the tool implementation before anything is written — the model decides what to do, code decides whether it is allowed.
- Instead of
- Trust the model to follow its instructions
A small in-house tool registry.
The assistant needed two tools, not a framework. A registry of a few dozen lines kept the tool-calling contract fully visible and free of a dependency that would have outweighed the feature.
- Instead of
- Adopt an external tool-protocol library
Results
~160s
case summarization
15+
REST endpoints shipped
50+
cases validated
How it is measured
- Runtime
- Summarisation time averaged across real runs, reported from use rather than a saved benchmark.
- Validation
- Functional testing across more than fifty real cases, by hand.
- Not measured
- Summary quality and transcription accuracy were never scored.
What I Learned
- Put a price on the work before doing it. Sizing a case first turned model choice from a guess into a rule, and cost into something visible up front.
- Optimisers need a floor. The solver is only useful because the greedy fallback guarantees an answer; without it, a hard day would produce an empty plan.
- When a model acts on the world, the checks belong in the tools. Prompt rules describe good behaviour; code enforces it.
Limits
Runtimes are reported from real use, not a benchmark with a saved log. Validation across fifty-plus cases was functional and manual — there is no automated test suite. Summary quality and transcription accuracy were never scored against anything, so how good the output is rests on review rather than measurement. Deployment evidence points at a UAT environment, so nothing here is presented as production-scale.
What I'd Improve
- Score summary quality against summaries caseworkers write themselves. Right now quality is reviewed, not measured. First fix.
- Automated tests around the booking assistant — the capability that makes commitments to real people.
- Persist assistant conversations. They live in process memory today, so they vanish on restart and do not follow a user across instances.
- Benchmark the planner on real backlogs with a saved log, so its runtime is evidenced rather than reported.