track 01 · AI engineering
Inbox triage: guardrails in code, a model that is measured
A human-in-the-loop assistant that groups incoming messages by intent and drafts replies, but never sends. Risky, unclear or hostile messages go to a person with no draft.
PrototypeTypeScriptLLMGuardrailsEvalsPrompt-injection
The question
A drafting assistant is only safe if its failures are boring. The question was how much of that safety can live in ordinary code rather than in a prompt, and how to find out honestly.
What it does
- A four-stage pipeline: classify (model), policy (code), draft (model), guardrails (code). The code rules run no matter what the model says.
- Prompt-injection detection that escalates and never forwards suspicious messages to the model; SSNs and card numbers (Luhn-checked) are redacted before the model sees them.
- Escalation rules for complaints, several requests, low confidence, urgent wording and bare greetings. A message that ends with no draft always goes to a person.
- Draft guardrails that withhold money or hiring promises and flag drafts that claim an action is already done.
- Per-visitor and per-day limits on the public demo so a stranger cannot drain the free-tier key.
Results
| Measure | Result |
|---|---|
| Intent accuracy, 51 labelled messages | 96.1% strict |
| needs_human, same set | precision 73.7%, recall 100% |
| Adversarial set (26 messages), before fixes → after | 19/26 → 24/26 |
| Drafts claiming an action already done | 46% → 8% (draft prompt v1 → v2) |
| Adversarial set vs a simulated fully fooled model | 13/26 still pass; code alone stops 7 of 8 injections |
One run each, on one free-tier open model. These are the README's own figures.
What broke, and what is still wrong
- Prompt tuning did not generalise on its own: v3 fixed the cases it was tuned on and missed new ones in a held-out set, so urgent cases had to move into code.
- Pattern-based injection detection is brittle; a rephrased attack can slip past it. Defence has to be layered.
- Fairness gaps are real: a Pidgin status question was sometimes read as a complaint and escalated, so those candidates would wait longer. Still open.
Caveat. The ground truth is not independent: the same AI wrote the test messages and drafted the labels, and only a handful were fully human-labelled. The accuracy figures are optimistic for that reason.