A multi-tenant Retrieval-Augmented Generation platform for a healthcare product — tenant-isolated vector stores, secure authentication, and strict data partitioning, taken from zero to production and load-tested for 2,000+ concurrent sessions at sub-second latency.
SYS-02 · REAL-TIME VOICE + TEXT LLM · unscrew for spec ⟳
Clinical Simulation Platform
An LLM-powered clinical simulation platform for physician training — prompt-engineered patient personas driven by authored case content, over real-time text and voice, plus an automated feedback report that scores the trainee. Load-tested for 3,000 concurrent sessions at sub-second latency.
SYS-03 · MULTI-AGENT SDLC AUTOMATION · unscrew for spec ⟳
DevFlow Kit
Multi-agent SDLC automation that turns Jira tickets into production PRs with zero added infrastructure. Refinement, implementation, and Jira-sync agents decompose complex tickets into parallel subtasks and cut the ticket-to-PR cycle from days to hours.
Multi-agent regulatory compliance automation. Feed it a regulatory PDF and your control matrix — a compliance research agent extracts every obligation, a gap analyzer checks each against your policies via RAG and scores the risk, and a report generator produces an audit report with a human-in-the-loop approval gate. Includes a drift-detection evaluation harness.
Open-source, self-hosted LLM cost tracking SDK. A drop-in instrumentation layer wraps OpenAI, Anthropic, Groq, and Bedrock clients to record usage straight to ClickHouse or PostgreSQL — real-time token metering, per-request cost attribution, and a bundled analytics dashboard.
Cloud-agnostic resource monitor with ML-powered waste detection. Flags idle instances, unattached volumes, and stale snapshots across providers in a unified cost-optimization dashboard — then eliminates them safely with dry-run previews, human-in-the-loop approval, and 7-day rollback.
Architectural trade-offs, and how I actually decided.
When is RAG the wrong tool?
When you already know exactly what context the model needs.
The clinical simulation platform has no RAG in it, on purpose. Every turn
needs the same two things: the persona and guardrails, and the specific case
being simulated. Neither is a search problem. The system prompt is fetched
and version-managed from LangSmith, the case JSON comes from a CDN, and the
service assembles the prompt from both.
Retrieval would have added latency to a voice conversation that can't afford
it, and a failure mode — retrieving the wrong passage — in exchange for
solving a problem I didn't have.
How do you guarantee one tenant never sees another's data?
By making tenant a property of every layer, so there's no single filter to
forget.
On the healthcare platform, documents land in per-tenant Blob Storage
containers and are embedded into a per-tenant Azure AI Search index — the
data is separated before any query exists. At request time Azure AD B2C
establishes tenant identity and claims, and a tenant filter is applied on
every retrieval. Conversation history sits in Cosmos DB, a collection per
tenant, so isolation covers memory and not just retrieval.
The point of doing it at every layer is that no one code path can leak by
itself. And it was verified under load, not just in tests — isolation that
only holds at low concurrency isn't isolation.
How do you keep a multi-agent system from compounding its own errors?
You test it like a system, not like a chatbot.
In RegLens every stage — ingestion, retrieval, gap analysis, risk scoring,
routing — has component evals against a labelled golden dataset. On top of
those are behavioural tests that check properties rather than exact outputs:
monotonicity, so a more severe input never scores lower risk; invariance, so
irrelevant rewording doesn't change the verdict; and counterfactuals.
A single accuracy number hides silent regressions. Property tests don't.
When do you reach for ML, and when are rules enough?
Rules first. I reach for ML when the boundary isn't a threshold.
In Cloud Waste Hunter, idle EC2 detection uses an Isolation Forest, because
"idle" isn't one clean cutoff — utilisation patterns vary between workloads,
so outlier detection works better than a fixed number. Unattached EBS
volumes over 30 days and snapshots over 90 days with no associated AMI are
plain rules, because those genuinely are thresholds.
Using ML there would add a model to maintain and explain, and buy nothing.
Being able to preview it, approve it, and undo it. The detection is the easy
half; the safety design is what makes anyone actually press the button.
Cloud Waste Hunter never deletes straight from a finding. Every action can
be dry-run first, showing exactly what would go and what depends on it. A
human confirms each execution — there is no fully automatic destroy path.
Everything is written to an audit log: who approved what, when, and what
happened. And executed actions stay reversible for seven days, because
snapshots are taken before deletion.
A tool that deletes cloud resources and can't answer "who did this and can
we undo it" doesn't get installed.
The requirement I cared about most in DevFlow Kit was that there'd be
nothing new to run — no server, no queue, no hosted service to pay for and
secure. The pipeline executes on GitHub Actions you already have.
Same in CostTracker. Production targets ClickHouse because per-request
analytics is a columnar workload, but requiring ClickHouse just to try the
tool would put people off, so the same schema runs on PostgreSQL locally.
And instead of hand-maintaining LLM prices it syncs from LiteLLM's public
pricing data — 1,800+ models, no price file for anyone to forget.