APRA Q&A Agent
A question-answering assistant for dense banking regulation. It answers only from the official standards, always cites the exact source — and says "I don't know" rather than guessing when the answer isn't there.
Overview
In a regulated field, confidently answering outside the evidence is the failure that matters. So the interesting part of this project isn't retrieval — it's the refusal.
Ask about Australian prudential (APRA) standards and the assistant retrieves the relevant standards and answers from them, citing the source standard and clause. Ask something plausible but outside the corpus and it declines, instead of answering from the model's own pre-training. That trustworthiness — grounding you can verify — is the product.
The real distance covered is taking a grounding technique from a notebook to a deployed, auditable, cost-capped system — with an eval gate, guardrails, refusal as a first-class path, full observability, and everything in infrastructure-as-code behind OIDC-only CI.
Impact
How it works
- 1
Chunking that respects document boundaries
Each standard is chunked independently (2048 characters, 400 overlap) so no chunk ever spans two documents — preventing the confidently-wrong answers that come from cross-document contamination.
- 2
Retrieval in document order
Passages from the same standard are reassembled in document order before generation, not similarity order, so sequential provisions read as sequential provisions.
- 3
Grounding you engineer, then verify
A strict prompt does the work: establish role and register, restrict answers to the retrieved context, and require explicit refusal when the answer isn't there. Removing the source restriction is the whole difference between a grounded answer and a confident ungrounded one.
- 4
An eval gate in the pipeline
Deployment is blocked if the system's grounding behaviour regresses — refusal on out-of-corpus questions is measured, not assumed.
- 5
Deployed to be trusted
Serverless on AWS, cost-capped, fully observable, and defined in Terraform behind OIDC-only CI — with a streaming chat UI that shows the source documents alongside each answer.