- AI dependencies mapped to the critical operations they support
- Tolerance levels agreed for each AI-assisted operation
- Failover to a second provider or a rules-based mode, tested in production
- An exit plan for the primary model provider that has actually been rehearsed
- Board reporting that names AI providers alongside other material suppliers
Anonymised composite. Client names, identifying details and confidential figures are withheld; public facts are cited in the references.
The situation
A general insurer in Australia had moved quickly with AI. Claims intake used a hosted model to read documents and photos, the contact centre used it to draft responses, and the fraud team used it to summarise case files. Almost all of it ran through one external model provider, wired directly into each application.
CPS 230, APRA's prudential standard on operational risk management, has applied to banks, insurers and superannuation entities since 1 July 2025, with existing service provider contracts brought in at renewal or by 1 July 20261. It expects entities to identify critical operations, set tolerance levels for disruption, and manage material service providers2. The board asked a fair question: if the model provider has an outage, runs out of capacity or changes terms, what happens to claims?
Why now. Capacity is not a theoretical risk. Reporting on Microsoft's latest results notes that demand for its cloud still exceeds supply4, and AI capacity is the tightest part of that market. See the tech news update for 29 October.
What had to be true
- AI dependencies visible in the same register as other material service providers.
- Tolerance levels for each AI-assisted critical operation: how long, how degraded, before customers are harmed.
- A tested alternative for every critical use, not a slide.
- Privacy obligations met on every path, including fallbacks.
- No rebuild of the applications: changes behind a common interface.
Architecture
The data flow, end to end
- RequestAn application calls the gateway with an operation and criticality tag
- RouteThe gateway picks the primary provider within budget
- DetectErrors, latency or capacity limits breach thresholds
- Fail overCritical traffic moves to the second provider or private model
- DegradeLower-priority work queues or falls back to rules and people
- RecordEvery switch is logged against the operation's tolerance level
The hard part: deciding what to drop when capacity is short
Not every AI use is equally important, and pretending otherwise means everything fails together. The team tagged each request with the criticality of the operation it served, so the gateway could protect claims intake and shed or delay lower-priority work. The same idea appears in the Kubernetes Gateway API inference extension, where scheduling considers whether a request is critical and whether there is capacity before choosing a model server, and can drop non-critical requests when there is not5.
Resilience for AI is mostly a prioritisation decision. Decide in advance which work gets the last available capacity.
Every fallback path also had to meet privacy obligations. The OAIC's guidance on commercially available AI products expects due diligence on each product and ongoing review, not a one-off check3, so the second provider and the private model went through the same assessment as the primary. Routing patterns are covered in LLM model routing and Kubernetes inference gateways.
How it ran
- Critical operations register
- Tolerance levels
- Board reporting
- Model gateway
- Second provider and private model
- Telemetry and switches
- Degraded-mode playbooks
- Human queue capacity
- Scenario exercises
- AI uses and providers
- Link to critical operations
- Tolerance levels
- Model gateway
- Move apps behind it
- Telemetry per path
- Second provider
- Private model
- Degraded modes
- Failover drills
- Exit rehearsal
- Board reporting
Key decisions and trade-offs
| Decision | Alternative | Why |
|---|---|---|
| A single model gateway | Provider SDKs in each application | Failover and exit become configuration, not rewrites |
| Second provider from a different vendor | Second region with the same vendor | Reduces concentration on one company's terms and capacity |
| Small private model as a floor | Hosted providers only | A guaranteed baseline for the most critical step |
| Criticality tags on every request | Treat all AI traffic the same | Protects claims when capacity is short |
| Rehearse exit with live traffic | A written exit plan | Only tested plans count |
What I'd do again, and differently
Again: put a gateway in front of every model call before adding a second provider; it made everything else cheap. Differently: build the evaluation set for each critical operation first. The slowest part of adding a provider was proving its answers were good enough, not wiring it in. See evaluation sets that predict production and AI incident postmortems.
If you run AI in a regulated Australian business, my work with Australian companies explains how I approach it, or analyse your challenge and bring the result to a call.
- 1
- 2
- 3
- 4Microsoft's cloud growth accelerates, and so too does the spendingFortune · 28 Oct 2026
- 5Introducing Gateway API Inference ExtensionKubernetes blog · 5 Jun 2025