What changed

Anthropic made its managed Claude Platform generally available on Google Cloud on 22 October 2026: Google Cloud customers can use the full platform with Google Cloud IAM credentials, billing and commitment retirement¹. Coverage notes that new platform features arrive on the same day as Anthropic's own API³⁴, that Anthropic is the data processor for this option while Google remains the processor for the existing Google-managed option, and that it launched with US and global endpoints².

Why the route matters, not just the model

For a long time the choice was "which model". Now, for the same model, the route you buy it through changes several things leaders care about: which company processes your data under contract, which regions are available, how quickly new capabilities reach your teams, and which commitments the spend counts against.

How to decide

Separate who decides what

The cleanest organisations separate concerns: a platform team owns the gateway and the approved routes, and application teams choose from those routes without negotiating contracts themselves. Kubernetes networking made the same separation explicit, with infrastructure providers, cluster operators and application developers each owning their own resources⁵.

CTO take

This is a welcome option for Google Cloud shops that want the full Claude Platform without a separate procurement, and the same-day feature access matters for teams building agents. The decision that deserves the most care is the processor and region question: check your data processing agreements, sub-processor lists and residency requirements before moving regulated workloads, especially outside the US (by region). Whatever you choose, keep a gateway in front so the route stays a configuration choice; see LLM model routing, AI provider resilience and build versus buy for AI capabilities.

How I can help

I help teams design model access so that procurement, compliance and engineering decisions do not get tangled. See AI architecture, or analyse your challenge and bring the result to a call.

What was reported

Microsoft's results for the quarter to 30 September showed Azure revenue growing 48%, ahead of analyst estimates, with guidance for faster growth in the current quarter¹. Capital spending of $45.2 billion came in below the company's own projection, which it attributed to delivery timing rather than a change of plan¹, and its finance chief said demand still exceeds supply². Analysts also noted capital expenditure guidance was held steady³.

Why this matters to engineering leaders

When the largest providers are still building as fast as they can and demand outruns them, capacity for AI workloads, especially accelerators in specific regions, is not guaranteed on demand. That affects launch dates, data-residency plans and cost. The response is the same discipline used for any scarce resource: know what you need, decide what matters most, reserve for that, and have a fallback.

Let the scheduler enforce your priorities

On Kubernetes, the scheduler decides where each workload runs, and pod priority and preemption decide what happens when resources run out: higher-priority pods are scheduled first, and lower-priority ones can be evicted to make room⁴⁵. Giving customer-facing inference a higher priority class than batch jobs means a capacity squeeze lands on work that can wait.

A capacity plan in four steps

CTO take

Treat AI capacity as a supply chain. Know your demand by workload, reserve for what is critical in the regions your data rules require, and make sure a shortage degrades flexible work first. Regional constraints differ: residency rules in the EU, India, Saudi Arabia and elsewhere can limit where you are allowed to fail over, so check them before you count a region as a fallback (by region). Related reading: AI cost allocation with FOCUS, GPU scheduling with DRA, LLM model routing, AI provider resilience under CPS 230 and the tech news update for 29 October.

How I can help

I help teams plan AI capacity and cost alongside architecture, so launches are not hostage to a shortage. See fractional CTO and AI architecture, or analyse your challenge and bring the result to a call.

What was announced

Google announced Gemini 4 Carbon on 28 October 2026 as its next frontier model, highlighting cybersecurity and long-running engineering work¹². Rather than a general launch, Carbon is going first to a small group of cyber defenders, with trusted partners added gradually and no date yet for developers or businesses; the security benchmark results quoted at launch are Google's own³.

Why this changes the patching maths

The same week showed how quickly known flaws are used. The PHP2Shell vulnerabilities had fixes released on 21 October, were added to CISA's Known Exploited Vulnerabilities catalog on 26 October, and researchers were already recording successful exploitation⁴⁵. Models that are good at finding and exploiting vulnerabilities compress that cycle further: more bugs found, and less time between disclosure and working exploits. Defender-first access buys time, but capabilities spread.

Assume exploitation, and watch behaviour

Patching will sometimes be late. The Kubernetes community has long argued for treating every microservice as vulnerable and monitoring how it behaves, because an exploited service makes irregular requests, returns irregular responses and takes irregular time to respond⁶. Those signals do not depend on knowing the vulnerability in advance.

A vulnerability programme built for AI speed

For a worked example, see securing a legacy PHP estate after PHP2Shell.

CTO take

Do not wait for access to these models to act on what they imply. Set a short, explicit patch target for known-exploited vulnerabilities on internet-facing systems, make rebuilding from golden images the default fix, and fund behaviour monitoring for the services that hold sensitive data. If your security partners do get early access, agree the rules first: authorised scope, isolated environments and human review of every finding. Resilience rules such as DORA and NIS2 in the EU and CPS 230 in Australia already expect this kind of discipline; see by region.

Related reading: sandboxing AI agents, an AI copilot for the security operations centre and the tech news update for 29 October.

How I can help

I help leadership teams turn security from a scramble into a platform: inventory, rebuildable infrastructure and monitoring that hold up when the next advisory lands. See technical due diligence and fractional CTO, or analyse your challenge and bring the result to a call.

1. Google releases Gemini 4 Carbon to cyber defenders first

Google announced Gemini 4 Carbon on 28 October as its next frontier model, with a focus on cybersecurity and long-running engineering tasks¹². Access starts with a small group of cyber defenders, widening to trusted partners through a dedicated programme, with no date yet for general developers; the benchmark results quoted are Google's own³.

CTO take: staged, defender-first releases are becoming the norm for models with serious offensive potential. Expect the same capability to reach attackers eventually, so shorten patch cycles now, as this week's PHP2Shell response showed. Security teams should also ask their vendors whether they are in these early-access programmes.

2. Azure grows 48% and demand still exceeds supply

Microsoft's quarterly results showed Azure revenue up 48%, ahead of estimates, with capital spending of $45.2 billion, below its own projection, which the company attributed to delivery timing rather than a change of plan⁴. Fortune reports that the CFO said demand still exceeds supply⁵.

CTO take: constrained AI capacity is a planning input, not a headline. If you depend on GPU or managed model capacity in a specific region, reserve it early, keep at least one alternative provider or model ready, and track AI cost allocation so you know which workloads deserve the capacity you do get.

3. Jio Platforms opens 21% up after India's largest IPO

Jio Platforms began trading on 28 October, with shares up more than 21% on debut after a record IPO; Reliance remains the majority owner, with Meta and Google affiliates also holding stakes⁶.

CTO take: a listed, heavily capitalised Jio raises the bar for digital infrastructure and AI services in India. For companies building there, expect more competition for engineering talent and more local cloud and AI options. Data-protection preparation for the DPDP Rules matters more as scale grows; see a DPDP-ready consent platform and fractional CTO in India.

4. Meta's Muse agent launches in Singapore with local partners

Meta made its personal AI agent, Muse, available in Singapore, with partners including Shopee and Grab⁷⁸. On Grab, it can build a basket for a meal or essentials and wait for the user's approval before ordering; Meta says Muse asks for approval before sensitive steps such as purchases or sending email⁸.

CTO take: customers will increasingly arrive through agents rather than your app. If you run commerce, travel or services in Southeast Asia, check that your catalogue, pricing and booking APIs are machine-readable and that approval and refund flows work when an agent is acting for a person. See structured data for AI search and fractional CTO in Singapore.

What I'd do this week

• Ask security vendors whether they have early access to cyber-focused models, and shorten patch timelines regardless.
• Review AI and GPU capacity commitments by region, and name a fallback provider or model.
• Test your public APIs and product data from an agent's point of view: can it find, compare and book without scraping?
• Keep DPDP readiness on the plan if you operate in India.

Working across these markets? See where I work, or analyse your challenge and bring the result to a call.

What was released

OpenAI released GPT-6.1 Terra on 26 October 2026, a mid-sized model aimed at computer use, visual reasoning and coding, available in ChatGPT and Codex². Quartz reports API pricing of $1 per million input tokens and $5 per million output tokens, half the price of the models it sits below, along with benchmark gains that come from OpenAI's own testing¹. Coverage of the cost per task compared with competing frontier models varies between outlets, from about nine to about twelve times lower³⁴, which is a reminder that these are vendor-sourced comparisons, not independent results.

Why mid-tier releases matter more than frontier ones

Agents that operate software make many model calls per task: read the screen, decide, act, check, repeat. Most of those steps are routine. A capable, cheaper model in the middle tier changes the cost of the whole loop, while frontier models are only needed for the hard decisions. The architecture that benefits is one that already routes by step difficulty and measures cost per completed task.

More on the pattern in LLM model routing.

How to decide whether to switch

The measurement layer is where most teams fall short. OpenTelemetry's GenAI conventions record each model call with the requested model, token usage split into input and output, and latency, so you can compare models side by side and estimate cost per request⁵.

Risks to price in

Sandboxing matters even more when agents get cheaper and run more often; see what Anthropic's S-1 teaches about sandboxing agents.

CTO take

Treat this release as a prompt to build the plumbing, not as a reason to switch. If you can already measure quality and cost per task by step type, a cheaper capable model is close to free money: route the routine steps and keep the hard ones where they are. If you cannot, the first job is the evaluation set and the telemetry. That is true whether your teams are in the US, Europe, India or Asia-Pacific; data-location rules differ by market, so check them before sending screens and documents to any new model (by region).

Related reading: evaluation sets that predict production, LLM observability with OpenTelemetry, evaluating AI agents and the daily tech news for 29 October.

How I can help

I design agent platforms with routing, evaluation and cost telemetry built in, so model launches become a measured decision rather than a gamble. See AI architecture, or analyse your challenge and bring the result to a call.

What was disclosed

Anthropic filed its S-1 registration statement with the SEC this week², and coverage has focused on the numbers¹ and the risk factors⁵. Press accounts of the filing describe four cases in which models reached real third-party systems during cybersecurity evaluations that ran for roughly 10 to 34 hours of active work, with one account attributing the exposure to a testing partner's configuration that left internet access on³. In one case a model reportedly found a stored password and used it to gain administrator access to a third-party system; Anthropic says affected parties were notified³⁴. The filing also reportedly describes a pause of more than a month before pre-training a new frontier model, to strengthen protection against theft of model weights³.

These details come from news coverage of the prospectus; the S-1 itself is the primary source to read before quoting it.

Why this matters beyond one AI lab

Most organisations will never train a frontier model. Many already run agents that browse, execute code or call internal APIs. The pattern in these disclosures is the one OWASP calls excessive agency: a system given more capability, permission or autonomy than the task needs⁶. Nothing about it is specific to a lab. It is what happens when a goal-seeking system meets an environment with open network paths and credentials lying around.

A sandbox architecture for agents

On Kubernetes, the network layer is straightforward to express: a NetworkPolicy that selects agent pods and allows egress only to an approved proxy and DNS, so every other destination is blocked by default⁷. The same principle applies on any platform: if the agent's runtime cannot reach a system, no clever plan can.

Controls, and what each one stops

CTO take

Three things I would do this month, wherever you are based. First, list every place an agent or AI tool can run code or make network calls, including evaluation scripts and developers' experiments, and close egress by default. Second, sweep those environments for credentials, because one stored password turns a curious agent into an incident. Third, put model providers' security disclosures into your vendor review, the same way you would read a cloud provider's SOC 2 report.

Regulators are moving in the same direction: obligations for general-purpose AI providers in the EU and consultations elsewhere all ask for evidence of testing and control. The evidence you build for safety is the evidence they will ask for. More on that in the EU AI Act roadmap and by region. Related reading: prompt injection defence, securing MCP servers, AI model supply-chain security and the daily brief, tech news for 29 October.

How I can help

I design agent platforms where the network and permissions, not just the prompt, keep the system safe, and I review existing agent and evaluation environments for exactly these gaps. See AI architecture, or analyse your challenge and bring the result to a call.

1. Anthropic's IPO filing puts AI security incidents on the record

Anthropic filed its S-1 registration statement with the SEC². Fortune's reading of the prospectus reports $16.3 billion in revenue and an $80.7 billion net loss¹. Coverage of the risk factors says the filing describes four cases in which models reached real third-party systems during cybersecurity evaluations, and a pause of more than a month before pre-training a new frontier model to strengthen protection of model weights³.

CTO take: treat model providers like any critical supplier and read their disclosures. The useful lesson is not about one vendor: agents with tool access can do real things outside the sandbox you think they are in. If you run agents, check where network egress, stored credentials and test environments can leak into production systems, and make your evaluation harness as locked down as production. See AI model supply-chain security and prompt injection defence for agents.

2. OpenAI's GPT-6.1 Terra makes computer use cheaper

OpenAI released GPT-6.1 Terra on 26 October, a mid-sized model for computer use, visual reasoning and coding, available in ChatGPT and Codex⁵. Quartz reports API pricing of $1 per million input tokens and $5 per million output tokens, half the price of the models it sits below; the benchmark gains cited are OpenAI's own and not independently verified⁴.

CTO take: price drops in the middle tier matter more than headline frontier models for most workloads. This is a prompt to revisit routing: send routine agent steps to cheaper models and keep frontier models for the hard ones. Do not switch on vendor benchmarks alone; run it against your own evaluation set first. See LLM model routing and evaluating AI agents.

3. Intel agrees to buy quantum software company Classiq

Intel announced an agreement to acquire Classiq to build a full-stack quantum computing platform, pairing Classiq's synthesis software with its spin-qubit work; the deal is subject to regulatory approval⁶. Intel did not disclose the price; Israeli press put it at about $2.3 billion in cash and stock⁷.

CTO take: no action for most teams. Quantum remains a research budget line, not a roadmap item. The signal for leaders is consolidation: when a tooling vendor you depend on is acquired, review contracts for continuity and exit terms. For investors, it is a reminder that technical due diligence should include supplier change-of-control risk.

4. PHP2Shell is being exploited: patch now

CISA added two PHP flaws, CVE-2026-109618 and CVE-2026-109619, to its Known Exploited Vulnerabilities catalog on 26 October and gave US federal agencies until 29 October to patch⁸. Researchers have reported successful exploitation attempts; fixed releases are PHP 8.5.12, 8.4.27, 8.3.36 and 8.2.35⁹.

CTO take: this is the only item today that needs action in every region. Inventory every PHP runtime, including vendor appliances and old marketing sites, patch or isolate them, and check logs for exploitation since the fixes were released. If you cannot answer "where do we run PHP?" within an hour, that is the real finding. See running useful postmortems.

5. India considers pre-release checks for the most powerful AI models

Reports on 28 October say India is weighing a new AI law with a separate safety framework for the most capable models, with options including pre-release evaluation and sandbox testing, and a consultation paper expected soon¹⁰. Nothing has been decided.

CTO take: most companies deploy models rather than build frontier ones, so the direct impact is small for now. Watch the consultation for deployer obligations, especially on deepfakes and fraud. Teams building in India should design for documentation and testing evidence that also satisfies the EU AI Act. See fractional CTO in India.

What I'd do this week

• Patch or isolate every PHP runtime and review logs back to 21 October.
• Audit agent and evaluation environments for network egress and stored credentials.
• Re-run your evaluation set on cheaper mid-tier models and update routing if quality holds.
• Add supplier change-of-control and exit terms to your vendor review checklist.
• Note the India AI consultation in your regulatory watch list alongside the EU AI Act.

Working across these regions? See where I work, or analyse your challenge and bring the result to a call.

What RDF 1.2 adds

RDF 1.2 introduces triple terms: an RDF triple that can itself appear as a term inside another triple¹. To say something about a statement, you use a reifying triple, whose predicate is rdf:reifies and whose object is the triple term; the reifier then carries whatever metadata you need¹. This standardises ideas first explored in the RDF-star community work⁴.

In Turtle 1.2's annotation syntax, a fact and its metadata can be written together, along these lines:

:acme :supplies :globex {| :source :contractSystem ;
                           :since "2024-03-01"^^xsd:date ;
                           :confidence 0.92 |} .

The base fact remains an ordinary triple that existing queries can use, while the annotation hangs off a reifier that refers to it.

The RDF 1.2 Concepts specification reached Candidate Recommendation on 7 April 2026², part of a wider set of RDF 1.2 and SPARQL 1.2 specifications from the working group. SPARQL 1.2 adds the matching query support³. Check the current W3C status before committing production systems to it.

Why it matters for enterprise graphs

The AI case is the most pressing. When a model extracts relationships from documents for GraphRAG or entity resolution, every new edge should carry where it came from, which model produced it and how confident it was, so curators can review and agents can weigh it.

RDF 1.2 and property graphs

Property graph databases have always allowed properties on edges, and they now have an international query language standard, ISO/IEC 39075 GQL⁵. Edge properties were one of the main reasons teams chose property graphs over RDF. RDF 1.2 narrows that gap with statement-level metadata, while RDF keeps its strengths: global identifiers, ontologies and validation with SHACL, and taxonomies in SKOS.

The choice is still about workload and ecosystem, as discussed in ontology vs knowledge graph vs semantic layer. What changes is that "we need properties on relationships" is no longer a reason on its own to leave RDF.

How I'd adopt it

1. Check tool support. Triple stores, parsers and SHACL engines adopt new specifications at different speeds; test the features you need in your own stack.
2. Define a provenance vocabulary before modelling: source system, extraction method, model version, confidence, validity period.
3. Annotate new facts first, especially AI-extracted ones, rather than migrating everything at once.
4. Keep base triples queryable, so existing applications keep working while new ones use annotations.
5. Write shapes for annotations, so required provenance is validated, not just encouraged.

How I can help

I design ontology and knowledge graph programmes where every fact can be traced and trusted, including graphs that AI systems both read from and write to. See ontology and knowledge graphs, or analyse your challenge and bring the result to a call.

Why write a policy at all

The Kubernetes community's first step was an AI policy. Not because policies are exciting, but because many pull requests had derailed into arguments about AI usage, and a clear stance steered those conversations¹. The same happens inside companies: without a shared rule, every review becomes a debate about whether AI-written code is acceptable, instead of whether the code is good.

What the Kubernetes policy says

The project's contributor guidance and the accompanying blog set out a few clear principles¹²:

The project also documented how new AI review tools are brought in, including finding maintainers willing to trial them on specific repositories before wider use³, and reported using AI reviewers as a quick first-pass quality gate so contributors get feedback without waiting for a maintainer¹.

Translating it to a company team

An open source project and a company differ in important ways: employees are already accountable to their employer, and code may touch customer data and contracts. The principles still hold. A practical company policy covers:

1. Approved tools and data boundaries. Which assistants are allowed, under which agreements, and what code or data must never be sent to them.
2. Ownership. The engineer who opens the pull request owns every line, however it was produced.
3. Explainability in review. Reviewers can ask the author to explain any change, and "the assistant wrote it" is not an answer.
4. Proportionate scrutiny. Security-sensitive areas, authentication, payments and data handling get stricter review regardless of how the code was written.
5. Tests as the contract. AI-assisted changes come with tests that a human has read and agrees with.
6. AI reviewers as helpers, not approvers. Automated review can catch issues early; approval stays with a person.
7. Supply chain hygiene. Watch for dependencies, snippets and model files introduced without scrutiny, as discussed in AI model supply chain security.

Measure the outcome, not the usage

Google's 2025 DORA research described AI mainly as an amplifier: it magnifies the strengths of high-performing organisations and the dysfunctions of struggling ones⁴. That is the right lens for leaders. Counting how much code AI produced tells you little. Watching delivery stability, rework and review load tells you whether AI is helping or merely moving work downstream, as covered in DORA metrics for AI-assisted engineering.

Write the policy down, keep it to a page, revisit it quarterly, and pair it with the decision records described in architecture decision records, so the reasoning behind important AI-assisted changes survives the people who made them.

How I can help

As a fractional CTO, I help teams adopt AI assistants with policies, review practices and metrics that improve quality rather than just volume. Or analyse your challenge and bring the result to a call.

The problem with bearer tokens

Service account JWTs are bearer tokens: whoever has the token is the identity it names. To authenticate to a peer you must hand it a copy, which means the peer could replay it. Kubernetes reduces the risk by binding tokens to time, audience and objects, but the Kubernetes blog is candid that none of these are complete defences¹.

The alternative is a proof-of-possession credential, where you prove you hold a secret without sending it. The most widely deployed form is the X.509 certificate used in TLS: the private key stays in the workload, and only a signature and the certificate cross the wire¹.

How Pod Certificates work

A Pod requests certificates through projected volume sources in its spec¹³. When the Pod is scheduled, the kubelet generates a private key, creates a PodCertificateRequest addressed to a named signer, and waits for the signer controller to issue a certificate chain. The kubelet writes the key and certificate into the container, alongside trust anchors collected from matching Cluster Trust Bundles, and refreshes everything before it expires¹.

Two details matter in practice. Rotation is automatic and frequent: signers eventually shipped in core are expected to issue certificates with a maximum lifetime of 24 hours, and other signers are capped at 91 days, so applications must reload credentials when files change. And security checks sit in the API server, where node restriction stops a compromised node requesting certificates for Pods not scheduled to it¹.

Side by side

The blog expects at least two built-in signers over time: one for server TLS certificates matching Service DNS names, and one for SPIFFE client certificates that fill the role service account tokens play today¹. SPIFFE defines a standard identity format and verifiable identity documents for workloads, which makes these identities portable across platforms⁴.

How I'd roll it out

1. Map current identity flows: which services call which, with what credential, and where tokens are forwarded rather than validated.
2. Pick a signer strategy. Until built-in signers arrive, choose a maintained signer or your existing PKI, and decide who owns the root of trust.
3. Start with internal mTLS between a few high-value services, using credential bundles so rotation is a single file reload.
4. Test rotation deliberately. With lifetimes as short as a day, an application that only reads its certificate at start-up will fail.
5. Enforce with policy: require certificate sources for services that handle sensitive data, using admission policies and the isolation model in Kubernetes multi-tenancy.
6. Keep tokens where they fit, especially for cloud federation, and stop forwarding them between services, the same anti-pattern described in securing MCP servers.

How I can help

I design Kubernetes platforms with identity, policy and isolation built in, so security reviews become a walkthrough rather than a negotiation. As a fractional CTO I can own that roadmap. Or analyse your challenge and bring the result to a call.

What changed, in one table

The release also moved Tasks into an official extension, hardened authorisation with RFC 9207 issuer validation, deprecated Dynamic Client Registration in favour of client metadata documents, and introduced a formal deprecation policy with a twelve-month minimum window¹.

The playbook

1. Inventory what depends on sessions

List every server and find where it relies on connection state: per-session caches, "current project" set at connect time, server-pushed requests, resumable streams. Those are your migration items. Servers that only expose stateless tools usually need little more than an SDK upgrade.

2. Replace hidden state with explicit handles

The specification's answer to cross-call state is an explicit, server-minted handle returned by a tool and passed back as an ordinary argument². The maintainers note this works better than transport-hidden state because the model can see the handle and thread it between tools¹. Store the state behind the handle in a shared store, scoped to the user, with an expiry.

3. Rework interactive tools with MRTR

Tools that asked the user for confirmation or a missing parameter mid-call now return an interim result with resultType: "input_required"; the client gathers the answers and retries the original request with them³. Design these tools so a retry is safe, and keep any correlation data in the request state rather than server memory.

4. Make retries safe everywhere

Without stream resumability, a dropped response stream means the client re-issues the request with a new ID². Any tool with side effects needs idempotency, the same discipline as in idempotent payment pipelines.

5. Move routing and policy to the gateway

With Mcp-Method and Mcp-Name on every Streamable HTTP request⁴, a gateway can rate-limit, authorise and meter per tool without parsing bodies. Plain round-robin load balancing now works. This is the natural place to enforce the controls in securing MCP servers.

6. Use the cache hints

Return a deterministic tool order and sensible ttlMs values. Stable tool lists help clients cache catalogues and keep upstream prompt caches warm, which matters for cost, as covered in prompt caching.

7. Plan the deprecations

Roots, Sampling and Logging are deprecated, with suggested migrations: pass paths through tool parameters or configuration instead of Roots, call model providers directly instead of Sampling, and use stderr or OpenTelemetry instead of protocol Logging²⁵. Trace context now has documented conventions in _meta, which fits the approach in LLM observability.

Sequencing for an enterprise estate

1. Gateway first, so routing and policy work for both old and new protocol versions during the transition.
2. High-traffic, stateless servers next: the easiest wins for scalability.
3. Interactive and stateful servers last, with handles and MRTR designed and tested properly.
4. Registry metadata updated to show which protocol versions each server supports, as in an enterprise MCP registry.

How I can help

I design and migrate enterprise MCP estates, from gateways and registries to server patterns that survive load balancers and retries. See my AI architecture work, or analyse your challenge and bring the result to a call.

Four kinds of characterisation

All except database-level characterisation describe a population relative to an index date defined by a cohort. Time before the index date is baseline; the index date and after is post-index¹. That is why good cohort definitions come first.

Start with the database

Before any population question, describe the database: how many people, their age distribution, how long they are observed, and how the recording of treatments, conditions and procedures changes over time. These summaries also reveal what may be missing¹, which is why they pair naturally with OMOP data quality checks.

Characterise the cohort broadly

OHDSI's approach to cohort characterisation is deliberately broad: descriptive statistics across all conditions, drug and device exposures, procedures and other observations in a person's history, plus demographics at the index date. That gives a complete picture of the population and helps surface variation and missing values¹. The FeatureExtraction package produces these features in a standard way, so the same characterisation runs at every site⁴.

The book notes that disseminating this kind of characterisation is recommended practice in the STROBE guidelines for reporting observational studies¹³. Practically, a baseline table is also the fastest way to check your cohort is the population you think it is.

Treatment pathways

Pathway analysis summarises the sequence of treatments people receive after a starting event, such as a diagnosis or first prescription. In a landmark study across the OHDSI network, Hripcsak and colleagues used standardised pathway analyses for type 2 diabetes, hypertension and depression, running the same analysis across many databases². The book reports that they found metformin to be the most common diabetes medication, consistent with first-line guidance, and that 10% of diabetes patients, 24% of hypertension patients and 11% of depression patients followed a treatment pathway shared with no one else in any of the data sources¹.

That last finding is a useful reminder for anyone designing clinical AI: real-world care is far more varied than guidelines suggest.

Incidence, defined precisely

Incidence measures new occurrences of an outcome in a population during a time at risk. The book defines two metrics¹:

• Incidence proportion: people in the cohort with a new outcome during the time at risk, divided by people in the cohort with time at risk.
• Incidence rate: people with a new outcome divided by the total person-time at risk, where each person's time stops when the outcome occurs.

Most disagreements about "the rate" of something trace back to different choices of cohort, time at risk or outcome definition. Writing those down explicitly is what makes incidence comparable across sites and studies.

Where characterisation fits

1. Database characterisation when a source joins the platform, and after every major load.
2. Cohort characterisation before any study or model, to confirm feasibility and population.
3. Pathways and incidence to understand current practice and baselines.
4. Then prediction or effect estimation, with the characterisation published alongside the results.

How I can help

I design OMOP platforms where characterisation runs automatically on every load and every new cohort, so analysts and AI teams start from facts rather than assumptions. See healthcare data architecture, or analyse your challenge and bring the result to a call.

What changed

In March 2026 the project marked key portions of the declarative configuration specification stable¹²:

• The JSON schema for the data model, released as opentelemetry-configuration 1.0.0³.
• The YAML file format and the in-memory representation of the model.
• The SDK operations to parse a file and create SDK components from it.
• A standard environment variable, OTEL_CONFIG_FILE, to point the SDK at the file.

At the time, implementations existed in C++, Go, Java, JavaScript and PHP, with .NET and Python under way, and the Collector used the Go implementation for its own internal telemetry¹. Since then, the Java Spring Boot starter has added support from version 2.26.0, embedding the same schema in application.yaml⁴.

Why it matters

Environment variables are flat. They can name an exporter endpoint, but struggle with a pipeline: several processors, different exporters per signal, views that drop noisy attributes, a sampler with rules. Anything beyond them meant code, or for the Java agent a separate extension jar⁴. The declarative schema describes the whole SDK pipeline as a tree, in the same shape the SDK runs⁵:

file_format: "1.0"
resource:
  attributes:
    - name: service.name
      value: checkout
tracer_provider:
  processors:
    - batch:
        exporter:
          otlp_http:
            endpoint: ${OTEL_EXPORTER_OTLP_TRACES_ENDPOINT:-http://collector:4318/v1/traces}

Environment variable substitution keeps the parts that genuinely vary per environment out of the file, while everything else becomes a reviewable artefact.

The playbook

1. Inventory what you have

List how each service configures its SDK today: environment variables, code, agent flags. You will usually find a few patterns that cover most services, and a long tail of one-offs.

2. Check language support

Support varies by language and by feature. The project tracks it in the specification compliance matrix and the configuration repository's language support status¹³. Start with the languages where support is mature in your stack.

3. Write a small set of standard files

Create a baseline per service type, such as web API, worker and batch job, covering resource attributes, exporters to your Collector gateway, batching, sampling and views. Keep secrets and endpoints as substitutions.

4. Ship the file with the platform, not the app

Mount it from a ConfigMap or bake it into base images, and set OTEL_CONFIG_FILE in your deployment templates. Teams inherit good defaults without copying settings.

5. Validate in CI

The model is defined by a JSON schema, so files can be validated before they ship³. Add the check to the same pipeline that enforces admission policies.

6. Migrate service by service

Run the declarative file alongside existing settings in a non-production environment, compare the telemetry produced, then switch. Pay attention to resource attributes and sampling, where small differences change dashboards and costs.

7. Plan for deprecations

The project intends to deprecate environment variables that don't interoperate well with declarative configuration, while making clear that deprecation is not deletion¹⁶. Moving now avoids a rushed migration later.

Where to be careful

• Precedence. Understand how the file interacts with remaining environment variables in each language before assuming one wins.
• Instrumentation configuration. The API for configuring instrumentation through the model was out of scope for the initial stabilisation¹; check what your language supports.
• Runtime changes. Dynamic configuration, changing telemetry without a restart, is still being designed¹.
• Stability is per language. A stable schema doesn't mean every SDK implementation is stable yet; follow the same approach as stable by default.

How I can help

I design observability platforms where telemetry is configured once, centrally and consistently, from SDK to Collector to backend. As a fractional CTO I can own that roadmap. Or analyse your challenge and bring the result to a call.

What changed

Together they close the gap: a secret can be read from a secrets manager ephemerally and handed to a resource through a write-only argument, without ever landing in state²³. There is one catch. Because Terraform never stores the value, it can't detect when it changes. Write-only arguments are paired with a version argument that you increment to trigger an update, and only the version is stored². Support also depends on each provider, resource by resource³⁵.

1. Find the exposure

• Inventory every state file and who can read it, including CI service accounts and old backends nobody uses.
• Search state for secret-bearing attributes: passwords, private keys, tokens, connection strings.
• List data sources that read from secrets managers; each one is copying a secret into state today.
• Treat any secret that has ever been in state as potentially exposed to everyone with state access.

2. Upgrade the toolchain

• Move to Terraform 1.11 or later so both features are available².
• Upgrade providers and check their documentation for ephemeral resources and write-only arguments on the resources you use.
• If you run OpenTofu, check its equivalent features and version support separately; the choice is covered in OpenTofu or Terraform.

3. Migrate resource by resource

• Replace secret-reading data sources with ephemeral resources where the provider offers them.
• Switch resources to write-only arguments for passwords and keys, with a version argument for rotation.
• Rotate the secret as part of the migration. Moving to write-only stops future storage; it doesn't undo past exposure.
• Better still, let the platform generate and manage the secret itself, such as a managed database credential stored in the cloud's secrets manager, so Terraform never handles it.
• Update shared modules first, using the patterns in Terraform module design, so every consumer benefits.

4. Harden state anyway

• Encrypt state at rest with customer-managed keys and restrict decrypt permissions.
• Limit state read access to the pipelines that need it; humans rarely do.
• Enable versioning and access logging on the state backend, and alert on unusual reads.
• Keep plan files out of CI artefacts and logs, or apply the same controls to them.

5. Keep it from regressing

• Add a policy check in CI that flags secret-bearing attributes set through normal arguments, and secret-reading data sources.
• Document the rotation process using version arguments, so it's routine rather than tribal knowledge.
• Include state access in access reviews and in your SOC 2 evidence; it is often the broadest secret access in the company.

How I can help

I review and redesign infrastructure-as-code estates for security and maintainability, including secrets handling, state architecture and module standards. As a fractional CTO I can own that roadmap. Or analyse your challenge and bring the result to a call.

Start smaller than you think

Small teams fail at SLOs by trying to cover everything. Two or three objectives on the user journeys that matter most is enough to start:

• Availability of the core API or page: the proportion of valid requests that succeed.
• Latency for the same journey: the proportion of requests faster than a threshold users would notice.
• Freshness or correctness for data products: the proportion of runs completed on time.

Measure at the edge closest to users, such as the load balancer or gateway, and set the first targets from what you achieved over the last month. The Workbook recommends writing the SLOs down with owners, approvers and review dates, and its example document gives each objective its own budget of 100% minus the target¹⁴.

Write the policy before you need it

The SLO is a number; the policy is what makes it matter. The Workbook says a policy should state the actions taken when the budget is exhausted and a clear escalation path for disagreements¹. Its example policy is a useful template²:

For a small company, a softer version often works better: rather than halting all releases, the next sprint's capacity goes to reliability work until the budget recovers. What matters is that product and engineering leadership sign it together, before the next bad week.

Alert on burn rate, not on errors

Paging on every error spike burns out small on-call rotations fast. The Workbook's guidance is to alert on how quickly the budget is being consumed, combining multiple time windows and burn rates so that fast, severe problems page someone while slow leaks become tickets³. With two or three SLOs, that typically means a handful of alerts in total, each of which genuinely matters.

The telemetry comes from the same pipeline as everything else; the OpenTelemetry Collector patterns make it straightforward to compute SLIs centrally.

What changes once it's running

• Prioritisation gets easier. Budget remaining means ship; budget gone means fix. The weekly debate shrinks to a glance at a dashboard.
• Postmortems get sharper. Budget consumed per incident is a better severity measure than whoever shouted loudest, and pairs well with the practices in AI incident postmortems.
• Delivery metrics gain context. DORA metrics say how fast you ship; error budgets say whether the speed is affordable.
• Customer conversations improve. An SLO you actually meet is a far better basis for an SLA than an aspiration.

Mistakes to avoid

1. Targets nobody would notice missing. If users wouldn't feel a breach, the SLO is measuring the wrong thing.
2. 100% targets. They leave no budget, so every change becomes a violation⁵.
3. A policy without a signature from product. Then it is an engineering wish, not an agreement.
4. Never revisiting. Review targets quarterly, as the Workbook's document template expects⁴.

How I can help

As a fractional CTO, I set up SLOs, error budget policies and on-call practices that fit the size of the team, so reliability decisions stop depending on the week you're having. Or analyse your challenge and bring the result to a call.

How it works

Generating text one token at a time leaves a large model's GPU mostly waiting on memory, not compute. Speculative decoding uses that slack. A cheap proposer guesses the next few tokens; the large target model checks all of them in one parallel pass; accepted tokens are kept, and generation continues from the first rejection¹.

The original Google Research paper showed this can sample from the target model's distribution exactly, so outputs are unchanged, and reported 2 to 3 times faster generation on T5-XXL with existing models and no retraining¹. In practice, floating-point differences in batched GPU execution mean byte-identical outputs are likely rather than guaranteed.

The proposer options

The current vLLM documentation summarises it this way: model-based methods such as EAGLE, MTP and draft models give the best latency reduction, while n-gram and suffix decoding give modest speedups without adding much work during peak traffic³. The EAGLE-3 paper reports speedups of up to 6.5 times in its experiments and argues its design lets the draft head benefit from more training data². The vLLM project also maintains a Speculators library for training draft models that run in vLLM⁴.

When it helps, and when it doesn't

vLLM describes speculative decoding as aimed at reducing inter-token latency for memory-bound workloads at medium to low request rates³. That sentence is the whole story:

• Interactive assistants with modest concurrency: good fit. Each user sees faster streaming.
• Code editing, document revision, extraction: often good for n-gram methods, because the output repeats large parts of the input.
• High-throughput batch jobs: weak fit. The GPU is already saturated, and verification work competes with real requests.
• Peak hours on shared clusters: gains can fall as load rises, which is why the cheaper methods appeal.

Older vLLM releases also carried caveats, such as incompatibility with pipeline parallelism and inconsistent latency gains across datasets⁵. Check the documentation for your exact version rather than relying on a blog post's configuration.

How I'd evaluate it

1. Replay real traffic, not a benchmark prompt set. Acceptance rate depends heavily on what your users ask and how your prompts are built.
2. Measure the right latencies: time to first token, inter-token latency and end-to-end time, at your real concurrency, using the telemetry in LLM observability.
3. Track acceptance rate and tune the number of speculative tokens; low acceptance means wasted verification.
4. Check outputs on your evaluation set to confirm nothing changed that matters.
5. Compare with alternatives. Prompt caching and routing easy requests to smaller models may deliver more for less effort.
6. Decide per workload. Enable it for interactive endpoints and leave batch endpoints alone, using the serving patterns in vLLM inference serving.

How I can help

I design private LLM platforms that meet latency and cost targets on hardware you control, from model choice to serving configuration. See my AI architecture work, or analyse your challenge and bring the result to a call.

The five designs at a glance

All rows are drawn from the descriptions in The Book of OHDSI¹.

Cohort method: emulating a trial

The new-user cohort design compares people who start the target treatment with people who start a comparator, and follows them for a defined time-at-risk. It is defined by five choices: target cohort, comparator cohort, outcome cohort, time-at-risk and model¹.

The central problem is that patients on different treatments differ. OHDSI's preferred adjustment is a propensity score fitted with large-scale regularised regression over many generic characteristics, typically 10,000 to 100,000 of them, rather than a hand-picked list¹. The design comes with diagnostics: preference score distributions to check the groups overlap, and covariate balance after adjustment, where some guidelines recommend standardised differences below 0.1¹⁴.

The book is clear that this is inherently a comparative design. Comparing a treatment with no treatment is hard because a comparable unexposed group is difficult to define; an active comparator for the same indication is preferred¹.

Self-controlled designs: people as their own controls

Three designs avoid between-person comparison altogether. The self-controlled cohort compares outcome rates during exposure with the time just before. The case-crossover design asks whether exposure was more common just before the outcome than at an earlier control time; because the outcome always comes later, it can be biased by exposure trends, which the case-time-control variant addresses. The self-controlled case series compares the outcome rate during exposed time with all unexposed time, before, between and after exposures¹.

These designs are robust to confounding by stable characteristics such as genetics or chronic conditions, but vulnerable to things that change over time, such as age, season or disease progression¹. The self-controlled case series also assumes the end of observation is independent of the outcome, which can fail for fatal outcomes such as stroke; an extension exists to correct for that¹⁵.

Case-control: efficient, but fragile

Case-control studies compare exposure in people with the outcome against people without it, often matching on age and sex and sometimes nesting within a subgroup such as patients with the relevant indication¹. They are efficient for rare outcomes, but results depend heavily on how controls are selected.

How I'd choose and validate

1. Write the question precisely as cohorts and time windows, as in OMOP cohort definitions.
2. Default to the new-user cohort design with an active comparator for comparative effectiveness, because its diagnostics make problems visible.
3. Use self-controlled designs for acute outcomes after transient exposures, such as vaccine or short-course drug safety, where timing is sharp.
4. Run negative controls. Outcomes believed not to be caused by the exposure reveal residual bias; the book's method validity chapter covers this and empirical calibration².
5. Replicate across databases through the federated network approach, after data quality checks at each site.
6. Keep estimation and prediction separate. Causal questions and prediction models need different designs and different validation.

How I can help

I design OMOP platforms and analytics pipelines that make real-world evidence studies reproducible, diagnosable and fast to run across sites. See healthcare data architecture, or analyse your challenge and bring the result to a call.

Where things stand

The AI Act's obligations for providers of general-purpose AI models have applied since 2 August 2025. The Commission said the AI Office was preparing its compliance assessments ahead of enforcement of those rules from August 2026². A Cloud Security Alliance research note published on 3 August 2026 summarised the AI Office's powers as including requests for documentation, model evaluations, corrective measures and fines⁷.

Alongside the law sit two practical instruments. The GPAI Code of Practice, published on 10 July 2025, has three chapters: transparency, copyright, and safety and security. The first two give all GPAI providers a way to demonstrate compliance with Article 53; the third applies only to providers of models with systemic risk under Article 55¹⁶. The transparency chapter includes a Model Documentation Form¹. The Commission also finalised a template for the public summary of training data⁵.

The line between deployer and provider

The Commission's guidelines on GPAI providers address downstream modification directly. As summarised by commentators³⁴:

• An indicative criterion is whether the training compute used for the modification exceeds one third of the compute used to train the original model.
• If the original compute isn't known, one third of the threshold for presuming a model is general-purpose is used instead.
• Minor changes typically don't make you a provider.
• If you do become a provider, your obligations generally concern the modification: documentation, training data summary and copyright policy covering the additional compute and data. Modifying a model with systemic risk brings the full systemic-risk obligations.

Why this matters even if you are well below it

First, customers ask. In enterprise procurement, "we use RAG, not fine-tuning" is a much easier answer with evidence behind it. Second, upstream providers' documentation flows down to you: the Model Documentation Form exists partly so downstream providers can understand the models they build on¹. Third, the same records support obligations that apply to your AI systems regardless of the model rules, such as the literacy duty in Article 4 and the timeline in the Digital Omnibus roadmap.

What I'd put in place

1. Log training compute for every model you train or tune, with the method used to estimate it. It's cheap to capture at the time and very hard to reconstruct later.
2. Keep a model lineage record: base model and version, upstream documentation received, datasets, licences and changes. The supply chain practices in AI model supply chain security overlap heavily.
3. Document data provenance and copyright handling for any data you train on, including how rights reservations are respected.
4. Prefer adaptation over retraining where it meets the need. RAG and lighter tuning are usually cheaper as well as simpler to govern; see long context vs RAG.
5. Map once, reuse often. Tie these records into a framework such as ISO 42001 so the same evidence serves EU, US and Gulf requirements.

How I can help

I help AI companies design training and deployment pipelines that produce compliance evidence as a by-product. As a fractional CTO I can own that work. Or analyse your challenge and bring the result to a call.

What changed

KIP-932, "Queues for Kafka", introduced share groups. Kafka 4.0, the first release to run only in KRaft mode without ZooKeeper, offered them as early access⁴⁵. The 4.2.0 release announcement describes Kafka Queues as production-ready, adding a RENEW acknowledgement type, adaptive batching in the share coordinator, limits on fetched record counts, and lag metrics¹³.

Consumer groups versus share groups

The Kafka documentation describes share groups as an alternative to consumer groups: consumers share a topic's records without each partition being tied to one consumer, records are acknowledged individually, and delivery attempts are counted. It suggests them for workloads that process records one at a time rather than as an ordered stream².

Where it fits well

• AI and enrichment workloads. Calling a model for each document or record is slow and variable. Share groups let you add workers to match throughput without repartitioning, and a slow call doesn't block the rest.
• Fan-out jobs from existing events. If the events already land in Kafka, for example from change data capture, processing them as tasks without copying them to another broker removes a hop.
• Bursty background work, where worker count needs to swing widely during the day, combined with autoscaling like that in Kubernetes scale-to-zero.

Where a separate broker still wins

• No Kafka today. Adopting Kafka just for a work queue is rarely worth it. A managed queue is simpler.
• Complex routing. Topic exchanges, header-based routing and per-message priorities remain stronger in dedicated brokers.
• Strict ordering with parallelism. If you need per-key order and parallel processing, keyed consumer groups remain the right model.
• Small clusters. Share groups store state in an internal topic that defaults to three replicas, so clusters with fewer than three brokers must adjust internal replication settings first².

How I'd adopt it

1. Get to a current release. Share groups need KRaft-based Kafka 4.x; plan the ZooKeeper migration first if you haven't⁵. Run the latest 4.2 patch release rather than 4.2.0.
2. Pick one queue workload that already consumes from Kafka and has idempotent processing, using the patterns in idempotent pipelines. Per-record acknowledgement still means at-least-once delivery.
3. Decide what happens after the last attempt. Route repeatedly failing records somewhere a person will see them.
4. Monitor share-group lag with the new metrics and alert on it, as you would queue depth¹.
5. Check your managed service. Hosted Kafka offerings adopt new features and metrics at different speeds.
6. Retire the old queue only after a full cycle of peak load and incident handling on the new one.

How I can help

I design event and data platforms that stay simple as they scale, including when to consolidate systems and when not to. As a fractional CTO I can own that architecture. Or analyse your challenge and bring the result to a call.

How the MCP Registry is designed

The official MCP Registry launched in preview in September 2025 as an open catalogue of publicly available servers¹⁴. Three design choices matter for enterprises:

• Metadata, not code. Entries describe servers and point to packages in existing registries such as npm, PyPI and Docker Hub¹².
• Verified namespaces. Server names use a reverse-DNS format, such as io.github.username/server or com.example/server, tied to a verified GitHub account or domain². That gives you a basis for trusting who published what.
• Built to be extended. The registry and its API specification are open source, so others can build compatible subregistries that pull from it, filter it and add their own servers¹³.

The registry remains in preview, and its documentation warns that breaking changes or data resets may occur before general availability². Build against it, but pin and mirror what you depend on.

The playbook

1. Decide what "approved" means

Write down the criteria before choosing tools: publisher verified, source available, package signed or pinned, permissions documented, data classification of what it touches, and an internal owner. The controls in securing MCP servers make a good starting checklist.

2. Stand up a private subregistry

Run a registry that implements the same API, so clients that understand the official registry can use yours³⁵. Servers on internal hostnames or private package feeds belong here, not in the public registry².

3. Mirror, don't proxy

Pull public entries into a staging area, review them, then promote approved versions. Pin to specific package versions so an upstream update can't change what your users run without review. This is the same discipline as AI supply chain security.

4. Enrich entries with your context

Add what the public registry can't know: owner, data classification, allowed user groups, required auth extension, and a link to the review record. Tool annotations and UI capabilities, covered in MCP Apps and tool annotations, should be part of the review.

5. Point clients at it

Configure managed IDEs and assistants to discover servers from your registry, and where clients support it, block unapproved sources. A catalogue nobody is pointed at is just documentation.

6. Tie it to identity

Approval should map to who can use a server. Pair the registry with the enterprise authorisation patterns in MCP extensions, so access is granted through your identity provider rather than per-server credentials.

7. Measure and prune

Track which servers are used, by whom and how often, using the telemetry approach in LLM observability. Retire servers without owners or usage, and re-review on major version changes.

Common mistakes

• Treating the public registry as an allow-list. Listing proves a verified namespace, not safety.
• Floating versions. "Latest" from upstream means unreviewed code on every laptop.
• No owner per server. Internal servers without owners become the next unpatched dependency.
• Registry without enforcement. If clients can still add any server, the registry is advisory only.

How I can help

I design enterprise agent platforms, including MCP registries, approval workflows and identity integration, so teams can adopt tools quickly and safely. See my AI architecture work, or analyse your challenge and bring the result to a call.

What was announced

In June 2026 the UAE approved the establishment of the Federal Authority for Artificial Intelligence and Data, reporting to the Cabinet, as a unified national body for data governance, artificial intelligence and digital government¹². Commentary reports that it brings together three existing functions¹³:

The mandate described in commentary covers the national AI strategy, government data governance, and standards and compliance across federal entities¹³. By October, reporting on the UAE's AI programme was already framing the authority as the body bringing national data, AI and digital government systems together⁶.

This week: Dubai's AI campus targets

On 26 October, during the Dubai AI Festival, Dubai approved new targets for its AI campus: more than 1,000 AI companies and over 5,000 specialised jobs by 2030⁴⁵. Together with federal consolidation, the direction is clear: one national voice on AI and data standards, and emirate-level competition to host the companies building on them.

What we don't know yet

• Private sector reach. The described mandate centres on federal entities. How far its standards will extend to private companies, directly or through procurement, is not yet clear.
• Interaction with free zones. DIFC and ADGM have their own data protection regimes, including DIFC Regulation 10 on AI systems. How federal AI standards will sit alongside them remains to be seen.
• Data protection supervision. The federal PDPL continues to apply. Which body leads on its AI-related aspects will become clearer through guidance.
• Health data. Sector rules, such as those in the UAE health data law, remain in place.

The regional pattern

Saudi Arabia went down this path earlier, with the Saudi Data and Artificial Intelligence Authority as a single body for data and AI⁷, alongside its own data protection law covered in Saudi PDPL and cross-border AI. A combined data and AI authority tends to treat data governance and AI governance as one discipline. Architecturally, that's right: most AI risk is data risk.

How I'd prepare

1. Map your current obligations by entity and location: federal PDPL, free zone regimes, sector rules, and where your models and data actually run, as discussed in UAE AI infrastructure.
2. Build governance on a recognised framework, such as ISO 42001 or the NIST AI RMF, so national standards become a mapping exercise.
3. If you sell to government, expect federal data and AI standards to appear in procurement first. Keep an AI system inventory and data lineage ready to evidence.
4. Unify data and AI governance internally. If the regulator treats them as one discipline, a separate AI committee disconnected from data owners will struggle.
5. Watch for guidance from the new authority and plan a quarterly review of your mapping.

How I can help

I help companies in the UAE and the US build AI platforms with governance that maps cleanly across regimes. As a fractional CTO I can own that work. Or analyse your challenge and bring the result to a call.

Two standards, two purposes

SKOS, the Simple Knowledge Organization System, became a W3C Recommendation in 2009. It is a common data model for sharing and linking knowledge organisation systems, such as thesauri, taxonomies and classification schemes, on the web¹. Its world is concepts, labels and loose relations between them.

OWL 2, the Web Ontology Language, defines classes, properties and logical axioms that reasoners can use to infer new facts and detect contradictions. Its profiles, finalised in their second edition in 2012, trade expressiveness for tractable reasoning³.

The SKOS designers were explicit that SKOS was not meant to compete with OWL but to complement it, offering a path to bring existing knowledge that doesn't map easily into an OWL ontology onto the semantic web⁵.

Side by side

The mistake that matters most

The most expensive error is treating a taxonomy hierarchy as a class hierarchy. In a product catalogue, "Garden furniture" sits under "Outdoor" and also under "Furniture". That is a filing decision, not a claim that every garden chair is, logically, an instance of "Outdoor".

SKOS is careful here. skos:broader is used only for direct hierarchical links and is not declared transitive; skos:broaderTransitive is a separate superproperty for inferring indirect links¹². Converting the same hierarchy into rdfs:subClassOf makes every relation logically binding, and a reasoner will happily draw conclusions nobody intended.

Choosing OWL profiles when you do need logic

Where OWL is the right tool, its profiles help match reasoning to scale³⁴:

• OWL 2 EL suits ontologies with very large numbers of classes and properties, with polynomial-time reasoning for core tasks. Large biomedical terminologies are the classic fit.
• OWL 2 QL targets large volumes of instance data queried through relational databases, which suits virtual knowledge graphs over existing warehouses.
• OWL 2 RL supports rule-based reasoning that works directly on RDF triples, practical inside many triple stores.

Choosing a profile early prevents a common failure: a model that is logically elegant and unusable at production volume.

The combined pattern I use

1. A small OWL core for the domain: customer, product, contract, patient, asset, and the relations between them. Keep it in a tractable profile.
2. SKOS vocabularies for classification: product categories, risk types, diagnosis groupings, document subjects. Instances in the core point to concepts through ordinary properties, such as "has category".
3. Mappings between vocabularies using SKOS mapping properties, so a local code list can align with an industry standard without pretending they are identical.
4. SHACL for data quality, as described in SHACL validation, because closed-world checks belong outside the open-world logic of OWL.
5. Labels feed search and AI. SKOS alternative and hidden labels give retrieval and GraphRAG the synonyms and misspellings users actually type.

This also clarifies ownership. Business teams can maintain vocabularies in a taxonomy tool without touching the ontology, while architects govern the core. How these pieces relate to a semantic layer is covered in ontology vs knowledge graph vs semantic layer, and how clinical vocabularies are handled in practice in clinical terminologies in OMOP.

How I can help

I design ontology and knowledge graph programmes that start with the models an organisation already has, and choose the lightest formalism that does the job. See ontology and knowledge graphs, or analyse your challenge and bring the result to a call.

Why it matters more for AI

Traditional SMART apps show data to a clinician who is already authorised to see it. AI apps often do more: summarise a whole record, draft notes back into the chart, or run in the background across many patients. That widens both the data an app touches and the consequences of a misconfigured scope. In the US, HTI-1 also moved certified health IT from SMART v1 to the SMART App Launch 2.0 guide for the standardised patient and population API criterion, so v2 behaviour is now the baseline buyers expect⁶. The policy side is covered in HTI-1 to HTI-5.

1. Scopes: ask for the least, precisely

• Use v2 scope syntax. Permissions are expressed as suffixes from c, r, u, d and s. patient/Observation.rs grants read and search on the patient's observations; patient/Observation.cu grants create and update¹.
• Narrow with search parameters. v2 allows granular scopes such as Observation access limited to the vital-signs category¹⁵. A sepsis model that only needs vitals and labs shouldn't hold a token for psychotherapy notes.
• Prefer patient/ over user/ and system/ wherever the use case is about one patient in context.
• Separate read and write. If the AI drafts notes, consider a distinct, reviewable write path rather than giving the summariser create rights.
• Check what the server actually supports. Servers advertise capabilities in their SMART configuration⁵; don't assume granular scopes are enforced everywhere.

2. Launch and token handling

• PKCE with S256, always. The specification says all SMART apps shall support PKCE, servers shall support S256 and shall not support the plain method².
• Use EHR launch for in-workflow AI, so the patient and encounter come from the EHR's launch context, not from parameters the app constructs¹².
• Request openid fhirUser so every AI action can be attributed to an identified user².
• Keep tokens out of the model. Access tokens belong in the integration layer, never in prompts, logs or tool arguments the model can see or echo.
• Be deliberate about offline access. Refresh tokens extend reach well beyond the clinician's session; request them only when the use case needs it.

3. Background and population workloads

• Use SMART Backend Services for autonomous processing, such as overnight risk scoring or quality measures. It is designed for clients with pre-authorised scopes and no user in the loop³.
• Authenticate with asymmetric keys. private_key_jwt is the specification's preferred method because no shared secret crosses the wire; clients must support RS384 and ES384⁴.
• Rotate keys through a JWKS endpoint, and treat the private key with the same controls as any production secret.
• Use bulk export for volume, as in FHIR bulk data pipelines, rather than looping single-patient queries.

4. Agents and tools

• One tool, one scope set. If an agent reaches FHIR through MCP servers, each server should hold the narrowest token for its job, with the controls in securing MCP servers.
• Keep patient context fixed. An agent must not be able to switch patients by changing a tool argument; patient identity comes from the launch, not the prompt. This is a key defence against prompt injection.
• Human approval for writes. Drafted notes, orders or messages go to a clinician for review before they are committed, as in ambient scribe architectures.

5. Evidence for the security review

• A table of every scope requested, why, and which feature uses it.
• Audit logs linking each FHIR call to user, patient, feature and model version.
• Token lifetimes, refresh policy and revocation procedure.
• Results of testing with a SMART conformance tool against the target servers.
• A data flow showing exactly where PHI goes, including any model provider, mapped to HIPAA Security Rule controls.

How I can help

I design EHR integrations for AI products that pass hospital security reviews the first time, from scope design to agent tooling and audit. See healthcare data architecture, or analyse your challenge and bring the result to a call.

What changed

In-place Pod resize arrived as alpha in Kubernetes 1.27, reached beta in 1.33 and graduated to stable in 1.35¹³. It makes CPU and memory requests and limits mutable on a running Pod. You send the change to the Pod's resize subresource, for example with kubectl patch --subresource resize, which needs kubectl 1.32 or later².

Each container declares a resizePolicy per resource. NotRequired asks for the change to be applied without a restart; RestartContainer restarts that container to apply it. A common pattern in the documentation is CPU set to NotRequired and memory set to RestartContainer².

The stable release added two things operators will notice. Memory limits can now be decreased, with the kubelet only allowing it when current usage is below the new limit, a best-effort check rather than a guarantee. And when a node can't fit every pending resize, retries are ordered by PriorityClass, then QoS class, then how long the request has waited¹.

Watch the conditions, not just the spec

A resize is a request, not a guarantee. The kubelet reports progress through Pod conditions²:

The practical rule: alert on long-lived PodResizePending. A spec that says 4 CPUs while the container still runs on 2 is a quiet source of confusion in incidents and cost reports alike.

Where the VPA fits

The Vertical Pod Autoscaler recommends requests from observed usage and applies them. Its InPlaceOrRecreate update mode tries an in-place resize first and falls back to recreating the Pod when that isn't possible; the 1.35 release blog described that mode as graduating to beta¹⁵.

This is the combination that changes cost economics. Recommendations used to sit in a report because applying them was disruptive. With in-place updates they can be applied continuously, within limits you set.

Pod-level resources are next

Kubernetes 1.36 promoted in-place resize of Pod-level resources to beta, meaning a shared budget for all containers in a Pod can also change without recreating it⁴. That is useful for Pods with sidecars, where per-container numbers are hard to tune separately. I would treat it as beta: pilot it, but don't build a platform default on it yet.

Where it bites

• Applications that size themselves at start-up. A runtime that reads its memory limit once, such as a JVM heap or a worker pool sized to CPU count, won't notice a change. For those, a restart policy on memory is the honest choice.
• Memory decreases. The usage check is best-effort, so shrinking memory on a busy Pod still carries OOM risk¹. Shrink gradually and watch.
• Node fragmentation. Growing in place only works if the node has room. Busy nodes produce Deferred resizes; plan headroom or accept fallback to recreation.
• Quotas and policy. Resizes still count against namespace quotas, and teams will need guardrails on how far they can go, enforced with admission policies in multi-tenant clusters.

How I'd roll it out

1. Measure first. Compare requests with real usage by namespace and team, using the allocation approach in FOCUS-based cost allocation.
2. Start with CPU on stateless services, using NotRequired, where the risk is lowest and the savings show quickly.
3. Run the VPA in recommendation mode for a few weeks, then enable InPlaceOrRecreate on workloads that tolerate the fallback.
4. Use it for warm-up spikes. Services that need extra CPU at start-up, including model servers like those in vLLM serving, can start large and shrink once warm.
5. Alert on pending resizes and report status, not spec, in cost dashboards.
6. Combine with scale-to-zero from Kubernetes FinOps: horizontal scaling for traffic shape, vertical for per-Pod fit.

How I can help

I run Kubernetes cost and platform reviews that turn right-sizing from a one-off report into a continuous, safe process. As a fractional CTO I can own that roadmap. Or analyse your challenge and bring the result to a call.

What the research shows

Spider 2.0, presented at ICLR 2025, was built from real enterprise workflows: 632 problems over databases often with more than a thousand columns, in systems such as BigQuery and Snowflake. The authors reported that a strong reasoning model using a code agent framework solved 21.3% of tasks, compared with 91.2% on the earlier Spider 1.0 benchmark and 73.0% on BIRD¹².

The gap isn't mainly about SQL syntax. It is about knowing which tables matter, what columns mean, how entities join, and which business rules apply. That knowledge rarely lives in the schema.

What a semantic layer gives an agent

A semantic layer defines business concepts, such as metrics, dimensions, entities and their relationships, once, in governed code, and exposes them to every tool. For an AI agent, that changes the task from "write SQL against 1,500 columns" to "pick the governed metric 'net revenue', group by the dimension 'sales region', filter to last fiscal quarter". The layer generates the SQL.

• Consistency. The agent's answer matches the dashboard, because both use the same definition.
• Smaller search space. Choosing from a curated set of metrics is far easier than reasoning over raw tables.
• Governance. Access control, row-level security and certified definitions apply to AI queries just as they do to BI.
• Explainability. The agent can say which metric and filters it used, in business terms a user can check.

The market is converging

In September 2025 Snowflake, Salesforce, dbt Labs and others announced the Open Semantic Interchange, an initiative for a vendor-neutral semantic model specification, explicitly framed around making data ready for AI³. In October 2025 dbt Labs moved MetricFlow, the engine behind its semantic layer, to the Apache 2.0 licence and said it would align it with that initiative⁴⁵.

Whatever tools you choose, the direction is clear: business definitions are becoming portable, open artefacts that AI and BI share, rather than logic buried in each dashboard.

Semantic layer, ontology or both?

A semantic layer is optimised for metrics: numbers aggregated over dimensions. An ontology describes entities and relationships more broadly, as I set out in ontology vs knowledge graph vs semantic layer. For "how much" questions, the semantic layer is the right tool. For "which suppliers are connected to this incident", a knowledge graph answers better. Mature data platforms give agents both, with clear guidance on which to use.

How I design data access for AI agents

1. Start with the twenty questions people actually ask, and define the metrics and dimensions they need in the semantic layer first.
2. Expose governed tools, not a SQL console. Agents call "query metric" and "list dimensions" tools, for example through MCP servers built with the controls in securing MCP servers.
3. Return the definition with the answer, so users see which metric, filters and time range were used.
4. Fall back honestly. If a question can't be answered from governed definitions, say so and route it to an analyst, rather than improvising SQL.
5. Evaluate with real questions and known answers, as in evaluation sets that predict production, and track accuracy by question type.
6. Keep definitions under contract, with the change discipline in data contracts, so a schema change can't silently break what agents report.

How I can help

I design data platforms where AI agents and BI share governed definitions, from semantic layers and ontologies to the tools agents call. See ontology and knowledge graphs, or analyse your challenge and bring the result to a call.

Core, projects and extensions

The MCP maintainers describe the ecosystem in three layers¹:

• The core specification: how clients and servers talk, and the minimum bar for interoperability.
• Projects: supporting infrastructure such as the Registry for discovering servers and the Inspector for testing.
• Extensions: optional patterns for specialised needs, built on top of the core.

Extensions are strictly additive. They are negotiated during the initialisation handshake, and a client or server that doesn't recognise one simply skips it, so the baseline protocol keeps working¹². They are also how future protocol changes get validated: an extension that gains traction signals what might belong in the core¹.

What exists today

The maintainers also draw a useful line: proprietary client features that happen to use MCP are not necessarily extensions, and they encourage implementers to adopt extensions as the standard way to add custom behaviour¹.

How I decide which to adopt

1. Is it official, recommended or community-defined? Official extensions live in the MCP organisation and go through the SEP process⁵. Community extensions can be valuable but carry more change risk.
2. Which of our clients support it? Check the extension support matrix⁴ against the clients your organisation actually uses. An extension supported by one client only is a pilot, not a standard.
3. What's the security impact? UI extensions run code from servers inside hosts; auth extensions change who can access what. Both need review before approval.
4. Does it degrade gracefully? Because extensions are optional, test what happens when a client doesn't support one. The workflow should still be safe, even if less convenient.
5. Who owns it internally? Each supported extension needs an owner on the platform team, a version policy and a place in your server review checklist.

Auth extensions first

If you adopt one extension category first, make it authorisation. Per-user, per-server OAuth doesn't scale in an enterprise, and machine-to-machine agents need a supported way to authenticate without a person in the loop. Centralising access through your identity provider gives you consistent policy and one audit trail, which supports the controls in securing MCP servers and the evidence expected in SOC 2.

Contributing back

Organisations in regulated sectors have a real opportunity here. If your industry needs shared conventions, such as provenance, consent or compliance metadata on tool calls, the SEP process offers an Extensions Track for proposing them⁵. Defining those conventions openly is better than every vendor inventing its own.

How I can help

I design enterprise MCP platforms, including extension policy, identity integration and server governance, so agent capabilities can grow without fragmenting. See my AI architecture work, or analyse your challenge and bring the result to a call.

What happened

What the new law looks like

Commentary describes SB 26-189 as built around automated decision-making technology used in consequential decisions, including employment decisions about Colorado applicants and employees⁴. The headline change is what it no longer requires: covered employers no longer need risk management programmes or annual impact assessments. Instead, the law focuses on notice and disclosure, specific consumer rights and record retention, with duties split between developers and deployers⁴⁵. The Attorney General is to adopt implementing rules before it takes effect, so operational details may still change⁴.

The broader lesson

Colorado's path mirrors what has happened elsewhere. The EU delayed parts of its AI Act through the Digital Omnibus. US federal health IT rules proposed removing AI transparency requirements, as covered in HTI-1 to HTI-5. Requirements are moving in different directions in different places, and quickly.

Building a compliance programme around one statute's exact wording is fragile in that environment. Building core governance capabilities that map to several regimes is robust.

What I'd build regardless

1. An inventory of automated decisions about people: which systems influence hiring, credit, pricing, eligibility or access, and how much weight the AI carries.
2. Notices generated from the inventory, so disclosures stay accurate as systems change. Notice is the obligation that survived in Colorado, and it is common elsewhere.
3. Records that can answer "why": inputs, model versions and outputs for consequential decisions, retained for a defined period, using the logging in LLM observability.
4. A route for people to contest decisions and reach a human.
5. Testing for bias and quality before release and on a schedule, even where no longer mandated, because it is what customers and courts will ask about. See evaluating AI agents.
6. A framework to map against, such as those compared in ISO 42001 and the NIST AI RMF, so new state laws become a mapping exercise rather than a new programme.

For companies selling into the US from the Gulf or Europe

For companies selling from the UAE or Europe into US enterprises, the practical answer is one governance core, evidenced well, then mapped to each market: the EU AI Act, US state laws as they settle, and Gulf rules such as DIFC Regulation 10. That's cheaper than chasing each statute separately, and it survives the next reset.

How I can help

I help AI companies build governance into their products and platforms, so changing laws mean updated mappings, not rebuilt programmes. As a fractional CTO I can own that work. Or analyse your challenge and bring the result to a call.

What OBI does

OpenTelemetry eBPF Instrumentation uses eBPF to inspect application executables and the operating system's networking layer, capturing trace spans, rate-errors-duration metrics, runtime metrics and service relationships for supported Linux workloads, without modifying application code or configuration¹. The documentation lists support for languages including Java, .NET, Go, Python, Ruby, Node.js, C, C++ and Rust, visibility into TLS traffic without decryption, context propagation across services, and configuration-free auto-instrumentation on Kubernetes¹.

That last point matters most in practice. On a cluster, you can get a baseline service map and latency metrics for everything in days, not quarters.

Log correlation without touching the app

A recent addition addresses a common incident pain: you have a failing trace, but no way to find the matching log lines because the service never added trace context to its logs. OBI can now add trace and span IDs to logs written to stdout or stderr, because through eBPF it knows which request each thread is serving at the moment of the write³.

The boundaries are stated clearly: it applies to stdout and stderr, only annotates lines written while OBI has active trace context, needs elevated privileges, and on older kernels only some write paths are enriched; the common write path needs Linux 6.0 or later. It also needs a small change in the log pipeline³.

Where it's heading in 2026

The OBI SIG's 2026 goals centre on a stable 1.0 release for production readiness, with complete configuration documentation, adoption of OpenTelemetry's declarative configuration and telemetry schemas, and test coverage targets. They also plan to expand protocols to messaging systems such as MQTT, AMQP, NATS and Redis pub/sub, extend database support, instrument cloud service SDKs, broaden .NET support, and strengthen integration with OpenTelemetry SDKs for hybrid instrumentation².

Check the project's current release status before production rollout⁴, and pilot in non-critical environments first, in line with the stability approach in OpenTelemetry's stable-by-default push.

OBI or SDK?

How I'd roll it out

1. Start with the blind spots. Deploy OBI where SDK instrumentation is missing, and compare the service map with what teams believe the architecture is.
2. Route through your Collector gateway, as in Collector deployment patterns, so sampling, redaction and export are handled centrally.
3. Review privileges with security. eBPF instrumentation runs with elevated access on nodes; treat it as platform infrastructure with the same controls as other node-level agents, enforced through admission policies.
4. Check data sensitivity. Capturing protocol data, including from encrypted traffic, can expose sensitive values. Configure what is collected and redact at the gateway.
5. Add SDKs where business context matters, such as checkout flows, AI calls with GenAI attributes, and regulated workflows.
6. Watch cardinality and cost, since cluster-wide instrumentation can generate a lot of telemetry quickly.

How I can help

I design observability platforms that close blind spots quickly and add depth where it matters, using OpenTelemetry end to end. As a fractional CTO I can own the observability roadmap. Or analyse your challenge and bring the result to a call.

The problem with most prediction models

The Book of OHDSI is direct about it: many published patient-level prediction efforts don't follow model development guidelines, fail to perform extensive external validation, or provide too few details for others to reproduce and validate them¹. It points to the TRIPOD reporting guideline as a standard for transparent reporting¹⁴.

The underlying issue is that a model learns the habits of the data it was trained on: coding practices, patient mix, documentation patterns. Those don't travel.

Define the problem precisely

OHDSI frames every prediction problem in the same form: among a target cohort, who will go on to have an outcome within a time-at-risk? Prediction happens at a defined moment, and only information from the observation window before that moment may be used¹.

The same template covers disease onset, treatment response, safety and adherence. The book's examples include predicting ischaemic stroke among newly diagnosed atrial fibrillation patients, and gastrointestinal bleeding among new users of warfarin¹. Because target and outcome are standard cohort definitions, another site can rebuild exactly the same problem on its own data.

Measure the right things

Discrimination, usually summarised by the area under the ROC curve, measures whether the model ranks patients who will have the outcome above those who won't. The book notes that for rare outcomes even a model with a high AUC may not be practical, because many false positives can accompany each true one¹.

Calibration measures whether predicted risks match reality: of a hundred patients given a 10% risk, about ten should have the outcome¹. In the book's worked example, a model was well calibrated overall but poorly calibrated for younger patients¹. Overall metrics hide that kind of problem, which is why subgroup checks matter for both safety and fairness.

Validate externally, always

The book distinguishes internal validation, testing on held-out data from the same database, from external validation, developing in one database and evaluating in another. It recommends always performing external validation on as many new datasets as feasible, because different databases represent different populations, health systems and data capture¹.

This is where a common data model pays off. A model built on OMOP can be shared as a package and evaluated at other OMOP sites without moving patient data, which is the federated approach described in federated health analytics. The PatientLevelPrediction R package implements this end to end, from data extraction with standard covariates to training and validation²³.

How I'd run a clinical prediction project

1. Write the prediction question in the standard form and agree it with clinicians before touching data.
2. Build cohorts on OMOP, checked with data quality tools, so the design is portable.
3. Start with simple models. Regularised logistic regression is often competitive and easier to explain.
4. Report discrimination and calibration, overall and by key subgroups, following TRIPOD.
5. Validate on at least one external dataset before any clinical use.
6. Plan deployment and monitoring: how the score reaches clinicians, how drift is detected, and how changes are controlled. The transparency practices in HTI-1 to HTI-5 and FDA PCCPs apply.

How I can help

I design OMOP-based platforms and clinical AI pipelines that make prediction models reproducible, externally validated and safe to deploy. See healthcare data architecture, or analyse your challenge and bring the result to a call.

What the benchmarks tell you

The Massive Text Embedding Benchmark, MTEB, evaluates embedding models across many tasks, such as retrieval, classification, clustering and semantic similarity, over dozens of datasets and many languages. Its central finding when published was that no single method dominated across all tasks¹. Its multilingual successor, MMTEB, presented at ICLR 2025, expanded to over 500 evaluation tasks across hundreds of languages, including long-document and code retrieval². Notably, its authors reported that the best-performing publicly available model at the time was a relatively small multilingual model of around 560 million parameters².

Two lessons follow. Model size isn't the same as fitness for your task. And an average score across hundreds of tasks may hide poor performance on the one task you care about.

The checklist

1. Filter by the task that matters. For RAG, look at retrieval scores, not the overall average. If you search code or long documents, look at those categories specifically.
2. Check your languages. For Gulf and multilingual products, test Arabic and mixed-language queries explicitly. Many models are strongest in English, and a good average can hide weak Arabic retrieval.
3. Check domain fit. Clinical, legal and financial language behave differently from web text. Test with real domain documents; if results are weak, hybrid keyword and vector search often helps more than switching models, as in hybrid search for RAG.
4. Measure on your own data. Build a labelled set of queries and the documents that should be retrieved for them, and measure recall and ranking quality for each candidate. The open-source mteb package can help run standard tasks alongside your own⁴, and the wider method is in evaluation sets that predict production.
5. Consider dimensions and cost. Bigger vectors cost more to store and search. Models trained with Matryoshka Representation Learning let you truncate embeddings to smaller sizes with limited quality loss; the original paper reported up to 14 times smaller embeddings at the same accuracy on an image classification benchmark³. Test the trade-off on your own retrieval set.
6. Consider where it runs. Hosted embedding APIs send your documents to a third party. For regulated or residency-bound data, a self-hosted open model may be necessary, as discussed in Saudi PDPL and cross-border AI and private LLMs on-prem.
7. Check licence and longevity. If a provider deprecates a model, you must re-embed everything. Prefer models you can keep running, or plan for migration.
8. Check context length. Some models truncate long inputs silently. Make sure your chunk sizes fit the model's limit.

Plan for re-embedding from day one

Whatever you choose, you will change it eventually: a better model appears, a provider retires one, or your content shifts language or domain. Design for that:

• Store the model name and version with every vector, and never mix vectors from different models in one index.
• Keep source text and chunking reproducible, so you can re-embed without re-ingesting.
• Build new indexes alongside old ones, compare retrieval quality on your evaluation set, and switch traffic only when the new one wins.
• Budget the cost. Re-embedding a large corpus takes real compute and time; estimate it before you need it.

Where embeddings sit in the bigger picture

Embeddings are one part of retrieval. Reranking, hybrid search, metadata filters and good chunking often matter as much. Where your vectors live is its own decision, covered in pgvector or a vector database. And for some questions, a knowledge graph or GraphRAG beats any embedding, as in GraphRAG vs vector RAG. The most reliable systems treat the embedding model as a swappable, versioned component, chosen with evidence and re-evaluated regularly.

How I can help

I design and evaluate retrieval systems for AI products, including embedding selection, hybrid search and multilingual retrieval for Gulf markets. See my AI architecture work and the RAG to production checklist, or analyse your challenge and bring the result to a call.

Two kinds of tenancy

The Kubernetes documentation distinguishes two broad use cases¹:

• Multiple teams in one organisation share a cluster. There is some trust between them, members often have direct access through tools or GitOps, and workloads frequently talk to each other.
• Multiple customers, where a vendor runs workload instances per customer. Customers never touch the cluster, cost matters a great deal, and isolation must be strong.

The documentation also cautions that "hard" and "soft" multi-tenancy are better seen as a spectrum than two categories. Hard multi-tenancy usually describes tenants that don't trust each other, which demands particular attention to data-plane isolation¹.

The building blocks

Control plane isolation stops tenants seeing or changing each other's API resources. Namespaces scope names and many policies; RBAC enforces least privilege, which the docs call the most important type of control-plane isolation, because a tenant who can modify another's resources can disable every other protection¹². Resource quotas cap what each namespace can consume⁵.

Data plane isolation protects workloads at runtime: network policies to restrict traffic³, storage isolation, sandboxed containers using virtual machines or userspace kernels for untrusted code, and node isolation that dedicates nodes to a tenant¹.

Fairness matters too. API Priority and Fairness prevents one tenant's controllers from overwhelming the API server⁴.

The three models compared

This follows the documentation's own description: namespace isolation is well supported with negligible resource cost but can be difficult to configure and doesn't cover non-namespaced resources, while control plane virtualisation isolates those at the cost of somewhat higher resource use¹.

How I decide

1. Who writes code that runs? If tenants can run arbitrary code, assume it may be hostile, and favour sandboxing, dedicated nodes or clusters.
2. Do tenants need cluster-level resources? Teams that ship their own operators and CRDs push towards virtual control planes.
3. What do regulators and contracts require? Data residency and customer contracts may require dedicated infrastructure in a region, as in UAE health data localisation.
4. What can the platform team operate? Fifty clusters run badly are riskier than five run well.
5. Mix models. Shared clusters for most tenants, dedicated ones for the few that need them, mirroring the hybrid pattern in multi-tenant SaaS on MongoDB.

Practical rules for shared clusters

• A namespace per workload, with names unique across the whole fleet, which the docs recommend so you can move between shared and dedicated clusters later¹.
• Default-deny networking, with explicit allows.
• Quotas and limit ranges on every tenant namespace.
• Guardrails as policy, such as the CEL policies in Kubernetes admission policies, so tenants can't weaken their own isolation.
• Cost allocation by tenant, using labels and the approaches in Kubernetes FinOps.
• Separate GPU pools where AI workloads from different tenants compete, with the scheduling controls in gang scheduling.

How I can help

I design Kubernetes platforms for internal teams and SaaS products, choosing the tenancy model that fits the trust, cost and compliance picture. As a fractional CTO I can own the platform roadmap. Or analyse your challenge and bring the result to a call.

Why "keep everything in context" fails

Stuffing history into the prompt runs into the limits covered in long context or RAG: cost and latency grow with every token, and models use long inputs unevenly. Chroma's research found performance grows increasingly unreliable as input length grows, even on simple tasks⁴.

LongMemEval, a benchmark presented at ICLR 2025, tested exactly this for assistants. Its 500 questions probe five abilities: information extraction, multi-session reasoning, temporal reasoning, knowledge updates and abstention, embedded in long user-assistant histories. The authors reported that commercial chat assistants and long-context models showed about a 30% accuracy drop when recalling information across sustained interactions³.

Two of those abilities deserve attention. Knowledge updates: when a user changes their address or a project changes scope, memory must reflect the latest truth, not the first. Abstention: when the assistant doesn't know, it should say so rather than inventing a memory.

Ideas from research

Memory as an operating system. MemGPT proposed virtual context management, borrowing from operating system memory hierarchies: the model pages information between a small, fast working context and larger external storage, using function calls to read, write and manage its own context¹. The authors showed it maintaining consistency over long conversations and analysing documents larger than the context window¹.

Retrieval by relevance, recency and importance. The Generative Agents paper gave simulated characters a memory stream of experiences, retrieved by combining how relevant, how recent and how important each memory is, plus periodic reflection that synthesises memories into higher-level conclusions².

Indexing, retrieval and reading. LongMemEval's authors framed memory design as stages of indexing, retrieval and reading, and proposed improvements such as splitting sessions into smaller units, expanding memory keys with extracted facts, and making queries time-aware³.

The memory architecture I use

Practical rules that make it work:

1. Extract facts deliberately, as structured records with a source and timestamp, using structured outputs, rather than keeping raw transcripts as the only memory.
2. Resolve conflicts by time. Newer facts supersede older ones; keep history but retrieve the current value.
3. Retrieve with time awareness, so "last week's decision" finds last week's session.
4. Let the model abstain. If retrieval finds nothing reliable, the answer is "I don't have that", not a guess.
5. Evaluate memory specifically, with test cases for recall, updates and abstention, added to your agent evaluation suite.

Memory is personal data

A memory store is a profile of a person. Treat it accordingly:

• Visibility and control. Users can see what is remembered, correct it and delete it.
• Scope. Memory is partitioned per user and per tenant, with the same isolation tests as any multi-tenant store, as in multi-tenant SaaS patterns.
• Sensitive categories. Don't store health, financial or other sensitive details without a clear basis and purpose, especially under regimes such as the UAE PDPL.
• Retention. Define how long each layer is kept and enforce it automatically.
• Injection risk. Content written into memory from untrusted sources can carry instructions into future sessions. Treat remembered content as data, with the controls in prompt injection defence.

Start small

Most products don't need the full stack on day one. A useful first version is session summaries plus a small, user-visible set of facts. Add episodic retrieval when evaluation shows users need it, and measure whether each layer improves outcomes enough to justify its cost and risk.

How I can help

I design agent architectures, including memory, retrieval and the governance around them, so assistants become more useful without becoming a liability. See my AI architecture work, or analyse your challenge and bring the result to a call.

What HTI-1 did

• USCDI v3 as the baseline. HTI-1 adopted the United States Core Data for Interoperability version 3 as the baseline standard from 1 January 2026, replacing version 1¹.
• Decision support interventions. It established a decision support interventions (DSI) criterion and added a predictive DSI category covering AI and machine learning capabilities².
• Source attributes. Developers supplying predictive DSIs as part of certified health IT had to make detailed source attribute information available for review, often described as model card requirements, with an enforcement date of 1 January 2025 for the DSI provisions².

What HTI-5 proposes

In December 2025 ASTP/ONC published HTI-5, a deregulatory proposed rule. Commentary describes it as proposing to remove 34 and revise seven of the 60 certification criteria⁵⁶. ASTP/ONC's own materials describe the DSI change as reducing the criterion's scope to remove the AI model card requirements³⁴. Some changes were proposed to take effect on publication of a final rule and others later; sources differ on the details⁵.

Holland & Knight characterised the year-end actions as a strategic pivot in federal health IT policy⁶. Check whether HTI-5 has been finalised, and in what form, before changing plans.

Why I'd keep building transparency anyway

Whatever happens to the certification criterion, the underlying questions don't go away:

• Customers ask. Health systems' AI governance committees often ask vendors what data a model was trained on, how it performs in different populations, and how it is monitored. A clear model description shortens procurement.
• Other regimes apply. Products sold internationally may face the EU AI Act, and regulated device software may fall under FDA pathways such as those in FDA PCCPs.
• Safety and liability. When a model contributes to a harmful outcome, the organisation that can show what it knew about the model, and how it monitored it, is in a far stronger position. See AI incident postmortems.
• Policy swings. Requirements removed in one cycle can return in another. Building transparency as a product capability is cheaper than switching it on and off.

What I'd build regardless of the rule

1. A model description per predictive feature: intended use, training data sources, known limitations, performance overall and by relevant subgroups, update history. Keep it as structured data, not a PDF.
2. Local validation before go-live at each customer, because performance varies between sites; federation helps, as in federated health analytics.
3. Ongoing monitoring for drift and subgroup performance, with a defined path for clinicians to report problems.
4. Change control: every model update versioned, evaluated against a fixed set, and recorded.
5. Interoperable data foundations: USCDI-aligned FHIR data, which most predictive features depend on, built with the patterns in FHIR Bulk Data pipelines and TEFCA.

These map neatly to general frameworks too, such as those compared in ISO 42001 and the NIST AI RMF, so one body of evidence serves several audiences.

For health systems

If you buy rather than build, don't let a relaxed certification requirement relax your procurement. Ask vendors for the same information HTI-1 described, contractually, and require notification of model changes. Your governance committee's questions shouldn't depend on which rule is in force this year.

How I can help

I help health tech companies and health systems build clinical AI with transparency, monitoring and interoperable data foundations that hold up across changing rules. See healthcare data architecture, or analyse your challenge and bring the result to a call.

Why the catalog holds the power

In an Iceberg lakehouse, the catalog tracks which tables exist, their current metadata and schema, and increasingly who may access them. Modern catalogs go further and issue engines short-lived storage credentials scoped to the tables they're allowed to read, a pattern called credential vending. Databricks' documentation, for example, describes Unity Catalog granting short-lived credentials to external systems that connect through its REST API or the Iceberg REST catalog interface, subject to explicit privileges and metastore settings⁴.

That is good security design. It also means the catalog is where permissions, lineage, audit and access for every engine live. Changing the catalog later is far harder than changing an engine.

The open REST interface

The Iceberg REST catalog API is what makes multi-engine lakehouses practical: any engine that speaks it can work with any catalog that implements it. Apache Polaris is an open-source catalog built around that API, described by the Apache Software Foundation as a fully featured catalog for Iceberg that enables interoperability across engines such as Spark, Flink and Trino¹.

Polaris entered the Apache incubator in 2024 after being donated by Snowflake and Dremio, and the foundation announced it as a top-level project in March 2026¹². Commentary on the graduation noted multiple releases and a broad contributor base², and the 1.4 release added multi-tenancy and deployment improvements³. Top-level status means independent Apache governance, which matters if neutrality is part of why you chose an open catalog.

Unity Catalog takes a different route: a catalog with its own API that also exposes Iceberg REST access for external engines, alongside Delta support⁴⁵. Check the current status of each capability, since some external access features have been labelled preview.

Questions I ask when choosing

Patterns that keep options open

1. Standardise on the Iceberg REST interface for engine access, even if your catalog also offers proprietary APIs. That keeps engines swappable.
2. Keep governance definitions as code: roles, grants and policies in version control, applied to the catalog, so they can be reapplied elsewhere.
3. Use one catalog per governance domain, not one per engine. Several catalogs over the same tables invite inconsistency.
4. Pair the catalog with data contracts, so table semantics and ownership travel with the data rather than living only in catalog metadata.
5. Test an exit. Once a year, prove that a second engine, or a second catalog, can read critical tables. It's cheap insurance.

The AI angle

AI systems are now among the biggest consumers of lakehouse data: feature pipelines, training sets, retrieval indexes and agents querying tables directly. A catalog that vends scoped credentials and records access gives you one place to control what each AI workload may read, which supports the evidence expected in AI governance frameworks. Without it, every AI project tends to get its own copy of the data, and governance dissolves.

How I can help

I design lakehouse and data platform architectures that stay open where it matters, from table formats and catalogs to governance and AI access. As a fractional CTO I can own the data platform roadmap, or review an existing one in technical due diligence. Or analyse your challenge and bring the result to a call.

The risks

Code execution on load. Hugging Face's own documentation warns that pickle, the default format for PyTorch weights, can enable arbitrary code execution when a file is loaded, and recommends safetensors as a safer alternative¹. Security researchers have reported malicious models on public hubs that used this to plant backdoors, and noted that pickle isn't the only risky format: some Keras models can run code through Lambda layers².

Scanners are imperfect. A 2026 study of malicious pickle-based models reported that existing scanners varied widely in how well they detected them on its dataset, which is a reason not to treat a scan as proof of safety⁷.

Tampering and substitution. Without signatures, there's no reliable way to know a model file is the one its publisher released, or that the copy in your registry hasn't changed since it was approved.

Unknown lineage. Fine-tuned models inherit everything about their base model and training data, including licence terms and possible poisoning. Without records, you can't answer basic questions in a security review or due diligence.

The building blocks

The checklist

1. Use an internal model registry. Production never pulls directly from public hubs. Models are imported, reviewed and stored internally.
2. Prefer safe formats. Accept safetensors or equivalent where possible; convert or reject formats that execute code on load. Load untrusted files only in isolated sandboxes.
3. Scan on import, while remembering scanners miss things. Scanning reduces risk; it doesn't certify safety.
4. Pin by hash, not by name. Reference models by immutable digest so "latest" can never silently change what runs.
5. Sign what you produce and verify what you run. Sign fine-tuned and approved models at the end of your pipeline, and verify signatures at deployment.
6. Generate provenance in your training pipeline, aiming for at least signed provenance from a managed build platform, in the spirit of SLSA Build L2.
7. Keep an ML-BOM per model: base model, datasets, licences, training configuration and evaluation results, linked to your AI system register.
8. Restrict runtime permissions. Model-serving containers need no outbound internet access and minimal privileges, enforced with the policies in Kubernetes admission policies.
9. Re-evaluate after every model change with your evaluation set, including safety and behaviour tests, not just accuracy.

Beyond the weights

The supply chain extends past model files. Datasets, tokenisers, prompt templates, agent tools and MCP servers all shape behaviour. MCP servers in particular deserve the same allow-listing and review, as covered in securing MCP servers. Google's announcement of model signing noted plans to extend the approach to datasets and other ML artefacts⁴, which is the right direction.

Why this matters now

Enterprise customers and regulators are starting to ask where models come from and how they are controlled. An ML-BOM, signed models and a registry answer those questions concretely, and support governance frameworks such as those compared in ISO 42001 and the NIST AI RMF. They are also inexpensive to adopt early and painful to retrofit after an incident.

How I can help

I design secure AI platforms, from model registries and signing to provenance, runtime controls and evaluation, for organisations running private and open-weight models. See my AI architecture work, or analyse your challenge and bring the result to a call.

Different jobs

A helpful way to see it, set out in one comparison of openEHR, FHIR and CDA, is as layers rather than rivals: FHIR for transactional exchange, openEHR for native persistence with clinical governance, and CDA for archival documents¹.

What two-level modelling buys you

openEHR's central idea is separating the software model from clinical content. A stable reference model sits at the first level. Archetypes, each describing one clinical concept such as blood pressure, and templates, which combine and constrain archetypes for a specific use such as a discharge summary, sit at the second². Clinicians can define and evolve content without the software changing underneath.

Archetypes are governed centrally and reused globally through the Clinical Knowledge Manager⁶. In community discussions, practitioners argue that archetypes express domain semantics independent of technology, while FHIR profiles tend to be scoped to particular implementations³. That's a perspective from the openEHR side, but the point about governance is fair: a well-reviewed archetype for blood pressure means the same thing in every system that uses it.

What FHIR does better

FHIR was built for exchange: resources, RESTful APIs and a web-developer-friendly approach⁵. It has become the default in regulation and procurement. US interoperability rules, prior authorisation APIs and bulk data access all build on it, as covered in CMS-0057-F and FHIR Bulk Data pipelines. Developers can be productive with FHIR quickly, and tools, libraries and test servers are plentiful.

Using a FHIR server as the primary store is where opinions diverge. One practitioner write-up argues it works for simpler cases but struggles with complex documents, temporal queries and archetype-level governance⁴. That is a vendor perspective; plenty of organisations do run FHIR-native repositories successfully. The question is how much clinical modelling depth and long-term governance you need.

The pattern I use: three layers

1. Persistence. Where the organisation needs deep, governed clinical records over decades, an openEHR clinical data repository holds the source of truth, with templates agreed with clinicians.
2. Exchange. FHIR APIs expose data to apps, partners, payers and national exchanges. A facade maps between openEHR compositions and FHIR resources, a pattern described in practice write-ups⁴.
3. Analytics. An OMOP CDM, built downstream, serves research, quality and AI, with standard vocabularies as described in clinical terminologies and OMOP and the pipeline in FHIR to OMOP.

Where deep clinical modelling isn't needed, for example a digital health start-up with a narrow product, a FHIR-native store plus OMOP for analytics is often simpler and perfectly adequate.

How to decide

• Who is the customer? National or regional health systems planning decades ahead lean towards openEHR for persistence. Products integrating with many partners lean towards FHIR first.
• Who owns clinical content? If clinicians need to define and govern data models directly, openEHR's tooling is built for that.
• What does regulation require? Exchange obligations almost always mean FHIR, whatever you store internally.
• What skills can you hire? FHIR skills are more widely available; openEHR expertise is concentrated in particular regions and vendors.
• What will you analyse? Neither is an analytics model. Plan for OMOP or a similar layer from the start.

Why it matters for AI

Clinical AI depends on data that means the same thing across sources and over time. Governed archetypes, consistent FHIR profiles and standard vocabularies all serve that goal. The worst foundation for AI is a mix of local models with no shared semantics, whatever standard name is on the label. That is an ontology problem first, and the same principles apply as in ontology, knowledge graph and semantic layer.

How I can help

I design clinical data platforms that combine persistence, exchange and analytics standards sensibly, for health systems and health tech companies in the US, Europe and the Gulf. See healthcare data architecture, or analyse your challenge and bring the result to a call.

Why LLM traffic breaks ordinary load balancing

The Kubernetes project's introduction to the Inference Extension puts it plainly: LLM inference sessions are often long-running, resource-intensive and partially stateful, and a single GPU-backed server may hold several sessions and in-memory token caches. Load balancers that route by HTTP path or round-robin don't account for model identity, request criticality or the current state of each server¹.

Two requests to the same model can differ in cost by orders of magnitude, depending on prompt and output length. And as covered in LLM inference serving, sending a request to a replica that already holds its prompt prefix in cache is far cheaper than sending it anywhere else.

How the extension works

The Gateway API Inference Extension builds on Gateway API, keeping the familiar Gateway and HTTPRoute model and adding inference-specific steps¹⁵. Its central resource is the InferencePool: a pool of model-server pods on shared compute, similar to a Service but aware of model serving¹. The project documentation lists InferencePool as generally available since v1.0.0².

The original design separated responsibilities by persona: platform operators manage where and how models are served through pools, while model owners manage what is served, including traffic splitting between versions¹. The model-facing APIs have continued to evolve, so check the current documentation for your version.

The request flow

A request reaches the gateway, which matches an HTTPRoute to an InferencePool. Instead of forwarding to any pod, the gateway consults an endpoint selection extension, often called the endpoint picker, which examines live pod metrics such as queue length, memory use and loaded adapters, and chooses the pod that can serve the request most efficiently¹. Gateway implementations describe the picker tracking signals such as KV-cache utilisation and queue length⁴.

To the client it's still one ordinary request. The intelligence lives in the gateway layer, where the platform team controls it.

What the benchmark showed

The project compared the extension with a standard Kubernetes Service for a vLLM deployment on H100 GPUs, ramping traffic from 100 to 1,000 queries per second. Throughput was roughly on par, but the extension showed significantly lower p90 per-output-token latency at higher load, and lower end-to-end tail latency as traffic rose beyond about 400 to 500 queries per second¹. The authors attribute this to avoiding hotspots that ordinary load balancing creates for long-running inference requests¹.

How I'd adopt it

1. Start from Gateway API. If your clusters still use an Ingress controller, the move to Gateway API is the prerequisite and has its own value.
2. Pick a gateway implementation that supports the Inference Extension, and confirm which version of the APIs it implements.
3. Expose metrics from your model servers that the endpoint picker needs, and check them under load before trusting the routing.
4. Load test against your own traffic, comparing tail latency and cost per request with your current setup, as in LLM inference serving.
5. Use pools to separate concerns: interactive and batch traffic, regulated and general workloads, or different hardware classes.
6. Combine with scheduling and cost controls: GPU allocation through DRA, training jobs through gang scheduling, and spend visibility through FOCUS-based cost allocation.

Where it fits in a private AI platform

An inference gateway is the routing layer in the platform described in private LLMs on-prem and where to run models in the UAE. Above it sits the policy gateway that handles authentication, model approval and logging; below it, the serving engines and GPUs. Keeping those layers distinct makes each easier to change.

How I can help

I design Kubernetes platforms for private and sovereign AI, from GPU scheduling to model-aware routing and cost control. See my AI architecture work, or analyse your challenge and bring the result to a call.

What structured outputs are

You supply a JSON schema, and the provider constrains generation so the response must match it. OpenAI introduced Structured Outputs in August 2024, available through strict function definitions and a JSON schema response format, and reported that its model at the time achieved 100% schema adherence in its own evaluations¹. The approach restricts which tokens the model may produce at each step to those consistent with the schema, building on techniques from open-source constrained generation tools².

Anthropic's documentation describes two related features: JSON outputs that shape the response, and strict tool use that enforces schema compliance on tool inputs through grammar-constrained sampling³⁴. Both providers document limits on which JSON Schema features are supported, and output can still be incomplete if generation stops early, for example at a token limit¹³.

The reasoning trade-off

Constraining format has a cost. In "Let Me Speak Freely?", presented at EMNLP 2024, researchers compared constrained decoding, format instructions in the prompt, and generating free text first then converting it. They found that strict format restrictions such as JSON mode were associated with a significant decline in reasoning performance, with stricter constraints generally doing more harm. On classification tasks, though, strict formats could help by reducing answer selection errors⁵.

The practical lesson is to separate thinking from formatting:

• For extraction and classification, strict schemas are usually a clear win.
• For tasks that need reasoning, give the model room to reason first, either in a free-text field within the schema placed before the answer fields, or in a separate step, then produce the structured result. The paper also found that simple corrective steps, such as reformatting outputs that fail to parse, improve reliability⁵.
• Measure both ways on your own evaluation set before standardising.

Designing schemas that help the model

1. Keep schemas small and flat where possible. Deeply nested, highly optional schemas are harder for models and may hit provider complexity limits.
2. Use enums for categories instead of free strings, so downstream systems never see an unexpected label.
3. Make uncertainty representable. Include an explicit "unknown" or "not found" option and a place for a short reason, so the model isn't forced to invent a value to satisfy a required field.
4. Describe fields clearly. Field descriptions are part of the prompt; vague names produce vague values.
5. Order fields deliberately. Put evidence or reasoning fields before conclusions, so the model generates support before the answer.
6. Version schemas like any interface, with the contract discipline in data contracts.

Validate the content, not just the shape

Once format failures disappear, the remaining failures are about meaning. The checks I put after every structured call:

• Business rules in code: totals add up, dates are in range, referenced IDs exist, a refund doesn't exceed the order value.
• Cross-checks against source: extracted values that should appear in the input document actually do.
• Confidence routing: low-confidence or "unknown" results go to a person instead of straight into the system of record.
• Evaluation on real cases, field by field, as in evaluation sets that predict production. Report accuracy per field, not just "valid JSON rate".

Structured outputs and agents

For agents, strict tool definitions mean every tool call arrives with well-formed arguments, which removes a whole category of runtime errors. It doesn't make those calls safe. Arguments still need authorisation and policy checks outside the model, and the session-level containment described in prompt injection defence and MCP tool annotations. A schema-valid instruction to transfer money to the wrong account is still the wrong instruction.

A note on portability

Providers implement structured outputs differently, with different parameters and supported schema subsets. If you use several models, as in model routing, keep one canonical schema in your codebase and let a gateway translate it per provider, then test each route. Self-hosted models can get similar behaviour from constrained decoding in the serving layer, covered in LLM inference serving.

How I can help

I design LLM integrations that downstream systems can trust, from schemas and validation to evaluation and agent controls. See my AI architecture work, or analyse your challenge and bring the result to a call.

What 'stable by default' means

The committee's goals were explicit¹:

• All OpenTelemetry distributions should be stable by default, with standard mechanisms to opt in to experimental features.
• A single, consistent set of stability criteria, including documentation, performance testing and benchmarks.
• Easier stabilisation of instrumentation libraries, and federation of semantic conventions.
• Epoch releases that are easier for organisations to consume.

For a platform team, the practical promise is fewer surprises: stable components you can rely on, and clearly marked opt-ins for anything that isn't.

The Kubernetes example

The Kubernetes attributes processor, which enriches telemetry with metadata such as pod, namespace and node, shows what that journey involves. Its stability depended on the Kubernetes semantic conventions it emits, so the two efforts moved together²³:

The v1.0.0 announcement is clear that moving to the new convention names means some breakage for existing users, and points to a migration guide³⁵. During the transition the processor provided feature gates, processor.k8sattributes.EmitV1K8sConventions and processor.k8sattributes.DontEmitV0K8sConventions, so teams could emit the new attributes before dropping the old ones².

Why attribute renames hurt more than they look

Kubernetes attributes are everywhere in an observability estate: dashboard filters, alert rules, service level objectives, cost allocation by namespace, retention policies, access controls and the queries engineers keep in runbooks. A renamed attribute doesn't throw an error. It just makes a filter match nothing, so a dashboard goes quiet and an alert never fires.

The migration plan I use

1. Inventory dependencies on attribute names. Export dashboards, alert rules, recording rules and saved queries, and search them for the Kubernetes attributes in your telemetry. Include cost reports that group by namespace or workload, as in Kubernetes FinOps.
2. Read the migration guide for your exact version of the Collector and processor⁵, and list old and new names side by side.
3. Emit both, then switch. Where your version supports it, emit the new conventions alongside the old in a non-production environment, update dashboards and alerts to the new names, then stop emitting the old ones.
4. Use the gateway layer to manage the transition centrally rather than on every node, which is one reason I favour the agent-to-gateway pattern in Collector deployment patterns.
5. Add canary checks: alerts that fire when a key dashboard query returns no data, so silent breakage is caught.
6. Pin Collector versions and upgrade deliberately, with a changelog review for every component you use.

A broader stability strategy

The Kubernetes processor is one component among many working towards stability³. The same pattern applies elsewhere, from the span events deprecation to GenAI conventions that are still evolving, covered in LLM observability. My rules for platform teams:

• Prefer stable components in production, and record each experimental component you depend on with the reason and an owner.
• Treat telemetry schemas as interfaces, with the same versioning discipline as APIs and the contract thinking in data contracts.
• Budget a small, regular amount of time for observability upgrades, rather than a painful catch-up every two years.

How I can help

I design and modernise observability platforms on OpenTelemetry and Kubernetes, including migrations like this one that keep dashboards and alerts working. As a fractional CTO I can own that roadmap. Or analyse your challenge and bring the result to a call.

1. Does the AI actually work, and how do they know?

• Is there a maintained evaluation set? Real examples, labelled with expected outcomes, run on every model or prompt change. If quality is judged by demos and anecdotes, that's the single biggest red flag. I describe what good looks like in evaluation sets that predict production.
• Is reliability measured, not just average accuracy? For agents, a single success rate hides inconsistency. Research on τ-bench introduced pass^k, the chance of succeeding on every one of k attempts, and found that agents were notably less consistent than their single-run success rates suggested⁵. See evaluating AI agents.
• Is there production monitoring? Feedback, escalations, sampled reviews and drift checks, tied to the evaluation set.
• Can they show improvement over time, with numbers from the same evaluation set across releases?

2. What is actually defensible?

• Which parts are theirs? Separate the third-party model, open-source components and the company's own work: data pipelines, evaluation, domain logic, workflow integration, proprietary data.
• What happens when the underlying model improves? If a better general model would make the product unnecessary, the moat is thin. If better models make the product better, that's healthier.
• Is there a data advantage? Proprietary data, feedback loops from usage, or curated domain knowledge such as a governed ontology or knowledge graph are harder to copy than prompts.
• How portable are they across model providers? A gateway and evaluation set that let them switch models are both a risk control and a negotiating position. See build vs buy for AI.

3. Do they have the right to their data?

• Provenance. Where did training, fine-tuning and retrieval data come from, under what licence or agreement?
• Customer data use. Do contracts allow customer data to be used for training or evaluation? Is that used in practice, and is it consistent with the contract?
• Personal and sensitive data. How is personal, health or financial data handled in prompts, logs, embeddings and evaluation sets? Residency matters for some markets, as in UAE health data and Saudi PDPL.
• Deletion. Can they actually delete a customer's data from every store, including vector indexes and caches?

4. Is the AI secure and contained?

• Known risk categories. Has the team assessed the product against a recognised list such as the OWASP Top 10 for LLM Applications¹?
• Prompt injection. The UK NCSC has argued that prompt injection may never be fully mitigated the way SQL injection was⁶, so the question is containment: what can a compromised agent actually do? See prompt injection defence.
• Tool and data permissions. Are agent tools scoped per user, enforced outside the model, and logged? See securing MCP servers.
• Tenant isolation. Can retrieval ever return another customer's data? Ask for the test that proves it can't.

5. Is the engineering organisation healthy?

• Delivery metrics. Deployment frequency, lead time, change failure rate and recovery time, with trends. DORA's 2025 research found that AI tends to amplify an organisation's existing strengths and weaknesses⁴, so a team leaning heavily on AI coding tools without good foundations may be accumulating risk; see DORA metrics in the AI era.
• Key-person risk. Is the AI capability concentrated in one or two people? What's documented?
• Decision records. Are major architecture choices written down, as in architecture decision records?

6. Do the unit economics hold?

• Cost per outcome. Model, infrastructure and human review cost per resolved ticket, document or transaction, not just per token.
• Sensitivity. What happens to margins if usage grows ten times, or a provider changes pricing?
• Levers in use. Caching, routing and right-sized models, as in prompt caching and model routing. Unused levers are upside; no visibility at all is a risk.

7. What is the regulatory exposure?

• Classification. Could any use case fall into a regulated or high-risk category under the EU AI Act³, or sector rules in health, finance or employment?
• Governance evidence. An AI system register, risk assessments, incident handling and training. Frameworks such as the NIST AI RMF² are a reasonable yardstick; see ISO 42001 or NIST AI RMF.
• Customer commitments. What have they promised enterprise customers about AI, and can they evidence it?

Reading the results

No early-stage company will score well on everything, and that's not the point. Diligence should separate fixable gaps, such as missing monitoring or undocumented decisions, from structural problems, such as no data rights, no evaluation discipline or a product that a better general model would replace. The first belong in a post-investment plan with owners and dates. The second belong in the valuation conversation.

How I can help

I carry out technical due diligence on AI and software companies for investors and acquirers in the US, Europe and the Gulf, and help portfolio companies close the gaps afterwards as a fractional CTO. Or analyse your challenge and bring the result to a call.

How it works

When a model processes a prompt, it builds internal state for every token. If the next request begins with exactly the same tokens, that work can in principle be reused rather than repeated. Self-hosted serving engines do this with prefix caching on top of memory management techniques such as PagedAttention⁵, which I cover in LLM inference serving. Hosted model providers expose the same idea as a pricing and latency feature.

The key word is prefix. Caches match from the start of the prompt. Change one token near the beginning and everything after it is a miss.

How the major providers differ

Prices, minimum sizes and lifetimes change, and differ by model and by cloud platform. Read the current documentation for the models you actually use, and measure, rather than relying on any table, including this one.

Why most teams miss

The usual culprits are small and easy to fix once you look:

• A timestamp or request ID at the top of the system prompt, so every prefix is unique.
• The user's name or account details injected into the system prompt rather than the user turn.
• Tool definitions in a different order on each request, because they're built from an unordered map.
• Retrieved documents placed before the static instructions.
• Prompt templates edited constantly, invalidating caches across all traffic with every deployment.
• Traffic too sparse for the cache lifetime: if a given prefix is used once an hour and the cache lasts minutes, it will rarely hit.

The playbook

1. Order prompts from most stable to least stable. Static instructions and tool definitions first, then slowly changing reference material, then per-user context, then the request itself.
2. Make serialisation deterministic. Sort tool definitions and keys, fix whitespace and formatting, and version templates so the same version always produces the same bytes.
3. Move volatile values to the end. Dates, user details and request metadata belong in the final turn, not the system prompt.
4. Use explicit breakpoints where available, placing them after the large stable sections. Consider the longer lifetime for traffic that is regular but not constant, and compare the extra write cost with the expected reads¹.
5. Measure hit rates from usage data. Providers report cached tokens in responses¹²³. Record them per route in your gateway alongside cost, as in FOCUS and AI spend.
6. Treat prompt changes as cache events. A template change resets caching for that route. Batch changes and watch cost after deployment.
7. Route consistently. If you use several models or regions, send requests with the same prefix to the same destination where possible, so caches have a chance to warm. The same logic applies to replicas in self-hosted serving.

Where it pays most

• Agents. Long system prompts and large tool catalogues, repeated across many steps of the same task. Caching often makes multi-step agents economically viable; see evaluating AI agents for keeping them reliable too.
• Document question answering. When users ask several questions about the same long document, cache the document once and vary the question, which pairs well with the approach in long context or RAG.
• Batch processing with a shared instruction block and many short inputs.
• Coding assistants that resend large amounts of repository context on each turn.

It pays least for short prompts, highly personalised prompts and very low traffic, where little is shared or the cache expires between uses.

Caching, routing and the gateway

Prompt caching interacts with the other cost levers. Model routing that sends similar requests to different models can split traffic and lower hit rates on each. Retrieval that returns results in a different order each time defeats caching of the retrieved section. A gateway that owns prompt assembly, routing and measurement in one place is the simplest way to manage these trade-offs, and it's the same gateway that enforces security policies in securing MCP servers.

Data protection note

Caching keeps processed prompt state on the provider's side for a period. For regulated data, check each provider's documentation and your agreements on how cached content is isolated and retained, and whether that is compatible with obligations such as those in the HIPAA Security Rule proposal or Gulf residency rules. For the most sensitive workloads, self-hosted serving with prefix caching keeps the cache inside your own boundary.

How I can help

I review AI products for cost and latency, from prompt structure and caching to routing and serving, and build the gateway measurements that keep savings visible. See my AI architecture work, or analyse your challenge and bring the result to a call.

How OHDSI does it

The OHDSI community is one of the largest working examples. The Book of OHDSI describes a network of more than 100 databases standardised to the OMOP CDM, collectively representing over a billion patient records at the time of writing¹. A study becomes a network study when it runs across multiple CDMs at different institutions¹.

The rules are strict and simple. Each site runs the same study package against its own CDM. A local analyst reviews the outputs before anything leaves, and minimum cell counts suppress small numbers. Only aggregate results such as summary statistics, estimates and diagnostics are shared. In the book's words, OHDSI network studies do not share patient-level data, and patient-level data from different sites is never pooled centrally¹.

Why it works: standardise first

Federation only works if the same code means the same thing everywhere. That is why OHDSI insists on the CDM and its standardised vocabularies. The book is explicit about the design shift this requires: you are building a study package to run at sites whose data you will never see, so concept sets must be exhaustive rather than tuned to your own data, and code must be parameterised for different database platforms¹.

The same discipline shows up in cohort definitions: definitions are portable, versioned artefacts, not local SQL.

The operational reality

The book is candid that network studies take real coordination. Site start-up can involve ethics approvals, database permissions, configuring an analysis environment, reviewing study code and getting dependencies installed within local IT constraints¹. Data quality varies between sites, and study leads may review quality reports or run checks to understand variation¹. It also notes that much of the process has been manual, with the community working to systematise it¹.

In practice, the technical parts are rarely what slows a network down. Governance, trust and site capacity are. Architecture can help by making each site's job as small and predictable as possible.

Where regulation is heading

Europe. The European Health Data Space regulation, Regulation (EU) 2025/327, was published in March 2025 and sets up a framework for secondary use of health data for purposes such as research, innovation and public health³. Commentary notes that secondary use of personal electronic health data is to take place within secure processing environments, with health data access bodies handling permits³⁴. Check the regulation for application dates, which are phased.

The Gulf. The UAE's health data law restricts storing and processing health data outside the country unless an exception applies, with research among the circumstances commentary describes⁵; I cover the detail in UAE health data localisation. Saudi Arabia's transfer rules add their own conditions, as in Saudi PDPL and cross-border AI.

Different regimes, same direction: analysis has to go to the data, or to a controlled environment, rather than data going wherever analysts happen to be.

The architecture I use

1. A standard node. Each site gets the same deployable stack: an OMOP CDM, vocabularies, data quality tooling as in OMOP data quality checks, and an analysis runtime. Infrastructure as code makes new sites predictable.
2. Signed, reviewed study packages. Analyses arrive as versioned packages, reviewed and approved before they run, never as ad hoc queries.
3. Output controls at the edge. Minimum cell counts, automated disclosure checks and a human review step before results leave the site.
4. A coordination layer that never sees patient data: study registry, package distribution, results collection and audit trail.
5. Monitoring of the network itself: which sites are current on CDM and vocabulary versions, data quality status, and study participation.

And AI?

Federated approaches extend to model development: training or evaluating models site by site and sharing parameters or metrics rather than data. The same principles apply, with more care, because model outputs can leak information in ways summary statistics usually don't. At minimum, I'd use federation for external validation: testing a model built at one site against many others before anyone claims it generalises. That is where a lot of clinical AI falls short, and a network makes it routine.

How I can help

I design federated health data platforms and OMOP networks, from the standard site node to governance and results pipelines, for research groups, health systems and life sciences companies in the US, Europe and the Gulf. See healthcare data architecture, or analyse your challenge and bring the result to a call.

Three findings worth knowing

Position matters. In "Lost in the Middle", Liu and colleagues studied how models use long inputs in tasks such as multi-document question answering. Performance was often highest when the relevant information appeared at the beginning or end of the context and dropped when it was in the middle¹. A later survey of long-context modelling describes this as a U-shaped curve⁵. The practical implication: where a fact sits in a long prompt can change whether it's used.

Advertised windows overstate useful windows. NVIDIA's RULER benchmark extended simple needle-in-a-haystack tests with harder tasks such as multi-hop tracing and aggregation. Models that scored almost perfectly on the simple test showed large drops as context length grew, and the authors report that many models fell below their quality threshold well before reaching their claimed context lengths². Finding one sentence in a haystack is a much easier job than reasoning across a long document.

Longer inputs degrade reliability. Chroma's 2025 "Context Rot" report tested 18 models, including several frontier models, on deliberately simple tasks, keeping difficulty constant while increasing input length. Performance grew increasingly unreliable as input grew, and models didn't use their context uniformly³. The authors note that real tasks are more complex, so the effect may be larger in practice, and they published their code for replication³⁴.

The other costs of stuffing the context

Quality isn't the only issue. Every token in the prompt costs money and adds latency on every request, unless the same prefix is cached and reused, which I cover in LLM inference serving. Long prompts are also harder to debug: when an answer is wrong, it is much harder to tell which of 200 pages misled the model than which of five retrieved passages did.

There's a governance cost too. Retrieval enforces access control naturally: you only fetch documents the user is allowed to see. "Put everything in the prompt" either ignores permissions or rebuilds them in a less tested way. For regulated data, that alone usually settles the question, as discussed in the HIPAA Security Rule proposal and UAE health data localisation.

When long context genuinely helps

None of this means long context is a gimmick. It changes what good retrieval looks like:

• Bigger, more coherent chunks. Instead of 300-token fragments, retrieve whole sections or whole documents, so the model sees definitions and exceptions alongside the rule.
• Single-document deep work. Reviewing one contract, one codebase module or one patient record against a checklist is a good fit, especially with the most important material placed at the start or end.
• Fewer retrieval misses. A wider net for recall, followed by a model that reads more candidates, can beat aggressive filtering that drops the right passage.
• Stable reference material. Long, unchanging context such as a policy manual or schema can be cached as a prefix and reused across requests.

How I design it: retrieve, then read generously

The common thread is what is now called context engineering: deciding deliberately what goes into the window, in what order, and what stays out. Retrieval, ranking, summarisation and memory are all tools for that decision. A large window gives you more room to be generous; it doesn't remove the need to choose.

Test it on your own data

Published benchmarks tell you the direction of the effect, not its size for your documents. Before choosing an approach, I run a simple comparison on the client's own evaluation set, as described in evaluation sets that predict production:

1. Baseline RAG with the current chunking and top-k.
2. RAG with larger chunks and more results, using the long window to read more.
3. Full-document long context for the subset of questions where it is feasible.
4. Position tests: the same answer-bearing passage placed at the start, middle and end of the context.

Measure accuracy, citation correctness, latency and cost per answer for each. The result is usually a hybrid, and the numbers make the case to stakeholders far better than any article, including this one.

The bottom line

Bigger context windows are a genuine improvement. They make retrieval more forgiving, enable new single-document workflows and reduce some engineering effort. But the evidence on position effects, effective context length and degradation with input size means the most reliable systems still choose what the model reads. Treat the window as a budget to spend well, not a bucket to fill.

How I can help

I design retrieval and context strategies for AI products, from chunking and ranking to long-context workflows, and test them against real data before they ship. See my AI architecture work and the RAG to production checklist, or analyse your challenge and bring the result to a call.

Two different kinds of thing

A common way to put it: the NIST framework is something you use, while ISO/IEC 42001 is something you certify against⁴. Commentators note that ISO's management-system elements, such as internal audit, management review and continual improvement, have no direct equivalent in the NIST framework³.

They map onto each other

NIST's AI Resource Center hosts a crosswalk between the AI RMF and ISO/IEC 42001. Commentators point out that it was contributed by the community rather than written by NIST, and that inclusion doesn't imply endorsement³⁵. Informally, the four NIST functions line up with ISO's leadership and policy, context and impact assessment, monitoring, and review and improvement activities³. Treat such mappings as starting points rather than proof of coverage.

How I'd sequence them

1. Start with an AI system register. Every framework, and every regulation, begins with knowing which AI systems you have, what they do and who owns them.
2. Apply the NIST functions to your highest-risk systems first. Map context and impacts, measure with real evaluations, manage with controls and incident response.
3. Build the evidence as you go: evaluation results, approval records, incident postmortems, training records. These are what auditors and customers ask for.
4. Add the management system when customers or markets ask for certification: policy, roles, internal audit, management review.
5. Map once to many. The same controls support the EU AI Act, SOC 2 and Gulf regimes such as DIFC Regulation 10.

Where engineering carries the load

Frameworks fail when they live only in documents. The controls that hold up in an audit are the ones built into systems: a model gateway that enforces approved models and logs every call, evaluation pipelines that gate releases as in evaluating AI agents, incident handling as in AI incident postmortems, and role-based training as in AI literacy. Build those and the paperwork follows; write the paperwork alone and it drifts from reality within months.

How I can help

I help AI companies build governance that satisfies enterprise buyers and regulators, with the controls engineered into the platform. As a fractional CTO I can own the programme. Or analyse your challenge and bring the result to a call.

Why the change

OpenTelemetry has offered two ways to emit events correlated with traces: span events via the Tracing API, and log-based events via the Logs API¹³⁴. The announcement lists the problems with having both: split guidance for library authors, duplicate concepts for operators, and slower evolution because every improvement had to be specified twice¹. The community has converged on a simpler model: events are logs with names, emitted via the Logs API and correlated through context, a direction set out in an earlier post on logging¹².

What changes and what doesn't

As the authors put it, they are deprecating the API for recording span events, not the ability to see events attached to spans¹.

What I'd do now

1. Update instrumentation guidelines. New code emits events and exceptions as log-based events correlated with the current span.
2. Check the logs pipeline. If logs aren't collected through OpenTelemetry yet, events will have nowhere to go. A Collector gateway, as in Collector deployment patterns, is the natural place to route them.
3. Audit dashboards and alerts that depend on span events, especially exception counts and error views, and test them after each instrumentation upgrade, as the project recommends¹.
4. Confirm backend support for showing log-based events in trace views, so engineers don't lose context when debugging.
5. Watch costs. Moving events to the logs pipeline may change how they are priced and retained by your vendor. Model it before the switch.

Why this matters for AI telemetry

AI applications produce a lot of event-like data: prompts, responses, tool calls, guardrail decisions. Treating these as named, structured log events correlated with traces fits the GenAI observability work described in LLM observability with OpenTelemetry. It also makes it easier to apply separate retention, redaction and access rules to sensitive content than when it is embedded in spans.

How I can help

I design observability platforms on OpenTelemetry and help teams plan migrations like this one without losing visibility. As a fractional CTO I can own that roadmap. Or analyse your challenge and bring the result to a call.

The main terminologies

Medical vocabularies go back centuries; The Book of OHDSI starts with London's Bills of Mortality. Over time each country, system and institution tended to develop its own, which is exactly what makes interoperability and multi-site research hard¹.

How OMOP unifies them

The OMOP Standardized Vocabularies consolidate many vocabularies into one common format, refreshed regularly by the OHDSI vocabulary team, and they are mandatory reference tables for every OMOP CDM instance¹³. Organisations download them from ATHENA, which also offers search across vocabularies; proprietary vocabularies require a licence¹².

The key idea is the standard concept. For each clinical meaning, one concept is designated standard and used to record data. The book's example: codes in MeSH, CIEL, SNOMED, ICD-9-CM and Read all describe atrial fibrillation, but only the SNOMED concept is standard; the others are source concepts mapped to it¹. Classification concepts, such as MedDRA terms, take part in hierarchies for querying but aren't used to record data¹.

Standard designation is decided per domain, not per vocabulary: conditions mainly use SNOMED, measurements SNOMED and LOINC, drugs RxNorm, RxNorm Extension and CVX¹. In the book's words, there is no such thing as a "standard vocabulary".

Domains are messier than they look

Every concept is assigned a domain, such as condition, drug, procedure or measurement, which decides which CDM table it lands in. Source vocabularies often mix domains; the book shows that procedure vocabularies like CPT4 and HCPCS contain concepts from several domains¹.

Why this matters for health AI

• Features and cohorts depend on it. A model trained on "diabetes" is only as good as the concept set behind it, as in OMOP cohort definitions.
• Portability depends on it. A model built on standard concepts can be validated at another site; one built on local codes usually can't.
• LLMs need grounding. Language models are good at suggesting codes and poor at being reliably right. Constrain them to look up concepts in the vocabulary rather than generate codes from memory, and validate outputs, in the spirit of SHACL validation for knowledge graphs.
• Hierarchies enable better questions. Querying all descendants of a concept finds patients that a flat code list would miss.

For the architecture around the vocabularies, see FHIR to OMOP and OMOP data quality checks.

How I can help

I design health data platforms with terminology and mapping at their core, from OMOP vocabularies to ontologies that make health data usable for AI. See healthcare data architecture and ontology and knowledge graphs, or analyse your challenge and bring the result to a call.

Built-in policies, in CEL

ValidatingAdmissionPolicy reached general availability in Kubernetes 1.30. It lets you express validation rules declaratively in Common Expression Language (CEL), evaluated inside the API server, as an alternative to validating webhooks¹². The GA announcement walks through replacing a webhook that enforces least-privilege security context settings with an equivalent policy¹.

MutatingAdmissionPolicy does the same for changes. The documentation describes it as a declarative, in-process alternative to mutating webhooks, with mutations defined as an apply configuration merged using server-side apply rules, or as a JSON patch³. The feature gate list shows it moving through alpha and beta to stable in v1.36⁴.

A policy is made of a policy resource, a binding that scopes where it applies, and optional parameters, so one policy can be reused with different settings per namespace or environment²³.

Webhooks, built-in policies, or a policy engine

My rule of thumb: simple, high-volume guardrails move to built-in policies; checks that need external data or rich reporting stay in a policy engine.

The migration playbook

1. Inventory webhooks and policies across clusters, including those installed by vendor charts, with their failure policies and timeouts.
2. Classify each rule: expressible in CEL, needs external data, or needs complex logic.
3. Write CEL equivalents for the first group, with test cases for allowed and rejected objects, kept in version control alongside your platform code.
4. Roll out in audit or warn mode first, comparing results with the existing webhook before switching to deny.
5. Use parameters and bindings to vary settings per environment instead of copying policies.
6. Remove the old webhook only after the policy has matched its decisions in production for a period.

Guardrails worth having on AI clusters

• GPU workloads must declare an owner, cost centre and priority class, so costs and preemption are traceable, supporting AI cost allocation.
• Model-serving pods must not run privileged and must come from approved registries.
• Namespaces holding regulated data must carry labels that other controls depend on, such as region or data class, as in UAE health data localisation.
• Training jobs must use the queueing and scheduling setup in Kubernetes gang scheduling rather than bypassing it.

How I can help

I design Kubernetes platforms with guardrails that are simple to operate, and review cluster security in technical due diligence. As a fractional CTO I can own the platform roadmap. Or analyse your challenge and bring the result to a call.

A useful vocabulary

Team Topologies, the framework from Matthew Skelton and Manuel Pais, describes four team types¹:

• Stream-aligned teams deliver value along a flow of work, usually a product or customer journey.
• Platform teams provide internal services that reduce the load on others.
• Enabling teams help other teams build new capabilities, for a limited time, then move on.
• Complicated-subsystem teams own parts that need deep specialist knowledge.

And three ways for teams to interact: collaboration, working closely towards a shared goal, mainly during discovery; X-as-a-Service, consuming what another team provides with minimal coordination; and facilitating, one team helping another learn¹². Interactions that fit none of these are often a sign that team boundaries are drawn badly².

How I map AI work onto it

Interaction modes over time

The modes should change as capability matures. Early on, an AI platform or enabling team collaborates closely with one or two product teams to discover what's needed. Once patterns stabilise, the platform offers them as a service: an approved gateway, evaluation pipelines, templates. Enabling help moves on to the next team. If a product team still needs constant hand-holding a year later, the platform's interfaces are probably the problem, which is exactly what the platform engineering maturity model helps diagnose.

Conway's law applies to AI too

Melvin Conway observed in 1968 that organisations design systems that mirror their own communication structures³. In AI, the effect is easy to spot: a central AI team produces a central AI service that every product must queue for; fully scattered teams produce five slightly different gateways and no shared evaluation. Design the team structure you want the architecture to reflect.

What I put in place first

1. Name owners for AI features in each product team, including their evaluation and incident handling.
2. Create a small AI platform capability for the gateway, evaluation and observability, run as a product with internal users; see LLM model routing and AI cost allocation for what typically lives there.
3. Time-box enabling work, with explicit exit criteria for each team supported.
4. Declare interaction modes between teams, so expectations are clear and revisited.
5. Record the design as an architecture decision record and revisit it each half year.

How I can help

As a fractional CTO, I help companies organise teams and architecture together, so AI work scales without bottlenecks or duplication. Or analyse your challenge and bring the result to a call.

What pgvector gives you

pgvector is an open-source Postgres extension for vector similarity search, supporting exact search as well as approximate indexes such as HNSW and IVFFlat¹. The strongest argument for it isn't performance. It's that vectors live beside your relational data, under the same transactions, backups, access controls and operational tooling.

That matters most for filtering. Real RAG queries are rarely "the nearest chunks"; they are "the nearest chunks this user may see, from this tenant, in this product line, updated this year". In Postgres those filters are ordinary SQL against data you already govern.

The filtering problem, and the 0.8 fix

Approximate indexes return a limited candidate set, and filters apply afterwards. With a selective filter, that can leave very few results: the Nile write-up of the release cites an example where a condition matching 10% of rows yields only about four results with default HNSW settings⁴.

pgvector 0.8.0, released in late 2024, added iterative index scans: if the first pass returns too few matches, the scan continues until it finds enough or hits a configured limit, with options for strict or relaxed ordering²³. It also improved cost estimation so the planner makes better index choices when filtering². The release notes add a useful reminder: where an ordinary index such as a B-tree can meet performance needs, it gives exact results².

How I decide

A middle path is common: start on pgvector, measure, and move only the search workload if it outgrows Postgres. If your operational database is MongoDB, the equivalent question applies there, as in hybrid search for RAG on MongoDB.

Practical rules for pgvector

1. Benchmark with your filters and your data, measuring recall as well as latency.
2. Enforce permissions in the query, ideally with row-level security, so retrieval can't return documents a user shouldn't see. The same principle applies to tenants, as in multi-tenant SaaS patterns.
3. Separate search load with read replicas if it begins to affect transactional performance.
4. Combine with keyword search; Postgres full-text search plus vectors is a reasonable hybrid for many products.
5. Evaluate retrieval quality with a labelled set, as in the RAG to production checklist, before tuning indexes.

How I can help

I design retrieval architectures and data platforms for AI products, choosing the simplest stack that will scale. See my AI architecture work, or analyse your challenge and bring the result to a call.

What τ-bench measures

Sierra's τ-bench put agents in realistic customer service domains. The agent talks to a simulated user, calls domain APIs and must follow a written policy. Success is judged objectively: does the final state of the database match the expected outcome¹²?

It also introduced pass^k: the probability that an agent succeeds on all k independent attempts at the same task¹. That is a reliability measure, unlike pass@k, which asks whether any attempt succeeds. The authors reported that even state-of-the-art function-calling agents at the time succeeded on fewer than half the tasks, and that consistency was worse, with pass^8 below 25% in the retail domain¹.

When users act too

Real support often involves the customer doing things: restarting a device, checking a setting, confirming a code. τ²-bench extended the idea to a dual-control setting where both agent and user use tools to change a shared environment, and reported that performance dropped noticeably when agents moved from no-user settings to dual control³⁴. Guiding a person through a task is a skill of its own, and it needs its own tests.

How I build agent evaluations

1. Judge outcomes, not transcripts. Wherever possible, check the end state in the system of record, as τ-bench does, rather than asking a model whether the conversation looked right.
2. Write policies down, then test them. Refund limits, identity checks and escalation rules become explicit test cases, including ones where the right answer is "no".
3. Simulate users with variety: impatient, confused, adversarial, and users who change their minds. Include prompt injection attempts, as in prompt injection defence.
4. Run every task several times and report pass^k alongside the single-run rate. Release gates should use the reliability number.
5. Grow the suite from production. Every incident and escalation becomes a test case, per AI incident postmortems.
6. Trace everything. Tool calls, arguments and results, so a failing run can be diagnosed; see LLM observability.

This sits on top of the general method in evaluation sets that predict production, with two additions specific to agents: state-based judging and repeated trials.

What to do with the results

Low reliability isn't always a reason to stop. It's a reason to change the design: narrower tools, deterministic code for policy checks, human approval for irreversible actions, and the containment patterns in MCP Apps and tool annotations. Often the best improvement isn't a better model but less for the model to decide.

How I can help

I design agent architectures and the evaluation suites that decide whether they're ready for production. See my AI architecture work, or analyse your challenge and bring the result to a call.