September 2026 briefing: GPUs get smarter scheduling, idle workloads scale to zero, and observability grows up

The month in one paragraph

HorizontalPodAutoscaler scale to zero is now Beta and on by default¹. Dynamic Resource Allocation (DRA) Extended Resource support reached GA, along with device taints and tolerations². The Workload and PodGroup APIs that enable gang scheduling went to Beta³. Rootless node components went to Beta⁴, and nodes gained standard conditions for drains and maintenance⁵. In observability, OpenTelemetry reported better Prometheus interoperability⁶ and a v1.0.0 Kubernetes attributes processor⁷.

Scale to zero, finally in core

Before v1.37 you needed an add-on, or an alpha feature gate, to scale a workload from zero. Now an HPA using an object or external metric, such as queue length, can take a workload to zero replicas and bring it back when work arrives¹. The post is honest about the limits: CPU and memory metrics cannot drive it because there are no pods left to measure, cold starts are real, and Kubernetes Services do not buffer requests while nothing is ready, so HTTP workloads need their own buffering layer¹.

GPUs: DRA and gang scheduling grow up

DRA Extended Resource support went GA. In plain terms, a pod that asks for something like example.com/gpu can now be served by a DRA driver without a separate device plugin, so existing workloads keep working while allocation moves to DRA underneath². Device taints and tolerations are also stable, which lets an operator take a single faulty accelerator out of service without draining the whole node².

On scheduling, the Workload and PodGroup APIs that enable gang scheduling moved to Beta, together with workload-aware preemption, and a new CompositePodGroup API arrived for multi-level topology constraints³. For distributed training, gang scheduling is the difference between a job that starts cleanly and one that holds half its GPUs while waiting for the rest.

Safer, more operable nodes

Rootless mode, where the kubelet, container runtimes, CNI plugins and kube-proxy run as a non-root user inside a Linux user namespace, graduated to Beta⁴. The motivation is simple: node components have had container-breakout vulnerabilities in the past, and running them without full root limits the damage.

Kubernetes also introduced five standard node conditions, including DrainInProgress, MaintenancePlanned and GracefulNodeShutdownInProgress⁵. Until now every tool had its own way to say a node was going away. A shared signal makes it far easier for schedulers, autoscalers and long-running jobs to react sensibly.

Observability: the Prometheus gap is closing

The OpenTelemetry End-User SIG surveyed users of both projects. The average ease-of-use rating rose from 3.1 to 3.6, and the share finding the two hard to use together fell from 29% to 10%⁶. Nearly half of respondents run Prometheus exporters and OTel receivers side by side rather than migrating, and 65% run a "vanilla" pipeline with no vendor transformation⁶. The sample is small (81 screened respondents), so I read it as direction rather than precise measurement.

The Kubernetes attributes processor reached v1.0.0, built on Kubernetes semantic conventions that became stable in June⁷. The move to stable attribute names does break existing users, so check the migration guide before upgrading.

That timeline is the useful reminder from the month's last OTel post: conventions take years to stabilise, and instrumentation libraries then have to catch up⁸. A stable convention is a starting point, not proof your telemetry follows it.

Healthcare: the FHIR API clock is ticking

In US healthcare, payers spent September reminding providers that the FHIR-based APIs required under the CMS interoperability and prior authorisation rule are due from 1 January 2027⁹. If you run a clinical data platform, that deadline is now one planning cycle away. I explain how I structure FHIR exchange alongside an analytical layer in FHIR to OMOP: a practical architecture.

What I'm watching next

Three threads from September will matter over the next quarter. First, how quickly the managed Kubernetes services pick up v1.37, because scale to zero and the DRA changes are only useful once your provider ships them. Second, whether GPU vendors' DRA drivers support device taints well enough to make single-accelerator maintenance routine. Third, how the semantic conventions conformance work in OpenTelemetry turns into something teams can check in CI.

None of these needs action today. They are the questions I would put on a platform team's agenda so the next upgrade is a plan rather than a surprise.

What I'd do with this

1. Move queue-driven AI jobs to scale to zero on external metrics, and leave user-facing inference alone.
2. Plan the DRA migration now for GPU clusters. Extended Resource support makes it gradual, so there is little reason to wait.
3. Use gang scheduling for multi-node training so jobs do not hold partial GPU allocations.
4. Wire node lifecycle conditions into your tooling so long-running jobs checkpoint before a planned drain.
5. Test the k8s attributes processor upgrade in staging, because attribute names change.

How I can help

I design AI and data platforms that keep GPU costs under control and stay observable as they grow. See my AI architecture work, my healthcare data architecture practice for the FHIR deadline, or analyse your challenge and bring the result to a call.