Kubernetes v1.37 'Garhwal': stability, pod identity, and AI/ML scheduling

AI-Generated Infrastructure: Why Your Terraform Code Is a Security Risk (And How to Fix It)

LLMs can generate Terraform code—but only 3 out of 7 models produce security-compliant IaC. A recent study benchmarks models for secure infrastructure generation and reveals how to use AI safely.

Kubernetes v1.37 — codenamed Garhwal, after a Himalayan region in Uttarakhand, India — landed on 26 August 2026. It ships 67 enhancements: 16 graduated to Stable, 23 to Beta, 27 to Alpha, and one deprecation/removal. As usual, the release is less about headline features and more about the quiet maturity of the control plane: fewer outages under load, first-class workload identity, and real scheduling primitives for AI/ML and HPC.

This article walks through what actually changed and what you should plan for before you upgrade. For the canonical list, see the upstream release blog.

Control plane resilience

The headline stability work is the completion of resilient watchcache initialization. The WatchCacheInitializationPostStartHook gate is now Stable and locked on (it already defaulted to on since v1.36). The practical effect: when the API server starts or recovers, watch-cache warm-up no longer floods etcd with list/watch requests. Instead of letting expensive requests overload etcd or exhaust API Priority and Fairness, kube-apiserver now delegates bounded requests and rejects the rest with HTTP 429. That directly lowers the risk of control-plane outages in large clusters — and it is a signal to operators: your controllers and operators must handle 429 Too Many Requests gracefully (respect Retry-After, exponential backoff).

Two supporting changes land alongside it: etcd RangeStream (Beta, on by default, needs etcd 3.7+) streams large lists in chunks instead of buffering one blob, and ConcurrentWatchObjectDecode (now on by default) moves per-event decode/transform onto a bounded worker pool — together cutting cache-init time by roughly 40–55 % at scale. And the metrics.k8s.io API finally graduates to Stable after nearly nine years in Beta, locking in the standard way HPA and kubectl top read CPU/memory. KYAML, a safer YAML subset, also reaches Stable (kubectl get -o kyaml).

Workload identity: Pod Certificates & ClusterTrustBundles

One of the most consequential GA features is Pod Certificates and ClusterTrustBundles (SIG Auth). Kubernetes now has first-class support for distributing private keys, X.509 certificates, and trust bundles to Pods:

• A signer controller watches PodCertificateRequest objects, issues and refreshes certs for eligible Pods, and maintains the matching ClusterTrustBundle.
• A workload opts in via a podCertificate projected volume with a chosen signer name, and can mount a ClusterTrustBundle projected volume for trust anchors.

This removes a whole class of bespoke init-container/cert-rotator glue and is directly useful for mTLS, SPIFFE-style identity, and service-mesh bootstrap — squarely in the sovereignty/security territory this site keeps returning to.

Security & admission

Manifest-based admission control graduates to Beta: admission webhooks and CEL policies can be loaded from manifest files on disk (staticManifestsDir in AdmissionConfiguration) instead of only the Kubernetes API. Policies loaded this way are enforced from API-server startup, keep working while etcd is unavailable, and can protect the API-based admission resources themselves from modification. For clusters where admission is the last line of defense, that bootstrap-time enforcement matters.

Also Beta: improved handling for undecryptable resources — admins can now find and remove resources the API server can no longer decrypt via the API (with safeguards), instead of poking at etcd directly.

AI/ML and HPC scheduling

This is where v1.37 does the most interesting work, and it aligns with the industry's pivot to training and inference at scale.

• Gang scheduling graduates to Beta via the Workload API and PodGroup: all-or-nothing scheduling so a group of Pods only lands when the cluster can host the whole group — preventing the partial-scheduling deadlocks that waste GPU nodes. It adds workload-aware preemption and PodGroup queueing, and explicitly addresses livelock when multiple workloads race the scheduler.
• Workload-aware preemption (Beta) lets the scheduler consider a PodGroup as a unit, so it stops disrupting individual Pods that don't actually unblock a workload.
• DRA keeps maturing: ResourceClaim status (with network interface data), extended-resource requests served by DRA drivers, device taints/tolerations, and a standard numaNode attribute are all Stable; ResourceClaim support for workloads is Beta; and Alpha adds derived attributes, device compatibility groups, and node allocatable accounting.
• Pod-level resource managers (Beta, off by default) let the topology/CPU/memory managers allocate for the whole Pod as a NUMA-aligned unit — valuable for tightly coupled AI/ML containers plus sidecars.
• Memory QoS with cgroups v2 is Beta and on by default, using memory.min/memory.low/memory.high to protect requested memory and throttle before hard limits.
• In-place Pod resize gets two Alpha follow-ups: scheduler preemption for resize, and dynamic resize of memory-backed (emptyDir, medium: Memory) volumes — no restart.
• HPA scale to zero (Beta, default on) lets object/external-metric HPAs scale to 0 and back via spec.minReplicas: 0 — handy for idle queue consumers, batch jobs, and GPU workloads.

For teams running TrainJob, JobSet, LWS or RayJob, the new Workload Aware Scheduling Controller APIs (Alpha) and spec.scheduling on batch/v1 Job provide a common framework so each controller exposes gang scheduling and topology constraints consistently.

Storage, autoscaling, observability

• Storage Version Migrator is now Stable and in-tree: declare a StorageVersionMigration and the control plane rewrites existing resources after an API upgrade or encryption change — no more hand-rolled kubectl get | replace scripts.
• PVC "last used" tracking (Beta, on by default) adds an Unused condition so you can finally see (and reclaim) orphaned PVCs.
• CSI-aware Cluster Autoscaler integration (Beta, opt-in) fixes a long-standing race where freshly created nodes without CSINode data got over-packed with volume-backed Pods.
• Native histogram metrics (Beta) adopt Prometheus native histograms for memory-efficient, finer-grained distributions.
• cAdvisor-less, CRI-full stats (Beta, off by default) move container/Pod stats to a single source of truth from the runtime.

Deprecations to plan for

• kube-dns is deprecated. CoreDNS has been default since v1.13; expect no new kube-dns packages after v1.40. Migrate now.
• kube-proxy ipvs mode is deprecated. It now logs a startup warning; disabled by default by v1.40, removed by v1.43. (Reminder: ipvs always used iptables underneath anyway.)
• kubectl run --filename/-f is deprecated — the generated Pod comes purely from CLI args.
• Static Pods can no longer reference Secrets/ConfigMaps (the long-standing bug is fixed; PreventStaticPodAPIReferences gates it).

Upgrade notes

Two caveats worth testing before you bump a production cluster:

1. SELinux mount change (Stable, default on). Volumes now mount with -o context=<label> instead of recursive relabeling — but only when the CSI driver opts in via .spec.seLinuxMount: true. A mount carries one SELinux context, so Pods with different SELinux labels that previously shared a volume can now fail to start. Set .spec.seLinuxChangePolicy: Recursive on a Pod to keep the old behavior (still switchable until v1.38).
2. maxUnavailable for StatefulSets is back on by default after a v1.36 bug (stuck CrashLoopBackOff) was fixed.

Take

v1.37 is a "make it boring" release: it hardens the control plane, gives workloads real identity and scheduling primitives, and quietly deprecates the cruft (kube-dns, ipvs). If you run AI/ML or HPC on Kubernetes, the gang-scheduling and DRA work is the part to pilot now — and if you care about sovereign, self-hosted infrastructure, Pod Certificates plus manifest-based admission are the security primitives worth adopting early.