December 19, 2023

LWKD: Week Ending December 17, 2023

Last Week in Kubernetes Development

Week Ending December 17, 2023

Developer News

Kubernetes 1.29 was released! Major features include KMSv2 encryption GA, nftables for Kube-Proxy, and APIs for IP address management. Find out more in the release notes.

Release Schedule

Next Deadline: 1.30 Cycle Begins, January 2024

We are in the holiday period between releases; dates for 1.30 have not been published yet. Release team selection is happening.

Patch releases are expected for all supported versions this week.

Featured PRs

#117046: client-go: allow adding indexes after informer starts

Indexers allow API clients to build complex query logic against their local streaming cache of objects. This can range from simple value-based lookups to more complex selector implementations. Up until now, it's been required to tell the API caches ahead of time, so they could be run as the caches were initialized. But Kubernetes clients, especially Operators which made heavy use of this feature, are complex and dynamic places and sometimes this became a frustrating limitation. No more! Now new indexes can be added at runtime and will automatically backfill the data for you. This unlocks many new options for Operator developers around dynamic object management and only paying the costs of indexing when it's needed.

#121912: CEL library extensions for IP Address and CIDR network parsing

As we've integrated CEL into more places, we've slowly found use cases that would be best served by having more CEL "standard library" functions. This PR adds helpers for IPs and CIDR masks, allowing for validation checks like "field must be an IP in canonical form" or "provided IP must match this CIDR mask".

• isIP(<string>) - Is the string a valid ipv4/ipv6 address
• isCIDR(<string>) - Is the string a valid ipv4/ipv6 subnet in CIDR notation
• ip.isCanonical(<string>) - Returns whether the IP address is in its canonical format
• ip(<string>).family() - Returns an integer (4 or 6) based on IP address network family
• `ip().isUnspecified() / ip().isLoopback() / ip().isLinkLocalMulticast() / ip().isLinkLocalUnicast() / ip().isGlobalUnicast()- Boolean values exposing netip.Addr methods for special IP addresses
• cidr(<string>).containsCIDR(cidr(<string>)) - is a particular CIDR contained within another CIDR
• cidr(<string>).containsIP(ip(<string>)) - is a particular IP contained within the CIDR
• cidr(<string>).containsCIDR(<string>) - is a particular CIDR contained within another CIDR
• cidr(<string>).containsIP(<string>) - is a particular IP contained within the CIDR
• cidr(<string>).ip() - Returns the IP representation of the CIDR address, can then be manipulated using ip() members
• cidr(<string>).masked() - Returns the masked representation of the CIDR object, can be used to compare whether the original CIDR was masked or not
• cidr(<string>).prefixLength() - Returns the prefix length of the CIDR network

Note that due to CEL compatibility rules, by default these functions will not be enabled in Kubernetes 1.30 as we require N-1 compat for objects. So this will be generally available in 1.31 instead.

KEP of the Week

KEP 2395: Removing In-Tree Cloud Provider Code

This KEP aims to remove all the cloud provider specific code from the kubernetes/kubernetes repository. This is helpful since it will allow the cloud providers to develop and release their code independent from the Kubernetes release cycle. The KEP goes into the details of how this change is to implemented in four phases. The cloud provider code would first be moved to staging. The controller managers of the respective cloud providers would be built from their own repositories in the second phase. In the third phase, all the cloud provider code would be moved to the respective k/cloud-provider-<provider> repository. In the final phase, the support for in-tree cloud providers would be disabled by setting the DisableCloudProviders and DisableKubeletCloudCredentialProvider flags to true by default.

This KEP graduated to beta in the recent v1.29 release.

Other Merges

• CEL gets IP address types and functions
• Cache namespace index for faster listing of large collections
• Scheduler NodeAffinity Queueing Hints are back although disabled by default, after a reversion, but Scheduler First Fit is still broken, and reverted
• client-go optimistically updates leader locks
• Purged some more leftover Dockershim code
• Contextual logging: client-go metadata
• Detect raw volumes correctly on reboot
• PodTopologySpread plugin understands scheduling hints, as does NodeResourceFit
• kubeadm cleans up after upgrade health check
• NodeInfo is available to scheduler plugins
• GenericAPIServer healthz code is shiny-new and refactored
• Time out etcd requests even when they're in decoding
• Daemonset controller ignores unrelated node events
• kubectl apply swaps prune-allowlist for deprecated prune-whitelist
• Don't accidentally make volumes local on reboot
• Users can mutate FSGroupPolicy and PodInfoOnMount in their CSI drivers
• Add PreScore point for VolumeBinding

Test Cleanup: autoscaling DNS

Deprecated

• GA FeatureGates being removed: IPTablesOwnershipCleanup, ExperimentalHostUserNamespaceDefaultingGate
• Drop --use-real-proxier from kubemark
• Remove ability to re-enable a long list of deprecated storage APIs

Version Updates

• kubedns and nodelocaldns to 1.22.28
• golangci-lint to 0.7.0

Subprojects and Dependency Updates

• containerd/nerdctl to 1.7.2: Reset default value of ulimit for nerdctl run.
• kubernetes-client/python v29.0.0 Alpha 1 is released.