LWKD: Week Ending November 6, 2022
Developer News
Code Freeze is coming, as is a mega-long list of merged changes and graduations.
The python-client developers are considering making major breaking changes, mainly for OpenAPI code. Add your feedback.
Release Schedule
Next Deadline: Code Freeze, Wednesday November 9th
Code Freeze is coming! This Wednesday at 5pm PST (November 10th 0100 UTC), we put a hold on all 1.26 merges that aren’t fixing failing tests. So get your last tweaks in! From here the deadlines are:
- Test Freeze: November 15th
- Docs Complete: November 29th
- Release Day: December 6th
Patch releases are also due out November 9th.
Featured PRs
#113274: [KEP-3521] Part 1: New Pod API .spec.schedulingGates
Pod objects have a fairly simple life cycle for their first few milliseconds. The API object is created, a scheduler sees that and tries to find a place to put it, if a node is found then the kubelet on that node wakes up and does Complicated Things. But what if you want to do some preparatory steps before all of this? Previously the only option was to make a validating webhook and reject the create in the first place, and arrange for the client to try again later when maybe the condition will be correct. This PR adds a better way, schedulingGates. This Spec field works kind of like finalizers but in reverse. When the scheduler sees a new unscheduled pod, it will check if the schedulingGates array is empty. If it’s not empty then no scheduling happens. Repeat any time the array changes. This means you can use mutating hooks on the create to inject a gate, wait until your condition to proceed is met, and then remove the gate and let things proceed as before. This opens up a lot of interesting possibilities with things like autoscaling, scheduler QoS, or fancy quota enforcement. If any of those sound up your alley, be sure to check out this alpha feature!
#109525: feat: make jsonpath wait logics consistent with condition
kubectl wait is one of the mainstays of “I just need to shim this one thing” shell scripts. Previously the --for=condition mode would behave as expected if the condition didn’t yet exist, it would keep waiting until the condition both existed and matched the requested state. But with --for=jsonpath, often used for sequencing load balancer and ingress setup, it would exit with an error if the path didn’t already exist. Now this behavior is unified so wait will stick around until the condition is met (the timeout expires) in both cases. This may fix some silent bugs in your scripts or it might be a good excuse to clean up any extra retry logic you built as a workaround.
#112011: Add ambiguous selector check to HPA
A potentially-breaking change for some folks, if multiple HPAs are configured against the same Pod, both with disable themselves and require you fix the configuration. Currently they would just both apply meaning whichever ran last would “win”. This does mean that when upgrading to 1.26, you should check for any such errors as your HPAs might have silently disabled themselves. You can also get a head-start on this now by checking for overly-broad selectors on your HPAs.
Other Merges
- Add descriptions for shell completion in Bash
- Try the alpha version of Windows host network
- Default to warning about podsecurity if you’re enforcing it
- APIserver persists lease names
- Plaintext option for
kubectl --explain - Allow specifying NUMA strategies in TopologyManger
- Handle field ownership when the user specifies
--server-side - Create etcd paths correctly and non-crashily
- Shutdown nodeipam controller non-leakily
- Allow auto-reload of encryption config and make sure KMS provider names are unique
- Fix node address validation; backported
- Disruption controller can patch stale pods status
- Publish events when turning TopologyAwareHints on and off
- Stop reusing stale VIPs on Windows
- Metrics: add deleting_pods_total, stability framework custom collectors, AP&F borrowing metrics, cronjob skew metric
- Don’t corrupt really long pod log lines
- Fix getAllocatableCPUs
- etcd resource versions are consistent for deleted objects
- Validate admission CEL expressions
- kube-proxy function naming cleanup
- Prevent devicemanager startup crash
- New disruption status for pods on failing kubelets
- Add pod start SLI/SLO histogram
- Rewrite less of iptables during syncs
- Allow disabling nodeports on localhost
- SPDY executor streams are cancellable
Test Overhaul: reusing/caching tests POC, naming of storage tests, code of storage tests, APIserver validation, NodeInclusionPolicy benchmarking, scheduler performance tests, large indexed job test, kubeadm reset, readWriteOncePod scheduling, APIserver tracing, disable cloud provider for tests, enabling NodeInclusionPolicy, podContainerManager, and improve formatting of e2e test output
Promotions
- Non-graceful shutdown to Beta (otherwise known as “cut the power cord”)
- JobTrackingWithFinalizers to GA, hopefully for the final time
- NodeInclusionPolicy to Beta
- ServiceInternalTrackingPolicy to GA
- WindowsHostProcessContainers to GA
- ProxyTerminatingEndpoints to Beta
- EndpointSliceTerminatingCondition to GA
- CSIMigrationVsphere to GA (time to migrate your storage, VMware users)
- Azure CSI Migration to GA (same to you, Azure users)
- RetroactiveStorageClass to Beta
- DelegateFSGrouptoCSIDriver to GA
- CPUManager to GA
- Kubelet DeviceManger to GA
- LoadBalancer mixed protocols to GA
- LegacyServiceAccountTokenNoAutoGeneration to GA, and we have a new candidate for Longest Kubernetes Feature Name
Deprecated
- Deprecate many unused
kubectl runflags,like--filenameand--timeout - Remove deprecated CRI v1alpha2 API and the last vestiges of dockershim
Version Updates
- Update golang to 1.18.8 in 1.24 and 1.19.3 in 1.25 and 1.26