LWKD: Week Ending July 16, 2023
Developer News
Priyanka Saggu of ContribEx has announced some changes to Org Membership requirements, mainly letting memberships expire after 1 year of inactivity. If you have submitted multiple PRs to Kubernetes, consider becoming an Org Member today. Please reach out to ContribEx on Slack or anywhere if you have any questions about the new guidelines or on becoming a member.
Release Schedule
Next Deadline: CODE FREEZE July 18th/19th
Code Freeze starts tonight (for folks in North America) or tomorrow morning (for everyone else). Is everything checked in? This is followed by the regular release countdown: open docs PRs by Thursday, publish deprecations blog on Thursday, test freeze and finalize docs next week. As always, once we're in Code Freeze, please respond to test failure issues with alacrity.
As if that weren't enough, patch releases are due out Wednesday, including a Go version bump. This will be the last patch for 1.24.
Featured PRs
#118642: ValidatingAdmissionPolicy: Variable Composition
As we've covered over many releases, CEL support in our API is big and growing feature in Kubernetes. As more functionality has been exposed, and folks try more complex use-cases, performance has become more of a concern. One very common pattern in CEL-driven validations is reusing the same chunk of expression code in multiple places, such as in both the validator expression and the error message to provide direct feedback about the rejected field. The naive way to approach this is to just duplicate the code in both places but we are programmers and that makes us all sad so now we have a better approach: variables. Much like variables in any other programming environment, you can set one to a value and then it will only need to be evaluated once and can be reused in other contexts.
As this is still declarative code we have to set up variables in our YAML:
variables:
- name: envVarNames
expression: spec.template.spec.containers[0].env.map(e, e.name)
validations:
- expression: variables.envVarNames.all(n, n.startWith("MYAPP_"))
- expression: variables.envVarName.indexOf("UNSAFE") == -1
Variables can refer to other variables and can be used anywhere in the same CEL scope. This should allow for both improved performance of expression evaluation and more maintainable CEL code through de-duplication.
#119232 and #118883: Index as a pod label
Two separate PRs but both adding the same feature for each type: exposing the pod's index as a label. For a while now, indexed Jobs have supported exposing it as an annotation so it could be read by the Downward API but as we all know, annotations can't be searched on which was a frequent request. And on the StatefulSet side the only option was to parse the pod name which is never very fun. Now both support similar labels: statefulset.kubernetes.io/pod-index
and batch.kubernetes.io/job-completion-index
. As with the annotation, these can be exposed down to the pod via Downward API volumes or environment variables, or used in label selectors in all the usual places.
And while we're on the topic of indexed Jobs, #118009 and #119294 have also landed providing the ability to control the failure handling on a per-index basis rather than only for the job as a whole. This mirrors similar workflow control features in other job-runner frameworks.
Other Merges
- CEL and ValidatingAdmissionPolicy features: type-checks messageExpressions via a controller including authorizer variable, namespace data access from expressions, cache authz checks
- CEL supports Kubernetes Quantities, and exposes authz errors
- Metrics: cloud-controller-manager node sync, forced detaches, revert behavior of
sync_proxy_rules_iptables_total
, encryption hot reloads, replaceapiserver_storage_db_total_size_in_bytes
withapiserver_storage_size_bytes
, count all pod deletion activity - DRA: Faster scheduling for pods with ResourceClaims, report NodePrepareResource errors, add CDIDevice IDs, reserve resources using pre-scheduled Pods, generate resource names without conflict, limit ResourceClaim permissions
- AP&F: endpoint that tracks currently executing requests, improve exempt borrowing, make Max seats configurable
- cgroupsv2: full swap support including QoS, detect the cgroup driver from CRI
- kube-proxy supports Ingress connectivity better, and logging format is configurable
- Add schedulerQueueingHints feature gate
- The EndpointSlice Reconciler has its own staging repo
- Support confirming deletion interactively via
kubectl delete -i
- Restartable init containers (aka sidecars) support readiness and liveliness probes
- Batch all prepare/unprepare operations for pods, allowing intelligent plugin behavior
- CRDs can automagically fail validation if they haven't fixed any of the prior validation failures
- flushFrequency can be an int or a string
- Prefer connections to running, not terminating, pods
- Make sure that websocket protocol is available everywhere in the API, per enhancement
- Expose Windows commit memory through CRI
- Hash the KeyID being logged by KMS
- IP Mode field for LB to allow CCMs to optimize traffic routing
- Swap usage endpoint in SummaryAPI and metrics
- Don't fetch CRD specs until an API request
- Enable Consistent Reads from the Watch Cache
- Deduplicate OpenAPIv2 parameters, reducing spec size by 55%
- Don't try to mount a volume until it's reconstructed
pvc.Status.AllocatedResourceStatus
is the newpvc.Status.ResizeStatus
- status.HostIPs field in the PodAPI and DownwardAPI
- Contextual Logging: kube-controller-manager, disruption controller, endpoints and slices
Promotions
- AdmissionWebhookMatchCondition to Beta, including metrics, tests, and webhook smart reload
- IPTablesOwnershipCleanup to GA, including deprecating old Kubelet flags
- Promote *_metric_total metrics to Beta
- APIserver flowcontrol metrics to Beta
- GetAllocatable podresources endpoint to GA
- NUMA node policy to Beta for TopologyManager
- Non-graceful Shutdown to GA
- ProbeTerminationGracePeriod to GA
Deprecated
- Remove KUBECTL_EXPLAIN_OPENAPIV3 ENV var, since OpenAPIv3 is required
- Remove deprecated
--lock-*
scheduler flags - Deprecate
apiserver_flowcontrol_request_concurrency_limit
- Deprecate Ceph CSI migration tools since everyone should be on CSI already
- Remove ability to serve deprecated Policy v1beta1 types
Version Updates
- Use go 1.20.6 for all supported versions
- python-client to v27.2
- distroless-iptables to 0.2.6
- debian images to bookworm, including setcap
- cadvisor to v0.47.3