Weekly GitHub Report for Kubernetes: April 28, 2025 - May 05, 2025 (12:01:14)
Weekly GitHub Report for Kubernetes
Thank you for subscribing to our weekly newsletter! Each week, we deliver a comprehensive summary of your GitHub project's latest activity right to your inbox, including an overview of your project's issues, pull requests, contributors, and commit activity.
Table of Contents
I. News
1.1 Recent Version Releases:
The current version of this repository is v1.32.3
1.2 Version Information:
The version release on March 11, 2025, introduces key updates and changes to Kubernetes, as detailed in the linked changelog, with additional binary downloads available for users. Notable highlights or trends from this release can be found in the Kubernetes announcement forum and the comprehensive changelog documentation.
II. Issues
2.1 Top 5 Active Issues:
We consider active issues to be issues that that have been commented on most frequently within the last week. Bot comments are omitted.
-
DRA kubelet: NodeUnprepareResources does not block pod removal when using zero terminationGracePeriod: This issue highlights a problem with the Kubernetes kubelet where the
NodeUnprepareResourcesfunction does not block pod removal when using a zeroterminationGracePeriod, leading to a failure in setting up the container during stress testing. The error encountered is related to a locked cache operation, which prevents the kubelet from retrieving claim information for a resource claim, causing the pod to fail to start as expected.- The comments discuss a potential race condition in the claim cache, with users attempting to reproduce the issue under different conditions and configurations. Some users report difficulty in reproducing the problem, while others identify that the issue may be related to the use of a zero
terminationGracePeriod, which bypasses normal pod shutdown procedures. There is a suggestion to improve claim info tracking to handle such cases better, and a user volunteers to investigate further. - Number of comments this week: 29
- The comments discuss a potential race condition in the claim cache, with users attempting to reproduce the issue under different conditions and configurations. Some users report difficulty in reproducing the problem, while others identify that the issue may be related to the use of a zero
-
Stop Supporting, Building and Releasing Artifacts for s390x: This issue discusses the proposal to stop supporting, building, and releasing artifacts for the s390x architecture in the Kubernetes project due to inadequate support, low user demand, and the significant burden it places on the community. The conversation highlights the challenges faced in maintaining s390x, including the lack of continuous integration infrastructure, the disproportionate time spent on build failures, and the minimal user interest compared to other architectures.
- The comments reveal a debate about the necessity of continuing support for s390x, with some arguing for its discontinuation due to low demand and high maintenance costs, while others emphasize the importance of s390x in critical industries and the efforts made by IBM to support it. There is a suggestion to set up non-release-blocking platform tests managed by trusted partners, and a discussion on the fairness of supporting s390x compared to other architectures like MIPS and RISCV. The conversation also touches on the historical engagement of the s390x team with the community and the potential impact of dropping official support.
- Number of comments this week: 12
-
Refactor pkg/scheduler/backend/queue tests to use mocks instead of a real metric registry: This issue involves refactoring the tests in the
pkg/scheduler/backend/queuedirectory of a Kubernetes project to use mock objects instead of a real metric registry. The goal is to replace the use ofmetrics.RegisterandMetricAsyncRecorderwith mock versions to improve testability, while retaining a single test case that verifies collected metrics using the current setup.- The comments discuss labeling the issue as a "good-first-issue" and "refactor," with some debate on the necessity of using mocks. One commenter questions the need for mocks, suggesting that they can create a gap between tests and reality, while another argues that mocks are necessary for testability in this case. The discussion includes a suggestion for initializing metrics in tests and questions the benefits of using mocks.
- Number of comments this week: 7
-
externalIPs DNAT rules are not installed when clusterIP is None: This issue describes a problem where DNAT rules for external IPs are not installed when a Kubernetes service is configured with
clusterIP: None, which is typically used for headless services. The user expected that the DNAT rules for external IPs would be installed similarly to when a cluster IP is specified, but this is not happening, leading to a potential bug in the service validation process.- The comments discuss that kube-proxy does not handle headless services, which is why external IPs cannot be added, and this behavior is consistent with iptables kube-proxy. There is a suggestion that using a
Gatewaymight be a better approach for accessing headless services externally, asexternalIPsis considered deprecated due to security vulnerabilities. The discussion also highlights that the Service API is designed for simplicity and not for infinite flexibility, suggesting that the Gateway API might be more suitable for complex use cases. - Number of comments this week: 5
- The comments discuss that kube-proxy does not handle headless services, which is why external IPs cannot be added, and this behavior is consistent with iptables kube-proxy. There is a suggestion that using a
-
code-generator/applyconfiguration-gen: Embedding imported Type results in <.EmbeddedIn.MemberType.Elem.Name.Name>: nil pointer evaluating *types.Type.Name: This issue describes a problem encountered when using the
applyconfiguration-gentool with a specific type that results in a nil pointer error related to embedded imported types. The error occurs when embedding an imported type, while embedding local types or not embedding the field works without issues.- The comments include tagging relevant teams and individuals for visibility and triage, categorizing the issue as a bug, and referencing the related API documentation.
- Number of comments this week: 4
2.2 Top 5 Stale Issues:
We consider stale issues to be issues that has had no activity within the last 30 days. The team should work together to get these issues resolved and closed as soon as possible.
As of our latest update, there are no stale issues for the project this week.
2.3 Open Issues
This section lists, groups, and then summarizes issues that were created within the last week in the repository.
Issues Opened This Week: 25
Summarized Issues:
- Dependency Update: The Kubernetes project needs to update the unsupported dependency
gopkg.in/go-jose/go-jose.v2to the supportedv4version. The current version is no longer maintained by the upstream repository, necessitating this update to ensure continued support and security.
- Kubelet and Pod Management Issues: The
NodeUnprepareResourcesfunction in the kubelet does not block pod removal with a zeroterminationGracePeriod, leading to container setup failures due to a locked cache operation error. Additionally, the kubelet fails to refresh immutable secrets after deletion and recreation, causing pods to access outdated secret values.
- Test Failures and Flakes: Several tests in the Kubernetes project are experiencing failures and flakes, including issues with node affinity scheduling, Windows test panics, and sidecar container termination order. These issues indicate potential underlying problems in the test setup or code logic that need to be addressed.
- Resource Management and Scheduling: Problems with pod resource management include OOM-killed pods incorrectly transitioning to a "Succeeded" phase and issues with the
Pod InPlace Resize Containerfeature. These issues can lead to unexpected behavior and resource allocation problems in Kubernetes clusters.
- Code Refactoring and Testing Improvements: Refactoring efforts in the Kubernetes project aim to improve testability and maintainability by using mock objects instead of real metric registries. This approach is being applied to various components to ensure more reliable and efficient testing.
- Feature Enhancements and Proposals: New feature requests and proposals include managing worker node draining as a job, adding support for ephemeral read-write image volume mounts, and setting custom gRPC metadata headers for probes. These enhancements aim to improve Kubernetes functionality and user experience.
- Architecture and Module Management: Discussions are ongoing about stopping support for the s390x architecture due to low demand and moving the
k8s.io/cri-apimodule out of the staging area for independent management. These changes are intended to streamline development and resource allocation.
- Bug Fixes and Code Improvements: Various bug fixes and code improvements are being addressed, such as eliminating unnecessary conversions in CEL bindings and aligning ratcheting behavior for atomic data types. These efforts aim to enhance code efficiency and consistency across the project.
- Cluster Initialization and Configuration Issues: Problems during cluster initialization include missing
ServiceAccountreferences inClusterRoleBindingresources and issues with the cpuset controller causing container exits. These issues highlight the need for better configuration management and error handling.
- Test and Code Generation Bugs: Bugs in test and code generation tools, such as the
applyconfiguration-gentool, are causing errors like nil pointer dereferences. Addressing these bugs is crucial for maintaining the reliability and accuracy of generated code and tests.
2.4 Closed Issues
This section lists, groups, and then summarizes issues that were closed within the last week in the repository. This section also links the associated pull requests if applicable.
Issues Closed This Week: 11
Summarized Issues:
- Kube-proxy IP Protocol Detection: The kube-proxy's IP protocol detection needs enhancement by implementing a preliminary check on Linux systems to verify the existence of "/proc/sys/net/ipv4" and "/proc/sys/net/ipv6" directories. This ensures that both IPv4 and IPv6 kernel subsystems are enabled before proceeding with specific IPtables and nftables checks, addressing compatibility issues on certain Linux distributions.
- CustomResourceDefinitions (CRDs) Conflict: A test failure occurs in the Kubernetes project due to a conflict when creating multiple CRDs of the same group and version but different kinds. The presence of an existing CRD causes the test to fail, highlighting a need for better handling of CRD conflicts.
- Resource Allocation Errors with Sidecar Feature: After introducing the sidecar feature, Kubernetes' default scheduler fails to reuse resources from the init container. This leads to Kubelet rejecting pods due to resource allocation errors when both the init and service containers request eight NPUs each.
- CephFS CSI Driver NodeResizeError: A Persistent Volume Claim (PVC) using the CephFS CSI driver remains in a NodeResizeError condition despite successful controller-side resizing. This results in persistent alerts and misleading status indicators, even though the filesystem is correctly resized and functional.
- DNS Query Timeout After Kubernetes Upgrade: DNS queries are timing out in all pods after upgrading Kubernetes from version 1.29 to 1.31. The issue persists despite downgrading CoreDNS, suggesting a potential configuration issue or a bug introduced in the newer Kubernetes version.
- Windows KubeProxy HNS LoadBalancer Issue: The Windows KubeProxy faces a problem where the HNS LoadBalancer fails to update its endpoints after the ModifyLoadBalancerAPI is invoked. This leads to a mismatch between the expected and actual endpoint counts in the HNS, despite using HNS version 15.4.
- CronJob Patch Error with podFailurePolicy: An error occurs when attempting to patch an existing
CronJobwith apodFailurePolicyconfiguration that lacks a requiredstatusfield. The documentation suggests the field is optional, but a workaround involves explicitly settingstatus: 'True'.
- Pod InPlace Resize Container Test Load: The Pod InPlace Resize Container tests in Kubernetes cause excessive load on the namespace controller. Each test creates and tears down its own namespace, leading to timeouts in other tests, and suggests sharing a single namespace among these tests to alleviate the problem.
- Kubelet Image Pull Failure from Private Registry: In a Kubernetes environment, attempts to pull an image from a private registry using kubelet fail due to inconsistent HTTP responses. The issue indicates a potential problem with the container runtime's handling of authentication or network requests.
- Incorrect 'kubtctl auth can-i' Command Result: The command 'kubtctl auth can-i get pods/exec' incorrectly returns 'yes' even though the user lacks the necessary 'pods/exec' privilege. The issue was due to using the wrong verb in the command and was resolved in Kubernetes version 1.32.1.
- Custom Readiness Probes for Kubernetes Nodes: A proposal for custom readiness probes for Kubernetes nodes aims to ensure they are fully prepared before being marked as ready. This addresses a concern linked to a previous issue in the kubernetes/autoscaler project.
2.5 Issue Discussion Insights
This section will analyze the tone and sentiment of discussions within this project's open and closed issues that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.
Based on our analysis, there are no instances of toxic discussions in the project's open or closed issues from the past week.
III. Pull Requests
3.1 Open Pull Requests
This section provides a summary of pull requests that were opened in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.
Pull Requests Opened This Week: 43
Key Open Pull Requests
1. Add kube-api-linter verify scripts: This pull request introduces scripts and configurations to enable the use of the kube-api-linter on the API staging repository, aiming to automate API reviews by enforcing standards with two initial rules: optionalorrequired, which ensures struct fields are explicitly marked as +optional or +required, and requiredfields, which mandates that required fields do not use pointers or have omitempty, thereby facilitating the gradual enforcement of API conventions.
- URL: pull/131561
- Merged: No
2. Declarative validation: Simplify handling of subresources: This pull request aims to simplify the handling of subresources in the Kubernetes project by funneling all validation of a kind into a single validation function, thereby eliminating the need for special casing subresources that share a kind, and laying the groundwork for future enhancements as outlined in a related Kubernetes enhancement proposal.
- URL: pull/131560
- Merged: No
3. e2e: fixing hardcode values and formatted text: This pull request addresses the replacement of hardcoded values with variables in the Kubernetes project's end-to-end tests, specifically in the pod_lifecycle.go and service.go files, to ensure that output messages align with the dynamically set httpPort and gracePeriod variables, thereby reducing errors and improving code maintainability.
- URL: pull/131571
- Merged: No
Other Open Pull Requests
- CEL Field Path Identification: This pull request introduces a feature that allows for the identification of field paths accessible by a CEL (Common Expression Language) expression. It is essential for declarative validation to adjust based on fields affected by a validation rule and includes documentation on handling cases where CEL functions lack visibility into accessed object fields.
- kubectl get crd Enhancement: This pull request enhances the
kubectl get crdcommand by adding additional columns—GROUP, SCOPE, VERSIONS, and CREATED AT—to the output. This provides users with improved visibility into the API group, scope, served versions, and creation timestamp of each CustomResourceDefinition.
- BoundedFrequencyRunner for kube-proxy: This pull request introduces a new, simplified BoundedFrequencyRunner for kube-proxy, refining the proxy update scheduling logic. It ensures updates occur only once per minimum interval, at least once per maximum interval, and at least once during a retry interval if a retry is scheduled, while removing the burst functionality.
- Kubelet Eviction Logging: This pull request aims to enhance the Kubernetes project by migrating the
evictioncomponent of thekubeletto use contextual logging. It is part of a cleanup effort and addresses part of an ongoing issue tracked in the project's repository.
- In-Place Memory Resizing for QOS Pods: This pull request introduces a feature that allows for in-place resizing of memory for guaranteed Quality of Service (QOS) pods when a static memory policy is enabled. It ensures that the QOS class of the pod remains unchanged during the resize, which was previously marked as infeasible.
- VolumeAttributesClass GA Promotion: This pull request aims to promote the
VolumeAttributesClassfeature and its associated API to General Availability (GA) within the Kubernetes project. It transitions tostorage.k8s.io/v1and includes updates to meet GA graduation criteria, as well as fixes for related issues and unit tests.
- String Concatenation Refactor: This pull request aims to refactor the code by replacing instances of string concatenation using the
+operator withfmt.Sprintf. It enhances consistency and readability across the codebase, particularly in test files, as part of a beginner-friendly contribution that adheres to Kubernetes code conventions.
- WatchWithContextFunc Introduction: This pull request introduces a new function,
WatchWithContextFunc, to the Kubernetes client-go library while deprecating the existingWatchFuncWithContext. It addresses naming inconsistencies in context-aware interfaces as part of a feature update.
- Kube-proxy iptables Error Logging: This pull request enhances the kube-proxy component by logging detailed iptables errors during the platformCheckSupported process. It helps in diagnosing issues related to iptables support on the host, such as permission problems or deprecated flag usage, thereby improving troubleshooting and debugging efforts.
- Filesystem Resize Process Update: This pull request proposes to disable the size checking performed during the resize process for ext and xfs filesystems by relying on
resize2fsandxfs_growto handle expansions. It maintains the existing behavior forbtrfsdue to a lack of detailed understanding.
- Namespace Creation Validation: This pull request introduces a feature that adds validation to the Kubernetes namespace creation process by implementing a new utility package called
nstld. It detects and warns users when they attempt to create a namespace with a name that matches a common top-level domain (TLD), such as "com", "org", or "net".
- Kube-proxy DNAT Rules Bug Fix: This pull request addresses a bug in the kube-proxy's nftables mode by ensuring that DNAT rules for external IPs are correctly installed for headless services. It adds an
isHeadlessfield to track such services and modifies theUsesClusterEndpoints()function for consistent handling.
- Testing Framework Enhancement: This pull request aims to enhance the testing framework for Kubernetes by adding channel test helpers to validate that watcher result channels close without error when the client stops the watcher. It also ensures that encoded watch events can be decoded correctly.
- Client-go Transport Debug Wrapping: This pull request proposes a cleanup in the Kubernetes client-go transport by allowing the option to force debug wrapping. It involves extending the transport.Config with an additional boolean to maintain backward compatibility while preserving the existing behavior when unset.
- OrderedNamespaceDeletion GA Promotion: This pull request aims to promote the "OrderedNamespaceDeletion" feature to General Availability (GA) within the Kubernetes project. It is currently open for review and not yet merged.
- NodeResizeError Bug Fix Cherry-Pick: This pull request is an automated cherry-pick of a previous commit (#131495) aimed at fixing a bug in the Kubernetes kubelet where an unexpected NodeResizeError condition appeared in the PersistentVolumeClaim (PVC) status. It is targeted for the release-1.33 branch.
- NodeResizeError Bug Fix Cherry-Pick for Release-1.32: This pull request is an automated cherry-pick of a previous change (#131495) to the release-1.32 branch. It addresses a bug in Kubernetes where an unexpected NodeResizeError condition appeared in the PersistentVolumeClaim (PVC) status.
- Go Version and Dependency Update: This pull request updates the Kubernetes project by bumping images, dependencies, and versions to Go 1.23.8 and distroless iptables. It is part of addressing issue 3944 on the Kubernetes release repository.
- Informer Panic Handling Bug Fix: This pull request addresses a bug in the Kubernetes project by ensuring that the
syncTracker.Finished()function is called even when a panic occurs in theAddFunchandler during initial list processing. It prevents the informer from being left in an unsynced state and potentially blocking controller startup.
- Pod Validation SYS_ADMIN Capability Bug Fix: This pull request addresses a bug in the Kubernetes pod validation process by ensuring that pods are denied the SYS_ADMIN capability when allowPrivilegeEscalation is set to false. It aligns with existing practices for CAP_SYS_ADMIN and adds a test to verify that pods cannot be updated to include this capability.
- Go Version and Dependency Update for Issue #3944: This pull request updates the Kubernetes project by bumping images, dependencies, and versions to Go 1.23.8 and using distroless iptables. It is part of the ongoing effort to address issue #3944, ensuring that Kubernetes is now built with the latest Go version.
- Volume Limit Fix in E2E Tests: This pull request addresses a cleanup task by fixing the volume limit for in-tree plugins in end-to-end tests. It ensures that the volume limit is obtained from the CSI node following the integration of previous changes in the Kubernetes project.
- Persistent Volume Validation Refactor: This pull request refactors the Persistent Volume (PV) mount option validation by moving annotation keys to a dedicated file. It renames and relocates the
volume.ValidatePersistentVolumefunction intocore.ValidatePersistentVolumeto enhance code organization.
- MountOption Validation Comment Fix: This pull request addresses a cleanup task by fixing incorrect comments in the Kubernetes project. It specifically relates to the lack of validation for the MountOption when creating a persistent volume, as highlighted in issue #42573.
- Kube-scheduler Alpha Metrics Enhancement: This pull request aims to enhance the kube-scheduler's alpha metrics by increasing the number and granularity of buckets for specific metrics. It introduces a feature gate called
SchedulerHighPrecisionMetricsto control the export of these metrics, thereby improving accuracy and reducing memory usage.
3.2 Closed Pull Requests
This section provides a summary of pull requests that were closed in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.
Pull Requests Closed This Week: 43
Key Closed Pull Requests
1. golangci-lint v2: This pull request updates Kubernetes to use the new golangci-lint v2, revisits check suppressions to fix broken code, and completes the migration started in a previous pull request by reorganizing suppression rules and addressing inefficiencies and bugs identified by the linter.
- URL: pull/131477
- Merged: 2025-05-03T06:03:55Z
2. verify-shellcheck cleanup: This pull request focuses on cleaning up the verify-shellcheck script by removing unused shell imports, switching to a minimal and more secure shellcheck image to reduce vulnerabilities and download size, and normalizing the KUBE_ROOT path using pwd -P.
- URL: pull/131594
- Merged: 2025-05-03T03:37:56Z
3. kubeadm: use slices package to reduce the codes: This pull request involves a code cleanup in the Kubernetes project by replacing loops used for condition checks with the more efficient slices package API, thereby reducing the overall code complexity.
- URL: pull/131488
- Merged: 2025-04-28T10:07:29Z
Other Closed Pull Requests
- Bug Fixes in Kubernetes: This topic covers multiple pull requests addressing various bugs in the Kubernetes project. These include fixing the OpenID Connect discovery document publishing process, resolving issues with NodeResizeError conditions in PersistentVolumeClaims, and ensuring default values for pod failure policies in cronjob objects are correctly applied.
- Feature Enhancements in Kubernetes: Several pull requests introduce new features to enhance Kubernetes functionality. These include adding a "Traffic Distribution" field to
kubectl describe service, introducing a lazy reflective CELref.Valwrapper, and adding the--authorization-webhook-cache-sizeflag for better scalability.
- Code Refactoring and Optimization: This topic includes pull requests focused on improving code quality and efficiency. These involve refactoring the
GarbageCollector.attemptToDeleteWorkermethod, optimizing the update process for DRA drivers, and replacingcontext.Backgroundwith the root command context.
- Tooling and Dependency Updates: Pull requests under this topic address updates to tools and dependencies used in the Kubernetes project. These include switching golangci-lint to a new module plugin mode and updating the misspell dependency to an actively maintained fork.
- Documentation and Configuration Improvements: This topic covers pull requests that improve documentation and configuration settings. These include correcting a typo in documentation, relaxing validation for the
--service-account-signing-endpointflag, and adding a new reviewer to the project.
- Testing and Validation Enhancements: Pull requests in this category focus on improving testing and validation processes. These include addressing flaky end-to-end tests for
CustomResourcePublishOpenAPIand modifying pod resize tests to prevent the creation of extra unused namespaces.
- Network and System Configuration: This topic includes pull requests related to network and system configuration improvements. These involve checking for global IPv6 enablement in kube-proxy and cleaning up service chain checks in nftables.
3.3 Pull Request Discussion Insights
This section will analyze the tone and sentiment of discussions within this project's open and closed pull requests that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.
Based on our analysis, there are no instances of toxic discussions in the project's open or closed pull requests from the past week.
IV. Contributors
4.1 Contributors
Active Contributors:
We consider an active contributor in this project to be any contributor who has made at least 1 commit, opened at least 1 issue, created at least 1 pull request, or made more than 2 comments in the last month.
If there are more than 10 active contributors, the list is truncated to the top 10 based on contribution metrics for better clarity.
| Contributor | Commits | Pull Requests | Issues | Comments |
|---|---|---|---|---|
| aojea | 15 | 6 | 7 | 81 |
| pohly | 24 | 10 | 9 | 59 |
| BenTheElder | 19 | 2 | 5 | 74 |
| roseteromeo56 | 61 | 0 | 0 | 0 |
| carlory | 22 | 14 | 1 | 19 |
| danwinship | 4 | 1 | 0 | 42 |
| bart0sh | 1 | 1 | 1 | 41 |
| gnufied | 17 | 5 | 1 | 12 |
| jpbetz | 12 | 6 | 1 | 11 |
| liggitt | 2 | 1 | 0 | 26 |