Weekly GitHub Report for Kubernetes: June 09, 2025 - June 16, 2025 (12:05:49)
Weekly GitHub Report for Kubernetes
Thank you for subscribing to our weekly newsletter! Each week, we deliver a comprehensive summary of your GitHub project's latest activity right to your inbox, including an overview of your project's issues, pull requests, contributors, and commit activity.
Table of Contents
I. News
1.1 Recent Version Releases:
The current version of this repository is v1.32.3
1.2 Version Information:
The version release information, created on March 11, 2025, highlights key updates in Kubernetes version 1.32, with detailed changes and additional binary downloads available in the linked changelog. Notable trends or highlights from this release can be found in the Kubernetes announcement forum and the comprehensive changelog documentation.
II. Issues
2.1 Top 5 Active Issues:
We consider active issues to be issues that that have been commented on most frequently within the last week. Bot comments are omitted.
-
Make native scheduling workload-aware: This issue is about initiating a discussion to transition from pod-based to workload-based scheduling in Kubernetes, aiming to improve scheduling capabilities and ensure workload integrity throughout their lifecycle. The issue highlights the need for a collaborative effort to frame the problem, set goals, and explore solutions, with a reference to an external document for further insights and contributions.
- The comments primarily consist of users tagging relevant groups and individuals to draw attention to the issue, with one user expressing difficulty in managing multiple related proposals and suggesting the inclusion of references in the document for clarity.
- Number of comments this week: 10
-
🐘 Allow linker to perform dead code elimination for kube-apiserver etc: This issue addresses the need to enable the Go linker to perform dead code elimination for the
kube-apiserverin the Kubernetes project, which involves removing unused functions to reduce the binary size. The process requires buildingkube-apiserverwith specific flags and using a tool to identify and fix problematic code locations, ensuring that future updates do not introduce new issues.- The comments discuss the organization of code and reference related pull requests and issues. They highlight the need for a new tag for a dependency, confirm disk space savings from the effort, and provide additional details on how the linker handles dead code elimination. There is also a mention of challenges with certain functions and a discussion about potential fixes, including forking a library. The comments conclude with references to related issues and acceptance of the triage.
- Number of comments this week: 8
-
Pod deletion timestamp is resetted after preStop hook is executed: This issue describes a potential bug in the Kubernetes system where the deletion timestamp of a pod is reset after the execution of a preStop hook, causing confusion as the timestamp appears to fluctuate unexpectedly. The user provides a detailed timeline and YAML configuration to illustrate the problem, noting that the kubelet seems to send a new DELETE request after the preStop hook, which might be causing the timestamp to revert to an earlier time.
- The comments discuss the similarity of this issue to a previous one, with a suggestion that a related PR might not have fully resolved the problem. The deletionTimestamp behavior is explained, noting that it can move back in time if the grace period is shortened, which is expected behavior. The final comment suggests that the issue might be working as intended, given the current logic.
- Number of comments this week: 6
-
[Flaking Test] [sig-node] Sysctls [LinuxOnly] [NodeConformance] should support sysctls: This issue pertains to a flaking test in the Kubernetes project, specifically within the sig-node group, where the Sysctls [LinuxOnly] [NodeConformance] tests are failing due to a connection reset error. The problem has been occurring since early June 2025, affecting multiple jobs and is linked to a potential crash in the cri-o service, which is causing the test containers to fail to start.
- The comments discuss the similarity of this issue to a previous one involving a cri-o crash, with a detailed log showing a SIGSEGV error leading to a core dump. It is clarified that this issue is not a blocker for the upcoming release, and an update in CRI-O has been merged to provide more logs for better diagnosis.
- Number of comments this week: 5
-
Tune APF list work estimator for large responses: This issue addresses the need to enhance the API Priority and Fairness (APF) mechanism in Kubernetes to better estimate the memory cost of large LIST responses, which currently can lead to memory exhaustion and reduced server stability. The proposal suggests incorporating response size as a primary factor in the APF LIST work estimator, replacing the object count estimator, and adjusting seat allocations to more accurately reflect resource usage.
- The comments discuss the potential benefits and considerations of the proposed changes, including the possibility of serving all requests from cache, the need for minimal invasiveness, and the importance of thorough review. There is also a mention of related proposals and the necessity of efficiently obtaining the total size of resources from Kubernetes.
- Number of comments this week: 5
2.2 Top 5 Stale Issues:
We consider stale issues to be issues that has had no activity within the last 30 days. The team should work together to get these issues resolved and closed as soon as possible.
- apimachinery resource.Quantity primitive values should be public for recursive hashing: This issue addresses the need for the primitive values within the
apimachineryresource.Quantitystruct to be made public to facilitate recursive hashing by controllers, which is currently hindered by these values being private. The lack of public access to these values complicates the detection of changes in Custom Resource Definitions (CRDs) for projects likekubernetes-sigs/karpenter, which rely on hash comparisons to identify specification drifts, impacting resource allocation and necessitating inefficient workarounds. - APF borrowing by exempt does not match KEP: This issue highlights a discrepancy between the Kubernetes Enhancement Proposal (KEP) and its implementation regarding how the exempt priority level borrows from other levels in the Kubernetes API Priority and Fairness (APF) system. Specifically, the KEP outlines a distinct formula for calculating the minimum concurrency limit for exempt levels, which is not reflected in the current implementation, leading to a default configuration where the exempt priority level is assigned a minimum concurrency limit of zero.
- Zone-aware down scaling behavior: This issue addresses a problem with zone imbalance during the scale-in process of Horizontal Pod Autoscaler (HPA) workloads in a Kubernetes deployment, where the expected topology spread constraint of
maxSkew: 1is not being respected, resulting in an uneven distribution of pods across zones. The imbalance leads to a scenario where one zone ends up with significantly fewer pods, causing high CPU usage on the lone pod in that zone, while the other zones have a more balanced distribution of pods. - apimachinery's unstructured converter panics if the destination struct contains private fields: This issue describes a problem with the
DefaultUnstructuredConverterin the Kubernetesapimachinerypackage, where it panics when attempting to convert a destination struct that contains private fields. The panic occurs because the converter tries to set values on unexported fields, which is not allowed in Go, and the user expects the converter to ignore these private fields instead of causing a panic. - Jsonpath impl does not support left match regex: This issue highlights a request for the addition of support for the
=~operator in jsonpath filter expressions within a GitHub project, specifically to enable matching using Golang regular expressions. The feature is needed to simplify the process of locating desired resources among many by allowing users to match items based on patterns, such as those whose descriptions start with a specific string, in a case-insensitive manner.
2.3 Open Issues
This section lists, groups, and then summarizes issues that were created within the last week in the repository.
Issues Opened This Week: 37
Summarized Issues:
- Pod Lifecycle Issues: This topic covers various issues related to pod lifecycle management in Kubernetes, including problems with pod deletion and recreation, synchronization errors, and unexpected behavior during pod termination. These issues can lead to persistent old pod objects, kubelet panics, and fluctuating deletion timestamps, affecting the stability and reliability of Kubernetes clusters.
- Flaky Tests: Several issues highlight flaky tests within the Kubernetes project, affecting various components such as container logs, sysctls support, and scheduling performance. These tests fail intermittently due to factors like segmentation faults, connection errors, and unexpected test results, complicating the development and release processes.
- Resource Management and Scheduling: Issues in this category focus on resource management and scheduling improvements in Kubernetes, including transitioning to workload-based scheduling, enhancing API Priority and Fairness, and supporting CPU and memory overcommitment. These changes aim to optimize resource allocation and improve workload integrity.
- Configuration and API Enhancements: This topic includes discussions on streamlining Kubernetes configurations and APIs, such as simplifying
ConfigMapKeyRefandSecretKeyReffields, transitioning API versions, and improving the kubelet's configuration. These enhancements aim to reduce complexity and improve maintainability.
- Test and Tool Improvements: Several issues address the need for improvements in testing tools and methodologies, including refactoring tests to use mock drivers, adding memory tracking capabilities, and aligning with new toolchain standards. These efforts aim to enhance test reliability and reduce maintenance overhead.
- Security and Access Control: Issues in this category focus on security and access control challenges in Kubernetes, such as RBAC misconfigurations, admission webhook failures, and security-related bugs. Addressing these issues is crucial for maintaining secure and compliant Kubernetes environments.
- Metrics and Monitoring: This topic covers issues related to metrics and monitoring in Kubernetes, including inaccurate metrics reporting, the need for new metrics, and discrepancies in data retrieval. These issues impact the ability to effectively monitor and manage Kubernetes clusters.
- Dependency and Code Management: Several issues highlight the need for better dependency and code management in Kubernetes, such as addressing circular dependencies, updating library references, and enabling dead code elimination. These efforts aim to streamline the codebase and improve build efficiency.
2.4 Closed Issues
This section lists, groups, and then summarizes issues that were closed within the last week in the repository. This section also links the associated pull requests if applicable.
Issues Closed This Week: 2
Summarized Issues:
- Kubelet's CPU Manager NUMANodeAffinity Calculation: The Kubelet's CPU Manager in Kubernetes has a problem with inaccurately calculating the narrowest matching NUMANodeAffinity. This issue arises because it does not consider reserved CPUs, leading to a mismatch in the bitmask used for pod scheduling under the restricted policy, which prevents the creation of a pod requesting 112 CPUs and 8 GPUs on a machine with 128 cores and 8 NUMA nodes.
- Kubernetes ResourceClaim API Naming Convention: There is a concern regarding the naming convention of the
deviceClassNamefields in the Kubernetes ResourceClaim API. The issue questions whether these fields violate the API conventions documentation by not using the{field}Refformat for object referencing field names, and seeks clarification on whether this deviation is intentional or a documentation oversight.
2.5 Issue Discussion Insights
This section will analyze the tone and sentiment of discussions within this project's open and closed issues that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.
Based on our analysis, there are no instances of toxic discussions in the project's open or closed issues from the past week.
III. Pull Requests
3.1 Open Pull Requests
This section provides a summary of pull requests that were opened in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.
Pull Requests Opened This Week: 49
Key Open Pull Requests
1. DRA: automated upgrade/downgrade testing: This pull request introduces automated testing for the upgrade and downgrade paths of the core Dynamic Resource Allocation (DRA) feature in Kubernetes by adding helper packages to manage a kind cluster and execute end-to-end (E2E) tests, thereby facilitating the promotion of DRA to General Availability (GA) and addressing the challenge of manually verifying and replicating test procedures.
- URL: pull/132295
- Merged: No
2. Add an interface that all apply configurations implement: This pull request introduces a runtime.ApplyConfiguration interface to standardize the implementation of apply configurations across the codebase, updates the applyconfiguration-gen tool and the unstructured.Unstructured type to implement this interface, and modifies the IsApplyConfiguration() method to ensure compile-time checks, thereby addressing issues related to server-side apply (SSA) usage in controller-runtime and fixing a related issue in the Kubernetes project.
- URL: pull/132194
- Merged: No
3. emeritus spiffxp and backfill OWNERS: This pull request involves a cleanup of the Kubernetes project by removing a long-inactive approver, spiffxp, and backfilling the list of current reviewers and approvers with active contributors, including adding bentheelder to conformance-related test reviewers and other directories, to ensure continued oversight and maintenance of the project's testing and conformance areas.
- URL: pull/132220
- Merged: No
Other Open Pull Requests
- Kubelet Contextual Logging Migration: This pull request focuses on migrating the kubelet component of Kubernetes to use contextual logging. It involves refactoring context handling and updating logging calls within the kubelet certificate package to enhance logging clarity and maintainability.
- Dynamic Resource Allocation Enhancements: A new helper function,
StandardDeviceAttributes(opts ...), is introduced for DRA drivers in Kubernetes to facilitate standardized device attributes. This includes the addition of two new packages for generating standardized attributes and handling PCI addresses.
- Kubernetes Controller Health Checks: This pull request introduces a custom health check function for Kubernetes controllers by implementing the
HealthCheckableinterface. It allows controllers to report readiness based on informer cache synchronization status, enhancing visibility into their request processing ability.
- Scheduler Framework Refactoring: Types and structs are moved from the main Kubernetes repository to the staging repository "k8s.io/kube-scheduler". This facilitates the import of scheduler framework interfaces without requiring the entire Kubernetes repository.
- Windows Server 2025 Busybox Images: This pull request introduces Windows Server 2025 (ltsc2025) busybox images into the BASEIMAGE files. It refactors the BASEIMAGE to utilize a version placeholder for dynamically fetching the busybox version, enhancing flexibility and maintainability.
- Bug Fixes in Kubernetes: Several pull requests address bugs in the Kubernetes project, including ensuring paginated list requests are served from the cache, correcting error messages in
kubectl logs, and adding validation forvolumeClaimTemplatesin StatefulSet. These fixes enhance the robustness and reliability of the system.
- Codebase Cleanup and Optimization: Unnecessary code related to the Alpha feature "MountContainers" is removed, and optimizations are introduced to methods like
ListAllandListAllByNamespace. These changes improve code efficiency and maintainability.
- Testing and Validation Enhancements: New tests are introduced to validate LIST requests sent to etcd and to test the generation of mirror pods. These enhancements ensure the reliability of operations and address previous fallback issues.
- Security and Conformance Improvements: The OrderedNamespaceDeletion test is promoted to Conformance, addressing a critical security behavior change. This ensures users can rely on this feature, with links to specific issues and CVEs for enhanced security tracking.
- Diff Function Implementation: A new implementation of the
Difffunction is introduced as a drop-in replacement forcmp.Diff. This aims to eliminate the use ofgo-cmp/cmpfrom non-test code and addresses issues related to dead code elimination.
- Documentation and Link Updates: Outdated links in the VolumeSource godocs are updated, removing links to deprecated source types and updating the iSCSI documentation link. This effort resolves issue #79 in the Kubernetes API repository.
- Metrics and Data Integrity: A new status code,
message-authentication-failed, is added to thetransformation_operations_totalmetrics. This helps identify instances where an object fails decryption due to AEAD open failures, enhancing the detection of potential data integrity issues.
- DeltaFIFO Batch Processing: A feature is introduced to enhance the DeltaFIFO component by enabling batch processing of items. This improves write throughput and overall controller performance, addressing read and write lock contention in large-scale busy clusters.
- SELinux Relabeling and Shellcheck: SELinux relabeling on the source directory bind mount is disabled for the entire container. This change streamlines the shellcheck process within the Kubernetes project.
- RealFIFO Configuration Options: A
RealFIFOOptionsstruct and aNewRealFIFOWithOptions()constructor are introduced to facilitate the addition of new configuration options. This change aims to prevent breaking changes while incorporating features like a logger or a FIFOMetricsProvider.
- cAdvisor Dependency Update: The cAdvisor dependency is updated to version 0.49.2 for the Kubernetes 1.31 release. This update addresses issue #131889 and opts for a smaller patch version due to significant changes in the master branch's cAdvisor version 0.51.0.
- Environment Variable Validation: The RelaxedEnvironmentVariableValidation feature gate is promoted to General Availability (GA). Associated tests are upgraded to Conformance tests, ensuring reliable passing as evidenced by the provided test grid links.
- Logging Optimization: An automated cherry-pick of a previous commit aims to reduce excessive logging when calculating sandbox resources. This change is intended for the release-1.33 branch of the Kubernetes project.
3.2 Closed Pull Requests
This section provides a summary of pull requests that were closed in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.
Pull Requests Closed This Week: 46
Key Closed Pull Requests
1. Match expression: This pull request aims to enhance the Kubernetes project by implementing a "match expression" feature, which involves updating the API definition, adding initialization code, and creating various unit tests to evaluate expression evaluation and device selection, as part of addressing the issue linked to the Kubernetes Enhancement Proposal (KEP) for constraints with CEL.
- URL: pull/132196
- Merged: No
- Associated Commits: 528ae, 1b92a, 11af9, 869d2, 57d68, 104a2, 9170b, 1c43a, f95b6, 6580c, 7338e, ea380, efd6d, 79c8f, ae48b, f51ca, bb16d, 43ad2, 605dd
2. feat(validation-gen): add default validation ratcheting support: This pull request introduces default validation ratcheting support in the validation-gen code generator for Kubernetes, enhancing the generated validation code by adding equivalence checks before emitting validators and including ratcheting test cases, as part of the Declarative Validation ratcheting initiative.
- URL: pull/132236
- Merged: Yes
3. typed client: remove support for API streaming : This pull request removes support for API streaming from the List() method of the typed client in the Kubernetes project, as part of a cleanup effort, and includes several commits such as removing the watchlist from client-go and client-gen, updating internal modules, and testing to confirm the non-streaming behavior of the List method.
- URL: pull/132257
- Merged: Yes
Other Closed Pull Requests
- Validation Test Helpers Enhancement: This pull request enhances the validation test helpers for the validation-gen tool by introducing a new
ExpectMatchesmethod for flexible, matcher-based assertions. It also removes outdatedExpectmethods and improves error formatting for the new matcher system, as part of a cleanup effort related to Kubernetes Enhancement Proposal 5073.
- API Streaming Removal: This pull request removes support for API streaming from the
List()method of the dynamic client in the Kubernetes project. It includes updates to the vendor scripts and tests to confirm the removal of streaming functionality.
- Documentation Updates: This pull request involves updates to the Kubernetes project documentation, specifically modifying the README.md and CHANGELOG-1.10.md files. However, it was not merged into the main codebase.
- Unit Test Modifications: This pull request addresses a cleanup task by modifying unit tests to accommodate changes introduced in the latest version of the golang.org/x/oauth2 library (v0.30.0). It ensures compatibility and stability in the project's test suite without updating the library version itself.
- Binary Size Reduction: This pull request addresses a bug by replacing the use of
html/templatein theapiserverwith direct string manipulation. This change prevents the disabling of dead code elimination, resulting in a significant reduction in binary size from 155MiB to 117MiB.
- Monitoring Role Update: This pull request updates the built-in
system:monitoringrole in Kubernetes to grant it permission to access kubelet metrics endpoints. It addresses issue #127990 and ensures that monitoring systems can retrieve necessary metrics data.
- LoadBalancer Test Improvement: This pull request aims to improve the LoadBalancer test by reusing a fixed number of TCP connections to prevent ephemeral port exhaustion. It also creates a pool of 100 clients to test load balancer behavior and reduce flakiness due to network issues.
- Logging Bug Fix: This pull request addresses a bug in the Kubernetes project by fixing a logging issue in the
pkg/kubelet/preemption/preemption.gofile. It ensures that theinsufficientResourcesarray is printed in a human-readable format, improving debuggability and observability.
- Storage Interface Tracing Enhancement: This pull request enhances the tracing capabilities of the Kubernetes storage interface by adding context to the
Count()method. It allows for better association of etcd traces with those from the apiserver, facilitating improved investigation of Etcd API usage.
- Golang Build Update: This pull request involves building Kubernetes using the latest commit of Golang by running "gimme master" to utilize newly built binaries. It adjusts GOROOT_BOOTSTRAP to use the existing Golang on disk for building the new commit, as part of a cleanup effort to avoid unnecessary full
kube-crossimage builds.
- Etcd Image Build: This pull request involves building the etcd image version 3.6.1 as part of a cleanup effort. It references the etcd release v3.6.1 and has been successfully merged into the Kubernetes project.
- Testing Manifest Removal: This pull request involves the removal of an unused and non-functional testing manifest from the Kubernetes project. It was merged to clean up the codebase without introducing any user-facing changes.
- LoadBalancer Test Rate Limiting: This pull request addresses the excessive reconnection attempts in the LoadBalancer rolling update test by rate-limiting the attempts. It reduces the rate from approximately 12,500 times per second to a more manageable 100 times per second, improving test reliability.
- Cobra Library Update: This pull request updates the Kubernetes project to use the latest version 1.9.1 of the
github.com/spf13/cobralibrary. It follows a previous discussion and incorporates changes from a related pull request in thecobrarepository.
- Dead Code Elimination: This pull request introduces the
grpcnotracetag for production (non-debug) builds in the Kubernetes project to enable dead code elimination. It reduces binary sizes by 10-15% when tracing is not needed, while allowing golang diagnostics only in debug builds.
- CRI-O Runtime Bug Fix: This pull request addresses a bug by fixing the construction of the cgroup CPU path in the CRI-O runtime. It is related to issue #131769 in the Kubernetes project.
- Codebase Comparison Update: This pull request aims to switch from using
cmp.Difftoreflect.DeepEqualin the Kubernetes codebase. It ensures dead code elimination and addresses issues identified in related discussions and issues.
- Go Version and Dependency Updates: These pull requests update the Kubernetes project by bumping dependencies, images, and versions to use Go 1.24.4 and distroless iptables. They address issue 4024 and ensure that Kubernetes is now built with the latest Go version.
- CI Job Fix for IPv6: This pull request addresses the issue of failing CI jobs for kind IPv6 by adding a mechanism to skip the nfacct test. It prevents the test from running in IPv6 environments, as it is intended only for IPv4 clusters.
- InterPodAffinity Plugin Optimization: This pull request optimizes the performance of the label selector match in the InterPodAffinity plugin. It reduces the number of map accesses, significantly decreasing latency, especially in large-scale clusters with many pods using PodAffinity.
- Alpha Metrics Removal: This pull request involves the removal of deprecated alpha metrics related to the encryption config controller in the kube-apiserver. It consolidates them into a single metric as part of a cleanup effort.
3.3 Pull Request Discussion Insights
This section will analyze the tone and sentiment of discussions within this project's open and closed pull requests that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.
Based on our analysis, there are no instances of toxic discussions in the project's open or closed pull requests from the past week.
IV. Contributors
4.1 Contributors
Active Contributors:
We consider an active contributor in this project to be any contributor who has made at least 1 commit, opened at least 1 issue, created at least 1 pull request, or made more than 2 comments in the last month.
If there are more than 10 active contributors, the list is truncated to the top 10 based on contribution metrics for better clarity.
| Contributor | Commits | Pull Requests | Issues | Comments |
|---|---|---|---|---|
| dims | 16 | 11 | 6 | 43 |
| pohly | 21 | 7 | 8 | 23 |
| liggitt | 0 | 0 | 0 | 56 |
| BenTheElder | 6 | 3 | 1 | 39 |
| serathius | 6 | 4 | 2 | 16 |
| ania-borowiec | 12 | 3 | 0 | 7 |
| cpanato | 11 | 10 | 0 | 0 |
| aojea | 2 | 2 | 3 | 14 |
| mkumatag | 1 | 0 | 3 | 15 |
| sreeram-venkitesh | 11 | 1 | 0 | 5 |