Weekly GitHub Report for Kubernetes: August 04, 2025 - August 11, 2025 (22:43:03)

Weekly GitHub Report for Kubernetes

Thank you for subscribing to our weekly newsletter! Each week, we deliver a comprehensive summary of your GitHub project's latest activity right to your inbox, including an overview of your project's issues, pull requests, contributors, and commit activity.

---

Table of Contents

• I. News
  • 1.1. Recent Version Releases
  • 1.2. Other Noteworthy Updates
• II. Issues
  • 2.1. Top 5 Active Issues
  • 2.2. Top 5 Stale Issues
  • 2.3. Open Issues
  • 2.4. Closed Issues
  • 2.5. Issue Discussion Insights

• III. Pull Requests
  • 3.1. Open Pull Requests
  • 3.2. Closed Pull Requests
  • 3.3. Pull Request Discussion Insights

• IV. Contributors
  • 4.1. Contributors

I. News

1.1 Recent Version Releases:

The current version of this repository is v1.32.3

1.2 Version Information:

The Kubernetes 1.32 release, announced on March 11, 2025, introduces several key updates and improvements detailed in the official CHANGELOG, with additional binary downloads available. This version continues the trend of enhancing cluster management and feature stability, as highlighted in the release notes.

II. Issues

2.1 Top 5 Active Issues:

We consider active issues to be issues that that have been commented on most frequently within the last week. Bot comments are omitted.

1. Improving handling of Deprecated and Hidden metrics: This issue addresses confusion around the meaning and handling of the DeprecatedVersion field for metrics in Kubernetes, particularly how deprecated metrics should continue to function for a defined grace period before being removed. It highlights a recent problem where a stable metric marked as deprecated caused a panic due to premature removal, and seeks to clarify the correct lifecycle and implementation of deprecation and hiding of metrics according to Kubernetes policy.

   • The comments discuss triaging and assigning the issue, emphasize the urgency due to release-blocking concerns, and outline a multi-step plan to fix deprecated and hidden metric handling, including code changes and policy updates. Contributors converge on clarifying that DeprecatedVersion marks the announcement of deprecation, with actual removal delayed per stability level timelines, and agree on handling the apiserver_storage_objects metric by announcing deprecation in 1.34 while continuing to serve it until later releases, with some discussion on version backfilling to maintain consistent timelines.
   • Number of comments this week: 13

2. scheduler memory increases after uninstalled pods.: This issue reports that the Kubernetes scheduler’s memory usage increases after pods are uninstalled, contrary to expectations that memory should be released. The reporter provides detailed heap profiles and configuration data, suspects the memory rise is related to pod watch handling, and a proposed fix involves properly clearing slice references in the scheduler’s pod removal code to allow garbage collection.

   • The discussion involved requests for raw pprof data and configuration details, analysis showing a subtle but real memory increase, and identification of a likely cause in the scheduler’s slice element removal logic. Contributors agreed the issue stems from lingering references preventing GC, proposed a fix clearing the slice element, and planned unit tests and benchmarks to validate the solution and confirm memory improvements.
   • Number of comments this week: 12

3. The pod is in the UnexpectedAdmissionError state.: This issue describes a problem where pods that depend on device plugins enter an UnexpectedAdmissionError state after the kubelet process restarts, because the device plugin may not have registered in time, causing pods to be rejected and containers to be killed. The reporter expects that containers should not exit due to kubelet restarts and seeks clarification on whether a race condition still exists in the device manager’s handling of kubelet restarts despite fixes introduced in Kubernetes 1.28.

   • The comments acknowledge the issue and reference prior fixes implemented in version 1.28 intended to handle kubelet restarts and prevent this error, including existing e2e tests; however, the reporter questions whether a race condition remains due to the timing of the sourcesReady.AllReady() method returning true, suggesting that the current logic may prematurely assume readiness and cause pods to be rejected, prompting further investigation into the device manager’s sync flow.
   • Number of comments this week: 8

4. Update json-patch to v5.9.10 / v4.0.13 or later, in-order to remove pkg/errors as an indirect dependency: This issue requests updating the json-patch dependency to version 5.9.10 or later to remove the indirect dependency on the archived github.com/pkg/errors package, which has been replaced by the standard errors package in the latest json-patch release. However, due to backward compatibility issues with version 5, the maintainers decided to instead update to a newer v4 release (v4.13.0) that drops the pkg/errors dependency, and this change has been integrated into upcoming Kubernetes releases.

   • The discussion initially considered upgrading to json-patch v5 but was closed due to incompatibility with v4 behavior; subsequently, a newer v4 release that removes the dependency was identified and merged, with testing confirming successful integration and the issue being reopened and actively worked on.
   • Number of comments this week: 6

5. HPA - optimize calculatePodRequestsFromContainers func specific container lookups: This issue addresses an inefficiency in the calculatePodRequestsFromContainers function, which currently iterates through all containers in a pod even when a specific container name is provided. The proposed optimization is to exit the loop early once the matching container is found and to handle the case where the specified container does not exist by returning an appropriate error.

   • The discussion includes agreement on the early exit optimization and a question about error handling for non-existent containers, which is resolved by deciding to return an error consistent with existing behavior. A pull request is submitted implementing the early exit, proper error handling, and maintaining backward compatibility, aiming to improve performance for pods with many containers.
   • Number of comments this week: 5

2.2 Top 5 Stale Issues:

We consider stale issues to be issues that has had no activity within the last 30 days. The team should work together to get these issues resolved and closed as soon as possible.

1. apimachinery's unstructured converter panics if the destination struct contains private fields: This issue describes a panic occurring in the apimachinery's DefaultUnstructuredConverter when it attempts to convert an unstructured object into a destination struct that contains private (non-exported) fields. The reporter highlights that the converter does not currently ignore these private fields, leading to a runtime panic due to reflection restrictions, and suggests that the converter should be fixed to skip private fields to prevent such panics.
2. Integration tests for kubelet image credential provider: This issue discusses the potential addition of integration tests specifically for the kubelet image credential provider, similar to the existing tests for client-go credential plugins. It suggests that since there are already integration tests for pod certificate functionality, implementing similar tests for the kubelet credential plugins would be a logical and beneficial extension.
3. conversion-gen generates code that leads to panics when fields are accessed after conversion: This issue describes a bug in the conversion-gen tool where it generates incorrect conversion code for structs that have changed field types between API versions, specifically causing panics when accessing fields like ExclusiveMaximum after conversion. The problem arises because the generated code uses unsafe pointer conversions instead of properly calling the corresponding conversion functions, leading to runtime errors when the converted fields are accessed.
4. Failure cluster [ff7a6495...] TestProgressNotify fails when etcd in k/k upgraded to 3.6.2: This issue describes a failure in the TestProgressNotify test that occurs when the etcd component in the Kubernetes project is upgraded to version 3.6.2. The test times out after 30 seconds waiting on a result channel, with multiple errors indicating that the embedded etcd server fails to set up serving due to closed network connections and server shutdowns.
5. Failure cluster [10593569...] ci-kubernetes-e2e-gce-node-containerd* failing during e2e-up.sh: This issue reports a failure cluster related to the ci-kubernetes-e2e-gce-node-containerd* tests consistently failing during the execution of the e2e-up.sh script, resulting in an exit status 2 error. The problem has persisted for over 30 days with multiple recent failures logged, indicating a recurring issue in the end-to-end testing pipeline for Kubernetes on GCE nodes using containerd.

2.3 Open Issues

This section lists, groups, and then summarizes issues that were created within the last week in the repository.

Issues Opened This Week: 24

Summarized Issues:

• Performance and Optimization Issues: Several issues highlight performance degradation and inefficiencies in Kubernetes components, such as the Horizontal Pod Autoscaler's container resource request calculation and the Validating Admission Policy's redundant processing causing high API server latency and CPU usage. These problems suggest opportunities for optimization by improving loop handling and caching immutable data.
• issues/133376, issues/133383

• Pod and Container Lifecycle Problems: Multiple issues describe pod lifecycle disruptions including pods entering error states after kubelet restarts due to race conditions with device plugins, admission controller rejecting pods after kubelet restarts because of stale container IDs, and static pods with high priority being terminated first during node shutdown due to incorrect priority determination. These issues cause service interruptions and improper pod management.
• issues/133382, issues/133451, issues/133442

• Flaky and Failing Tests: Several test failures and flakes are reported across different Kubernetes test suites, including scheduler preemption tests, API Streaming watch notifications, device binding conditions, Prometheus stack setup in scalability tests, PSI stats summary, and Kubelet configuration merging. These intermittent failures cause instability in CI pipelines and require investigation and fixes.
• issues/133395, issues/133396, issues/133435, issues/133446, issues/133456, issues/133457

• Deprecated Features and Compatibility Issues: Problems arise from deprecated Kubernetes features such as the serviceAccount field affecting serviceAccountName clearing, and patching pods using deprecated AppArmor annotations failing due to improper translation to new fields. These issues cause unexpected behavior and patch failures during updates.
• issues/133385, issues/133416

• Security and Compliance Concerns: There is a security concern regarding Kubernetes creating temporary volumes with overly permissive 0777 permissions by default, conflicting with security hardening policies. This raises questions about whether this behavior is intentional or planned to be addressed by maintainers.
• issues/133441

• Documentation and Clarification Requests: Some issues request clearer documentation and explanations, including clarifications on the clusterName field in cluster configurations and error messages related to OpenAPI spec loading failures. These requests aim to improve user understanding and reduce confusion.
• issues/133455, issues/133402

• Feature Enhancements and Strategy Improvements: Requests include adding support for maxSurge and maxUnavailable in DaemonSet RollingUpdate strategies to match Deployment capabilities, and improving error outputs for kubectl logs and kubectl exec to include valid container names when multiple containers exist. These enhancements aim to improve usability and feature parity.
• issues/133378, issues/133443

• Resource and Garbage Collection Issues: Problems include improper deletion of empty directories in PVCs after node reboot and delayed deletion of objects created with non-nil deletion timestamps due to insufficient garbage collection event handling. These issues affect resource cleanup and lifecycle management.
• issues/133399, issues/133388

• Metrics and Monitoring Clarifications: There is confusion around deprecated and hidden metrics handling, specifically the DeprecatedVersion field, which affects proper deprecation of metrics like apiserver_storage_objects without causing runtime failures.
• issues/133429

• SELinux and Volume Access Conflicts: The SELinux warning controller fails to detect conflicts between pods with differing SELinux labels, particularly when one pod has an empty label and another a non-empty label, potentially leading to undetected volume access conflicts.
• issues/133424

• Dependency Updates: A request to update the json-patch dependency to remove reliance on an archived package and replace it with the standard errors package aims to improve dependency hygiene and maintainability.
• issues/133400

2.4 Closed Issues

This section lists, groups, and then summarizes issues that were closed within the last week in the repository. This section also links the associated pull requests if applicable.

Issues Closed This Week: 7

Summarized Issues:

• Resource Allocation and Scheduling Failures: Several issues describe problems with resource allocation and scheduling in Kubernetes, including flaky test failures due to CEL selector errors and pods timing out because no nodes can allocate required resources. These issues cause intermittent test flakes and unschedulable pods, blocking PRs and delaying deployments.
• issues/133223, issues/133384

• Pod Crash and Restart Issues: Multiple reports highlight pods crashing or failing to restart properly, including a Python application entering CrashLoopBackOff immediately after start and pods not restarting due to incorrect liveness probe health reporting. These problems result in failed deployments and delayed container restarts lasting hours.
• issues/133374, issues/133375

• Logging and Test Flakiness: There are issues with excessive and incorrect logging during tests, such as nil errors being logged as unexpected websocket errors, causing log spam. Additionally, a segmentation fault in the apiserver during end-to-end tests is caused by improper handling of deprecated Prometheus metrics, leading to widespread test failures.
• issues/133387, issues/133405

• Community Support and Quality Assurance Initiatives: One issue discusses the creation of a free MCP server aimed at learning from common Kubernetes issues and improving community support and quality assurance through official documentation and community feedback. This initiative seeks to address recurring problems by fostering better collaboration.
• issues/133453

2.5 Issue Discussion Insights

This section will analyze the tone and sentiment of discussions within this project's open and closed issues that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.

Based on our analysis, there are no instances of toxic discussions in the project's open or closed issues from the past week.

---

III. Pull Requests

3.1 Open Pull Requests

This section provides a summary of pull requests that were opened in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.

Pull Requests Opened This Week: 42

Key Open Pull Requests

1. [WIP] Introduce node capabilities reporting and inferencing framework: This pull request introduces an initial draft implementation of a node capabilities reporting and inferencing framework, including feature gates, API fields, common libraries, scheduler and admission plugins, and integration with IP Pod Resources to enhance node capability awareness and scheduling in Kubernetes.

• URL: pull/133389

• Merged: No

• Associated Commits: b08b5, ba362, db6e8, a0ad2, f2f0a, 389b6

2. feat(kubectl get, api): add nextScheduleTime field to CronJobStatus: This pull request adds a nextScheduleTime field to the CronJobStatus API, enabling the CronJob controller to calculate and display the next scheduled execution time in the kubectl get cronjob -o wide output, thereby improving observability and user insight into CronJob scheduling behavior.

• URL: pull/133439

• Merged: No

• Associated Commits: 750ad, 96099, 2480c, 7e59a, 1e13a

3. [Carry 133278] kubelet: Don't ignore idsPerPod config #133278: This pull request fixes a bug in the Kubernetes kubelet by ensuring that the userNamespaces.idsPerPod configuration is properly honored during initialization, addressing the issue where this setting was previously ignored because the configuration was not yet set when the user namespace manager was created, and includes related improvements and tests.

• URL: pull/133373

• Merged: No

• Associated Commits: ad9e7, 71b47, e4e45, 53c35

Other Open Pull Requests

• Test reliability and flakiness improvements: Multiple pull requests focus on enhancing test stability by adding retry mechanisms, splitting tests to reduce timeouts, and normalizing dynamic test outputs. These changes help reduce flaky failures and improve the robustness of Kubernetes testing infrastructure.
  [pull/133392, pull/133398, pull/133421]

• Pod and resource admission/refinement: Several pull requests address pod admission logic and resource handling, including refactoring allocation feasibility checks for vertical scaling, updating nodeports scheduling for restartable initContainers, and fixing pod spec validation tests. These improvements streamline pod lifecycle management and scheduling accuracy.
  [pull/133427, pull/133390, pull/133381]

• CSI and volume handling fixes and tests: Pull requests fix potential null pointer dereferences in CSI volume handling and add end-to-end tests to verify SELinuxMount capability support in CSI drivers. These ensure safer volume operations and validate security-related features in storage drivers.
  [pull/133403, pull/133426]

• Bug fixes in API server and label parsing: Fixes include correcting system field clearing during patch-based resource creation and resolving label selector parsing errors with consecutive commas. These changes improve API correctness and selector parsing robustness.
  [pull/133418, pull/133418]

• Scheduler and memory management improvements: A critical memory leak in the scheduler cache is fixed by clearing PodInfo references, preventing memory bloat in production environments. This enhances scheduler stability and resource efficiency.
  [pull/133411]

• Kubectl command usability enhancements: Updates to kubectl attach reattach session messages now include namespace flags for non-default namespaces, improving user experience and command accuracy.
  [pull/133404]

• Topology Manager and resource hint filtering: Implementation of a filter to exclude preferred=false hints under the restricted policy in the Topology Manager is introduced, with ongoing work to update related unit tests. This refines resource hint handling for better topology-aware scheduling.
  [pull/133379]

• Metrics and monitoring improvements: Updates include adding a wrapper for prom client's DeleteLabelValues method and registering apiserver_resource_objects count in tests, enhancing metrics management and validation.
  [pull/133410, pull/133408]

• Error message and log output fixes: Fixes to error messages for kubectl get --watch with multiple resource types and a typo in log output related to caching improve clarity and troubleshooting capabilities.
  [pull/133380, pull/133414]

• Proxy and network connection enhancements: Adding egress dialer support to proxyClientConfig improves handling of connections to peer apiservers in mixed version proxy environments. Additionally, an e2e_node test verifies reuse of a single gRPC connection with the DRA plugin, ensuring connection efficiency.
  [pull/133407, pull/133401]

• Codebase maintenance and refactoring: A pull request proposes general refactoring to improve the Kubernetes codebase without adding features or fixing specific bugs, supporting long-term maintainability.
  [pull/133420]

• Removal of obsolete methods: The GetLoadBalancerName method and its tests are removed due to obsolescence and cloud provider ambiguity, simplifying the LoadBalancer interface.
  [pull/133423]

• Baseline documentation for LLM integration: A new AGENTS.md file and symbolic links for Gemini and Claude are added to support fine-tuning and integration of large language models with Kubernetes, laying groundwork for enhanced developer assistance.
  [pull/133386]

3.2 Closed Pull Requests

This section provides a summary of pull requests that were closed in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.

Pull Requests Closed This Week: 17

Key Closed Pull Requests

1. Automated cherry pick of #132791: Update pod resize test to accept new cpu.weight conversion.: This pull request is an automated cherry pick of #132791 onto the master branch that updates the pod resize test to accept the new cpu.weight conversion.

• URL: pull/133450

• Merged: No

• Associated Commits: 5158e, 25709, 80779, 8c050, 1cf22, 4edb9, 46693, 34234, 90acb, a8da2, 037ad, 9acd8, 96174, 8d680, 9b21e, 53f95, 6881c, d5730, b052c, cffe7, 55697, 90b73, c6b2c, f3a70, 04fbc, a67cb, 9a0e9, 74d51, a0cc1, 0c6ca, f43c1, 5dd1c, 8c75f, ee6ca, d7eae, 7baaa, b4808, bb2d0, 5b00c, 18639, fe18c, 4dba7, 1f41f, c02cc, 95990, 0ac22, cc49c, 6fd09, e1384, f80bc, f6bf1, d1425, 7ffc2, 188bc, 202ce, 74c7f, 932d5, 6c940, c24db, 10e4c, ced1d, 2b84c, 4e3d5, a4ac1, e010c, 6a64b, 87792, 1a133, 3435b, c3626, d8d2e, 12b46, c8144, 6e0c8, 60a8e, cab38, c1fde, 9016e, 5f304, 78d96, 25cbf, 20484, 9bd36, 41c99, f3228, 8b4d1, c3f78, 39d2e, 4f0b8, 2d5fa, 1e41f, d0b5a, a9960, 84d12, b9436, 3e878, fc716, 4fc67, dd5e2, 4e77d, 96990, 43760, 6fc37, c42ba, e6d81, 13ccb, b79f6, 80b82, 47f54, 8fee3, 3d2b1, 050f9, 1522e, b00ec, 7acdd, b7fd5, 1acd3, 9c264, 36f2a, af243, fbf2b, ba345, 83565, 13836, f959e, c8f22, c916d, d3c75, a624d, aa347, 5d8d2, 750d2, 9a57d, 48fa6, 00030, f4bd4, 24bf6, b59de, c0af2, 18f7f, f04f7, a257b, d0faa, a3adf, 9c646, da947, 814ec, 3a09a, 01aff, b5a17, 55a2f, 3f75c, 973b3, 5dc46, 00ebe, cce99, f240b, 633fd, 8adc0, 76747, 98942, 95c11, 99292, 514da, 2628e, fd586, 9326f, 54889, a3531, 5e720, b5d6f, 3755f, 1d923, 07011, 69dc9, 10466, 9a67e, 1fde2, d0dc6, f8ace, e00e4, 8ef42, c4f22, 40849, f20ad, 89d95, 4cd56, 060a4, ea3aa, 2bd8e, ec4c7, aab1b, 8588c, 886f7, 0e92a, e4762, 63c21, a792b, de829, 6144f, 448ee, b41aa, 18eda, 1fe90, 790dc, 6471c, e5410, 6ea58, 9a90a, 09750, 5b3e8, ee4b2, f900f, 2aa6d, d2be1, 88303, 9e85f, 123a7, 73b1f, 4ca50, 7246e, 612e0, d9288, 4c8c8, cb130, 2242f, 66b29, 05e72, 83613, d4a4a, a57b6, f208b, 036f7, 25a5b, 62bb9, 9b547, 02b9b, ee395, 2cf0e, 00da8, e52b1, 38512, 06807, e9c25, 54a0c, ecc8d, 6a931, 053e2, a95af, f95a5

2. Pluginmanager: unregister plugin on a service socket file removal: This pull request enhances the plugin manager by adding functionality to unregister plugins when their service socket files are removed, improving cleanup and reliability for setups like CSI and DRA that use separate registration and service sockets.

• URL: pull/133308

• Merged: No

• Associated Commits: 55314, 3c185, ba3d5

3. Respect metric deprecation policy for hiding metrics: This pull request updates the component-base's shouldHide function to fully comply with the Kubernetes metric deprecation policy by refining the logic used to determine when metrics should be hidden, addressing issue #133429.

• URL: pull/133433

• Merged: No

• Associated Commits: e570c, 9ad49, bf1f8

Other Closed Pull Requests

• Test Flakiness Fixes and Improvements: Multiple pull requests address flakiness and stability issues in tests by adding missing cache insertions, waiting for informers, reducing pod counts, and improving error logging. These changes ensure more reliable test execution and clearer log outputs during end-to-end testing.
• pull/133321, pull/133397, pull/133422

• Bug Fixes Related to Metrics and Logging: Several pull requests fix bugs involving incorrect logging of nil errors, removal of deprecated metrics, and proper handling of unregistered metrics to prevent panics. These fixes improve stability and correctness in metric handling and error reporting.
• pull/133391, pull/133406, pull/133412

• Temporary Test Disabling and Debugging Efforts: Some pull requests temporarily disable problematic tests or serve as work-in-progress drafts to debug failures in specific environments or restore test signals quickly. These are interim solutions aimed at maintaining CI health while longer-term fixes are developed.
• pull/133353, pull/133393, pull/133417

• Code Readability and Consistency Improvements: One pull request improves code readability by renaming a state file variable to use a consistent suffix and updating related comments. This change aligns naming conventions across managers for better maintainability.
• pull/133337

• Miscellaneous Additions: A pull request adds a simple text file named test.txt to the repository without further context or changes.
• pull/133413

• Release and Dependency Updates: One pull request adds configuration publishing bot rules for Kubernetes release 1.34, including updates for Go version 1.24.5 and related dependencies. This supports the release process and dependency management.
• pull/133447

3.3 Pull Request Discussion Insights

This section will analyze the tone and sentiment of discussions within this project's open and closed pull requests that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.

Based on our analysis, there are no instances of toxic discussions in the project's open or closed pull requests from the past week.

---

IV. Contributors

4.1 Contributors

Active Contributors:

We consider an active contributor in this project to be any contributor who has made at least 1 commit, opened at least 1 issue, created at least 1 pull request, or made more than 2 comments in the last month.

If there are more than 10 active contributors, the list is truncated to the top 10 based on contribution metrics for better clarity.

Contributor	Commits	Pull Requests	Issues	Comments
BenTheElder	6	2	3	99
ylink-lfs	21	7	2	27
pohly	20	2	13	16
liggitt	17	5	0	27
richabanker	7	7	4	25
dims	7	2	5	28
ffromani	13	2	3	17
aojea	4	2	2	26
stlaz	6	2	0	25
SergeyKanzhelev	4	2	0	23

Access Last Week's Newsletter:

• Link