Weekly GitHub Report for Xla: August 17, 2026 - August 24, 2026 (22:01:12)
Weekly GitHub Report for Xla
Thank you for subscribing to our weekly newsletter! Each week, we deliver a comprehensive summary of your GitHub project's latest activity right to your inbox, including an overview of your project's issues, pull requests, contributors, and commit activity.
Table of Contents
I. News
1.1 Recent Version Releases:
No recent version releases were found.
1.2 Version Information:
Please provide the version release information you would like me to analyze and summarize.
II. Issues
2.1 Top 5 Active Issues:
We consider active issues to be issues that that have been commented on most frequently within the last week. Bot comments are omitted.
-
[NVIDIA-GPU] [ERR: RUNTIME] Dynamic-slice fusion hides Mosaic-GPU collective kernels from collective-memory coloring: This issue describes a problem where the
DynamicSliceFusionRewriterV2pass in the XLA compiler fuses Mosaic-GPU collective kernels with slices of other operations, causing the fusion boundary to hide these kernels from collective-memory coloring. This results in buffers being allocated in default memory rather than symmetric memory, leading to NCCL symmetric window registration failures during execution, which breaks multi-process expert-parallel MoE transport workflows.- The comments discuss potential solutions, suggesting that instead of filtering out such custom calls, the collective-memory coloring should be fixed to propagate through fusion boundaries, noting ongoing work related to putting collectives into fusions that will face similar issues if symmetric memory requirements are not handled properly.
- Number of comments this week: 2
-
[ENHANCEMENT] [NVIDIA-GPU] [GPU] Add support for complex32 and bcomplex32: This issue requests the addition of support in XLA for two new complex number types, complex32 and bcomplex32, which correspond to complex numbers with float16 and bfloat16 components respectively. The motivation is to enable compatibility with NVIDIA's FFT libraries that support 16-bit FFTs, and the proposal includes detailed plans for type definitions, storage representation, serialization, and staged implementation steps.
- The comments include a request for updates on the status of these types in JAX and a detailed proposal from a contributor outlining the naming, storage format, serialization, and phased implementation approach, along with questions seeking confirmation on key design decisions before proceeding with a core PR.
- Number of comments this week: 1
-
[ERR: RUNTIME] XLA jit_compile=True crashes with segmentation fault when executing tf.image.rot90 followed by tf.experimental.numpy.cumsum: This issue describes a segmentation fault crash occurring when using XLA compilation with jit_compile=True on a TensorFlow graph that applies tf.image.rot90 followed by tf.experimental.numpy.cumsum, despite the same operations running successfully in eager mode. The problem manifests after successful compilation, where the executable crashes immediately with a segmentation fault, indicating a potential bug in the XLA compiler or runtime handling of these combined operations.
- The single comment suggests that this issue likely shares the same root cause as another issue and references a pull request that may address the problem.
- Number of comments this week: 1
-
[QUESTION] [AMD-GPU] ROCm BLASLt algorithm ordinal can resolve to different solutions across processes: This issue addresses the problem of non-deterministic ordering of ROCm BLASLt matmul algorithm heuristics across different processes, which causes the persisted algorithm ordinal to select different native solutions and produce inconsistent outputs. The root cause lies in the heuristic candidate enumeration within TensileLite, where tied heuristic scores and process-dependent memory layouts lead to varying orderings, and the current approach of storing algorithm selection as a vector index rather than a durable identifier results in mismatches during serialization and reconstruction.
- The comment provides an in-depth AI analysis explaining that the heuristic order varies due to tied scores and address space layout randomization affecting internal hash map iteration order, causing instability in algorithm selection indices; it suggests fixes including storing durable algorithm IDs instead of ordinals, implementing deterministic tie-breaking in XLA’s sorting, and an upstream fix in hipBLASLt’s heuristic function.
- Number of comments this week: 1
Since there were fewer than 5 open issues, all of the open issues have been listed above.
2.2 Top 5 Stale Issues:
We consider stale issues to be issues that has had no activity within the last 30 days. The team should work together to get these issues resolved and closed as soon as possible.
As of our latest update, there are no stale issues for the project this week.
2.3 Open Issues
This section lists, groups, and then summarizes issues that were created within the last week in the repository.
Issues Opened This Week: 3
Summarized Issues:
- Kernel Fusion and Memory Allocation Issues: The DynamicSliceFusionRewriterV2 pass fuses Mosaic-GPU collective kernels with slices of other operations' outputs, which hides these kernels from collective-memory coloring. This causes their buffers to be allocated in default memory, leading to NCCL symmetric window registration failures during execution.
- issues/47461
- Non-Deterministic Algorithm Selection in ROCm BLASLt Backend: The ROCm BLASLt backend's heuristic algorithm selection can produce different algorithm orderings across processes due to non-deterministic tie-breaking in candidate sorting. This inconsistency causes the persisted algorithm ordinal to reference different native solutions, resulting in inconsistent outputs when serialized and reconstructed in separate runs.
- issues/47659
- HLO Transform and Sharding Validation Errors: Registering a pre-scheduler HLO transform that returns the input unchanged causes XLA's post-transform HloVerifier to incorrectly reject a valid multi-slice sharding on a two-slice TPU compile topology. The rejection is due to a mismatch in device count validation between localized per-slice assignments and global multi-slice shardings.
- issues/47778
2.4 Closed Issues
This section lists, groups, and then summarizes issues that were closed within the last week in the repository. This section also links the associated pull requests if applicable.
Issues Closed This Week: 10
Summarized Issues:
- Compilation and Execution Failures with TensorFlow Graphs: Several issues describe crashes or failures during XLA compilation or execution when using TensorFlow graphs with jit_compile=True or specific operations like tf.where, tf.linalg.svd, and embedding layers. These problems include fatal internal assertion failures, segmentation faults due to dynamic dimension handling, and invalid argument errors that do not occur in eager execution mode.
- issues/44941, issues/44944, issues/45972
- Logging and Warning Management: One issue highlights excessive log bloat caused by repeated GSPMD sharding propagation warnings without suppression, suggesting the warning be modified to log only once. Another issue reports misleading error logs from hwloc_set_cpubind() failures on SLURM clusters due to cgroup CPU restrictions, recommending downgrading log levels or silent handling to reduce confusion.
- issues/38574, issues/45484
- LLVM Code Generation and Verification Failures: An issue describes a fatal crash during CPU backend compilation caused by emitted LLVM IR failing the llvm::verifyModule() check when compiling fusion kernels involving bfloat16 dtype conversions. This failure prevents graceful error handling and causes compilation to abort.
- issues/46954
- Multi-Process GPU Deployment Crashes: A crash occurs in multi-process GPU setups when enabling the device-initiated ragged-all-to-all kernel in JAX, triggered by a RET_CHECK failure due to peer access not being enabled. This is caused by a mismatch between global clique size and local process participants, preventing kernel execution despite correct single-process behavior.
- issues/47146
- StableHLO Module Execution and Pass Divergence Issues: Running the HLO pass
convert-movercauses divergent outputs across 16 StableHLO modules on different XLA backends, with divergence disappearing when the pass is disabled. Additionally, a segmentation fault and unhandled exception occur on CPU when running a specific StableHLO module withrun_hlo_module, reproducing across platforms without diagnostic output. - issues/47302, issues/47369
- Mosaic-GPU Collective-Metadata Custom Call Failures: Failures in Mosaic-GPU collective-metadata custom calls related to NCCL symmetric window registration are identified in multi-kernel multi-process programs. Key blockers include buffer alignment requirements, dynamic-slice fusion hiding custom calls from collective coloring, and shared spatially partitioned pool window reservations, with validated fixes enabling successful multi-node training.
- issues/47406
2.5 Issue Discussion Insights
This section will analyze the tone and sentiment of discussions within this project's open and closed issues that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.
Based on our analysis, there are no instances of toxic discussions in the project's open or closed issues from the past week.
III. Pull Requests
3.1 Open Pull Requests
This section provides a summary of pull requests that were opened in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.
Pull Requests Opened This Week: 27
Key Open Pull Requests
1. [ROCm] Migrate SMI device queries from rocm_smi to amd_smi: This pull request migrates the SMI device queries in the ROCm backend from the deprecated rocm_smi library to the newer amd_smi library by conditionally compiling against the appropriate path, refactoring the code to separate backend implementations, and aligning with RCCL's approach to ensure accurate PCIe bandwidth and xGMI topology reporting while preparing for the eventual removal of rocm_smi.
- URL: pull/47495
2. [ROCm] Move CI to rocm 7.14 take two: This pull request updates the continuous integration (CI) setup to use ROCm version 7.14, including related fixes such as resolving LLVM symbol clashes, adjusting test configurations, and enforcing dynamic linking to improve compatibility and correctness for GPU and CPU tests.
- URL: pull/47522
3. [XLA:GPU][oneAPI] Add an HLO pass to resize matmul workspace size with oneDNN: This pull request adds the SyclGemmWorkspacePass, an Intel-GPU-specific HLO pass that adjusts the cuBLASLt matmul workspace size to match the requirements of oneDNN's matmul primitive scratchpad, optimizing workspace usage beyond the default 4 MiB set by GemmRewriter.
- URL: pull/47696
Other Open Pull Requests
- AMDGPU Kernel Occupancy Model Enhancements: Two pull requests introduce a theoretical occupancy model for AMDGPU kernels in the ROCm profiler, replacing the deprecated
hipOccupancy-based implementation with a pure-arithmetic library that computes occupancy from kernel symbol data and hardware constants. These changes improve accuracy and coverage of occupancy statistics, add detailed handling of resource limits, and include comprehensive tests to validate correctness.
- oneAPI and SYCL Support Improvements: Multiple pull requests enhance oneAPI and SYCL support by adding F64 data type support for matrix multiplication, enabling SoL latency estimator for gfx942 GPUs with native FP8 types, adding oneDPL 2022.13.0 dependency with SYCL kernel stubs, and applying a workaround for a oneAPI 2026 compiler bus error. These updates improve backend capabilities and maintain compatibility with evolving oneAPI compiler versions.
- TensorFlow and CPU Backend Fixes: Several pull requests address CPU backend issues by adding AMX_FP8 CPU feature detection to fix build errors, correcting target triple and data layout for LLVM modules to support big-endian architectures, and updating oneDNN integration to fall back to a static Eigen threadpool when intra-op threadpool is unavailable. These fixes ensure build stability and correct execution across diverse CPU environments.
- GPU Performance and Backend Enhancements: Pull requests improve GPU performance by restoring coalesced memory writes in scatter operations, updating the GPU backend to detect maximum PTX version from LLVM, and adding a target check for Intel-XPU Triton ThreadDim extraction. These changes optimize execution efficiency and extend support for additional GPU targets.
- Collective Operations and Backend Refactoring: Two pull requests register the oneCCL collective backend for GPU support and extract
collective_rendezvousinto a separate library to isolate runtime support code from collective operations analysis. These changes modularize collective backend support and align GPU collective backends.
- Mathematical Function Fixes in XLA HLO: Pull requests fix the pole behavior of the
Digammafunction to align with TensorFlow CPU kernels and correct false shape-mismatch errors in several special functions by improving shape compatibility checks. These fixes ensure consistent and correct mathematical computations across platforms.
- Dynamic Dimension and Conditional Canonicalization: Two pull requests fix compiler crashes and compilation failures by enabling support for while loops with non-tuple loop-carried state and adding a ConditionalCanonicalizer pass to interpreter pipelines. These changes ensure correct handling of dynamic dimensions and conditionals in XLA's interpreter backend.
- Triton Tile Selection and Shared Memory Constraints: One pull request adds a shared-memory constraint to the Triton tile-selection process for transpose operations, rejecting tile sizes that exceed device shared memory capacity to prevent RESOURCE_EXHAUSTED errors. This improves reliability of kernel compilation by enabling fallback to smaller tiles.
- ROCm DNN Cleanup: One pull request removes the unused BW pool cache from the ROCm DNN component, which was off by default and never utilized, simplifying the codebase.
- oneDNN and AArch64 Integration Updates: One pull request updates the oneDNN integration for AArch64 by bumping the source archive version, updating Bazel build files, adding compatibility patches, and revising threadpool integration to align with the newer interface.
- GPU RaggedAllToAll Collective Optimization: One pull request fixes an O(n_devices) overhead in the GPU
RaggedAllToAllcollective by skipping zero-size peer slices in NCCL fallback paths, significantly improving performance for sparse neighbor exchange patterns.
- StableHLO Serialization Improvement: One pull request enhances StableHLO serialization by preserving the
chlo.erfcoperation as a stablehlo.composite, enabling round-trip serialization and improving compatibility for functions likejax.nn.gelu(x, approximate=False).
- CI Test Tagging for ROCm CPU Step: One pull request separates lit tests that use only FileCheck by tagging them with the gpu tag, enabling these tests to run specifically during the rocm_cpu step of continuous integration.
3.2 Closed Pull Requests
This section provides a summary of pull requests that were closed in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.
Pull Requests Closed This Week: 35
Key Closed Pull Requests
1. [ROCm] MORI Intergation-1: Bazelify MORI build and vendor rdma-core (…: This pull request introduces a hermetic Bazel build integration for the MORI RDMA transport by vendoring rdma-core's libibverbs headers as a header-only repository with include remapping, applying necessary patches for clang-18 and ROCm dependencies, and enabling MORI to compile under a hermetic toolchain without relying on system headers.
- URL: pull/44636
2. rocm sol latency estimator: This pull request aims to enable the SOL latency estimator for ROCm by ungating the estimator on gfx942 and gfx950 architectures, updating matmul and collective interpolator data, and improving code quality and formatting, although it has not been merged.
- URL: pull/46575
3. Fix byte-vs-bit shift bug in GPU PadToStatic dynamic size metadata: This pull request addresses a critical bug in the GPU PadToStatic dynamic size metadata handling by correcting a byte-to-bit shift mismatch in LLVM emitter helper functions, which previously caused silent data corruption for XLA GPU operations involving sub-32-bit data types with dynamic shapes, ensuring accurate dynamic dimension size storage and retrieval for workloads using 16-bit floats and ints.
- URL: pull/47509
Other Closed Pull Requests
- Collective operations support and improvements: Multiple pull requests introduce and enhance collective operations on various GPU backends. These include adding a new CollectiveReduce HLO for NVIDIA GPUs, enabling collective operations on Intel GPUs with oneCCL, and proposing a separate scheduling resource for scale-up collectives using NVLINK and data center networks to improve latency hiding and overlap.
- pull/46104, pull/46689, pull/47467
- Triton backend enhancements: Several pull requests improve Triton support in the XLA GPU backend by adding kRaggedDot operation support, integrating the oneAPI Triton pipeline into the shared compilation path, and implementing a fallback mechanism that retries kernel compilation with smaller tile sizes upon memory overflow. These changes enable more efficient ragged-dot computations and more robust kernel compilation.
- pull/45997, pull/47169, pull/47192
- ROCm platform support and fixes: Multiple pull requests focus on ROCm support improvements, including enabling the SoL latency estimator for gfx950 GPUs, removing unused RecordApiTrace API to clean up the interface, fixing a delay kernel for event-based timers to improve timing accuracy, and proposing ROCm support for unit tests to increase coverage. These efforts enhance ROCm backend stability, profiling accuracy, and test coverage.
- pull/47515, pull/47065, pull/47447, pull/47451
- Bug fixes in XLA core computations and compiler: Several pull requests address bugs in XLA computations and compiler internals. Fixes include correcting dynamic output size calculations in CPU scan-like reduce-window operations, handling zero-sized matrix dimensions in SVD to match eager TensorFlow and numpy behavior, and initializing dynamic dimension sizes in Literal::Broadcast to prevent uninitialized memory errors. These fixes improve correctness and stability in XLA computations.
- pull/46618, pull/46800, pull/46893
- StreamExecutor and PJRT improvements for GPU: A pull request enhances the StreamExecutor PJRT implementation by adding fine-grained per-result buffer definition events, enabling earlier consumption of results and better integration with ThunkExecutor's tracking. This update improves GPU buffer allocation mapping and output creation while preserving fallback behavior.
- pull/47012
- Security and coordination service enhancements: One pull request adds mutual TLS (mTLS) credentials support to the coordination service, exposes verification functions, and introduces a new client configuration parameter to replace a hardcoded timeout. These changes prepare the codebase for integrating mTLS into distributed initialization.
- pull/47226
- CUTLASS runtime and CUDA host allocation fixes: A pull request pins CUTLASS runtime artifacts for specific architectures and CUDA versions, exposes runtime dependencies for testing, and enables a vector-add FFI test. Another pull request disables transfer buffer coalescing to prevent intermittent CUDA errors by ensuring each transfer buffer corresponds to a single pinned host allocation.
- pull/40978, pull/42349
- Logging and thread affinity improvements: One pull request modifies GSPMD sharding propagation warnings to print only once per process to reduce log spam. Another adjusts NUMA thread affinity logic to skip binding when no allowed CPUs exist due to cgroup restrictions, avoiding misleading error logs on SLURM-managed clusters.
- pull/47133, pull/47153
- Precision and profiling accuracy improvements: A pull request improves bf16 scalar natural logarithm accuracy on gfx1250 by lowering to a correctly-rounded f32 log2 operation, reducing error and updating tests. Another fixes CUPTI V2 timestamp error handling by returning explicit error statuses to prevent valid profiling events from being discarded.
- pull/47222, pull/47270
- Unmerged and testing-only changes: One pull request proposes a fix for FP32 atomic_rmw in ROCm but was created mainly for testing and not merged.
- pull/14117
3.3 Pull Request Discussion Insights
This section will analyze the tone and sentiment of discussions within this project's open and closed pull requests that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.
Based on our analysis, there are no instances of toxic discussions in the project's open or closed pull requests from the past week.
IV. Contributors
4.1 Contributors
Active Contributors:
We consider an active contributor in this project to be any contributor who has made at least 1 commit, opened at least 1 issue, created at least 1 pull request, or made more than 2 comments in the last month.
If there are more than 10 active contributors, the list is truncated to the top 10 based on contribution metrics for better clarity.
| Contributor | Commits | Pull Requests | Issues | Comments |
|---|---|---|---|---|
| cxxhou | 0 | 0 | 85 | 0 |
| ezhulenev | 33 | 7 | 0 | 5 |
| draganmladjenovic | 37 | 4 | 0 | 1 |
| kodlan | 23 | 10 | 0 | 3 |
| alekstheod | 33 | 0 | 0 | 0 |
| shawnwang18 | 24 | 0 | 0 | 0 |
| pemeliya | 18 | 4 | 0 | 0 |
| nurmukhametov | 17 | 2 | 0 | 0 |
| mfrancepillois | 15 | 3 | 0 | 1 |
| akuegel | 0 | 0 | 0 | 13 |