Weekly GitHub Report for Llama.cpp

Thank you for subscribing to our weekly newsletter! Each week, we deliver a comprehensive summary of your GitHub project's latest activity right to your inbox, including an overview of your project's issues, pull requests, contributors, and commit activity.

Table of Contents

• I. News
  • 1.1. Recent Version Releases
  • 1.2. Other Noteworthy Updates
• II. Issues
  • 2.1. Top 5 Active Issues
  • 2.2. Top 5 Stale Issues
  • 2.3. Open Issues
  • 2.4. Closed Issues
  • 2.5. Issue Discussion Insights

• III. Pull Requests
  • 3.1. Open Pull Requests
  • 3.2. Closed Pull Requests
  • 3.3. Pull Request Discussion Insights

• IV. Contributors
  • 4.1. Contributors

I. News

1.1 Recent Version Releases:

The current version of this repository is b4991

1.2 Version Information:

The version released on March 29, 2025, introduces key updates and improvements, focusing on enhanced performance and user experience. Notable highlights include optimized features and bug fixes that streamline functionality and increase stability.

II. Issues

2.1 Top 5 Active Issues:

We consider active issues to be issues that that have been commented on most frequently within the last week. Bot comments are omitted.

1. [BUG-UNCONFIRMED] Eval bug: DeepSeep V4 garbled output on Strix Halo with ROCm: This issue reports that when running the DeepSeek V4 model with the ROCm backend on a Strix Halo GPU, the output is garbled despite attempts with different settings and versions, including the latest nightlies. Users discuss reproducing the problem, share detailed logs and patches for debugging, and eventually identify that the garbled output is linked to the use of the GGML_CUDA_ENABLE_UNIFIED_MEMORY environment variable, with removal of this variable resolving the issue for some, though the root cause remains a bug needing further investigation.

   • The comments show multiple users confirming the garbled output issue on ROCm with DeepSeek V4, sharing environment details, debugging patches, and logs; they compare ROCm versions and build configurations, and ultimately find that disabling GGML_CUDA_ENABLE_UNIFIED_MEMORY fixes the problem, though they acknowledge this is a workaround rather than a true fix and call for expert help to resolve the underlying bug.
   • Number of comments this week: 19

2. [SYCL/xe] -cb pins GPU at gt-c0 on Battlemage, prevents idle power savings: This issue reports that when using the -cb (continuous batching) flag with the SYCL backend on an Intel Arc Pro B70 GPU, the GPU remains stuck in a high-power active state (gt-c0) between inference requests, preventing it from entering a low-power idle state (gt-c6) and causing sustained high power draw and temperature. The root cause is identified as an interaction between the -cb flag keeping the SYCL command queue armed and the new xe driver gating power state transitions on queue drainage, which differs from previous driver behavior; the suggested resolutions include documentation updates, runtime warnings, or explicit queue draining to restore idle power savings.

   • Commenters generally agree with the diagnosis and support adding documentation to warn users about the -cb flag’s impact on power savings with the xe driver, though some report differing behavior on other hardware models like the B60; attempts to reproduce the issue vary, with some unable to observe the power state changes, and others confirming the problem on the B70, emphasizing hardware and driver version differences as factors.
   • Number of comments this week: 10

3. [BUG-UNCONFIRMED] Eval bug: Gemma 4 crashes with MTP: This issue reports a crash occurring in llama-server when running the Gemma 4 31b model with MTP enabled on a dual GPU setup using the tensor split mode. The crash happens on the second prompt in a session due to a bug related to splitting attention layers with a single KV head across multiple GPUs, which is not supported by the tensor split mode and causes memory and scheduling conflicts.

   • The comments discuss the root cause being the tensor split mode's inability to handle single KV head layers, suggest using layer split mode as a workaround despite throughput loss, and share a detailed patch addressing use-after-reset bugs in the GGML tensor-split backend; testing the patch reveals mixed results with some crashes persisting or worsening under certain flags.
   • Number of comments this week: 8

4. [BUG-UNCONFIRMED] Compile bug: Windows compilation with BLAS support using AOCL as BLAS vendor fails (using OpenBLAS works): This issue describes a compilation failure when building llama.cpp with BLAS support on Windows using AMD's AOCL as the BLAS vendor, while compilation succeeds using OpenBLAS. The problem is caused by macro conflicts with Windows headers that interfere with standard functions like std::max and std::min, and a manual workaround involving undefining these macros in the source file allows successful compilation with both MSVC and Intel compilers, though the user requests an official fix.

   • The comments discuss the difficulty of the macro conflict issue, share various compile commands and environments, confirm the workaround of undefining min/max macros or using the -DNOMINMAX flag, and highlight complications with OpenMP runtime DLL conflicts between MSVC and Intel compilers; users also share performance observations and detailed steps to generate necessary import libraries, concluding that the manual fix works but an integrated solution would be preferable.
   • Number of comments this week: 7

5. [BUG] [REGRESSION] [SPECULATIVE] MTP feature performance drops since b9935: This issue reports a significant performance regression in the MTP feature of the llama-server module starting from version b9235, with throughput dropping by approximately 15-20% compared to earlier versions like b9180 and b9222. The user provides detailed test results on a Windows 11 system with specific hardware, noting slower generation speeds especially in long context tasks, and seeks insight into what changes during the "clean-up" activity might have caused this slowdown.

   • The comments discuss attempts to identify the exact commit causing the regression, suggest running git bisect, mention a reverted pull request impacting Windows performance, and note that the regression may be linked to a cleanup PR for MTP; some users report not seeing the slowdown while others confirm similar issues, particularly with prompt processing speed.
   • Number of comments this week: 7

2.2 Top 5 Stale Issues:

We consider stale issues to be issues that has had no activity within the last 30 days. The team should work together to get these issues resolved and closed as soon as possible.

As of our latest update, there are no stale issues for the project this week.

2.3 Open Issues

This section lists, groups, and then summarizes issues that were created within the last week in the repository.

Issues Opened This Week: 65

Summarized Issues:

• Build and Compilation Issues: Several issues describe build failures and compile bugs affecting different platforms and configurations. Problems include read-only files causing permission errors on Linux, macro conflicts on Windows with AMD AOCL BLAS, incomplete UI asset embedding causing build failures, and missing shared libraries for SYCL on Kubuntu 24.04, all requiring patches or workarounds to resolve.
  • issues/25374, issues/25413, issues/25429, issues/25443, issues/25573

• Model Loading and Memory Management Bugs: Multiple issues report crashes, memory overallocations, and hangs during model loading or inference. These include out-of-memory errors on Mac M1 Ultra, excessive memory use in DeepSeek-V4-Flash, memory leaks in checkpoint freeing, and hangs on AMD RDNA4 hardware due to disk I/O and memory patterns.
  • issues/25424, issues/25437, issues/25468, issues/25520

• Backend and Hardware-Specific Failures: Several issues describe incorrect outputs, crashes, or performance regressions tied to specific hardware or backend configurations. Examples include garbled output on ROCm with Strix Halo GPU, OpenCL backend failures on Adreno GPUs, CUDA backend assertion failures, Vulkan backend crashes, and SYCL backend resource errors on Intel Arc GPUs.
  • issues/25376, issues/25436, issues/25438, issues/25455, issues/25477, issues/25530, issues/25579, issues/25582, issues/25594, issues/25598, issues/25612

• Performance Regressions and Optimization Requests: Some issues report significant slowdowns or propose performance improvements. Notable regressions include a 7-9% speed drop on CUDA RTX 5070, a 15-20% throughput drop in MTP on Windows, and a 25-30% CPU decode slowdown with OpenBLAS. There is also a proposal to improve Vulkan prefill throughput by removing redundant work.
  • issues/25422, issues/25489, issues/25491, issues/25565

• Server and Proxy Operation Bugs: Issues include proxy errors immediately after forwarding requests, server crashes on multi-GPU MTP tensor splits, failure to produce output without verbose logging, and key-value cache leaks causing context size errors. These bugs affect server stability and correct request handling.
  • issues/25396, issues/25400, issues/25456, issues/25522, issues/25566

• API and Endpoint Validation Issues: Several problems arise from improper error handling and validation in API endpoints. These include malformed tool call arguments causing HTTP 500 errors instead of 4xx, chat completions with invalid headers returning 500 errors instead of 400, and rerank endpoint returning near-zero relevance scores due to missing tensors.
  • issues/25447, issues/25510, [issues/25605](https://github.com/issues/25605]

• UI and Web Interface Bugs: Multiple issues affect the user interface, such as reasoning settings not appearing on mobile, inability to edit reasoning settings, missing reasoning enable toggle, and bugs in the WebUI MCP panel including missing checkboxes and indefinite timeouts.
  • issues/25415, issues/25534, issues/25591, issues/25625

• Quantization and Model Format Support: Issues include lack of dimension validation in the Python GGUF parser causing crashes, segmentation faults loading new MTP models, requests for support of gemini nano and diffusion models, and bugs in speculative decoding with quantized models causing divergent outputs.
  • issues/25378, issues/25392, issues/25427, issues/25461, [issues/25618](https://github.com/issues/25618]

• CUDA and FP16 Accuracy Bugs: There are bugs related to CUDA architecture handling and FP16 math, including incorrect FP16 acceleration on SM_60 causing accuracy loss and assertion failures in CUDA backend when handling non-contiguous quantized KV cache inputs.
  • issues/25530, [issues/25593](https://github.com/issues/25593]

• Miscellaneous Bugs and Requests: Other issues include typos in Dockerfiles, inconsistent model name capitalization causing unsupported errors, missing spaces in tool call requests, and requests for features like idle router worker termination and realtime statistics endpoints.
  • issues/25546, issues/25547, issues/25626, issues/25519, [issues/25570](https://github.com/issues/25570]

2.4 Closed Issues

This section lists, groups, and then summarizes issues that were closed within the last week in the repository. This section also links the associated pull requests if applicable.

Issues Closed This Week: 80

Summarized Issues:

• CUDA Backend Crashes and Errors: Multiple issues report crashes and errors related to CUDA operations, including GPU watchdog timeouts during tensor copies, matrix multiplication errors due to incorrect CUDA architecture flags, and fatal errors in CUDA kernel files causing execution failures. These problems often require workarounds like adjusting build flags, reducing batch sizes, or upgrading CUDA versions to fixed releases.
• issues/20146, issues/21289, issues/22499, issues/23539, issues/23469

• CUDA Version and Compilation Issues: There are compilation failures and incorrect outputs caused by specific CUDA versions, notably CUDA 13.2, which produces gibberish outputs for quantized models and compilation warnings due to large floating-point literals. These issues were resolved in later CUDA releases or require code adjustments to avoid warnings.
• issues/21255, issues/22824

• Vulkan Backend Bugs and Performance Problems: Several issues describe crashes, segmentation faults, and performance slowdowns in the Vulkan backend, including buffer size misestimations during CPU offload, token generation speed drops on AMD GPUs, and crashes on Intel GPUs during prefill stages. These problems affect stability and throughput and sometimes require disabling features or restarting the server.
• issues/21473, issues/21762, issues/22898, issues/23068

• Model-Specific Output and Parsing Issues: Various models such as Qwen 3.6, GLM 5.1, and MiniCPM5-1B exhibit output problems including gibberish text, XML or doubled tags in responses, and unsupported XML-style tool calls. These issues cause parsing errors, unexpected stops, or stuck states during inference and tool interactions.
• issues/21915, issues/22398, issues/23781, issues/22878

• Performance Regressions and Slowdowns: Users report significant performance drops in token generation speed and iteration throughput after updates to software versions, NVIDIA drivers, or during long sessions with large multimodal requests. These regressions impact user experience and sometimes require process restarts or driver rollbacks.
• issues/22443, issues/22899, issues/23765, issues/23537

• MTP (Multi-Token Prediction) and Speculative Decoding Issues: Several bugs and crashes occur when using MTP features, including device mismatches between ROCm and Vulkan backends, out-of-memory errors, pool overflow assertion failures, and crashes triggered by unsupported parameters during matrix multiplication. Proposed improvements include pipeline approaches to reduce redundant verification.
• issues/23184, issues/23199, issues/23244, issues/23469, issues/23500, issues/23523

• Backend and Hardware Compatibility Problems: Issues include OpenCL backend failures due to missing FP16 support, SYCL backend hangs and crashes on Intel GPUs, and HIP CI build breakage on AMD GPUs. These compatibility problems cause tool execution failures, hangs, or compilation errors requiring kernel parameter adjustments or build flag fixes.
• issues/22762, issues/25350, issues/25361

• Memory Management and Out-of-Memory Errors: Problems with memory pool overflows, underestimation of GPU memory requirements, and failure to prevent VRAM swapping on AMD GPUs cause crashes, slowdowns, or model loading failures. Workarounds include increasing container shared memory, using dummy display plugs, or disabling certain features.
• issues/23500, issues/23244, issues/25313, issues/25145

• Server and CLI Usability and Feature Requests: Users request improvements such as better --help formatting, conversation transcript saving, context usage gauges in the UI, and KV cache utilization metrics. Other usability issues include inability to exit sessions in Windows PowerShell and disabling intrusive UI pop-ups for privacy.
• issues/23016, issues/25404, issues/24346, issues/23632, issues/22779, issues/25274

• Prompt Processing and Checkpointing Inefficiencies: Several issues describe inefficient prompt processing due to fragmented batch handling, excessive checkpointing triggered by message flags, and prompt cache reuse failures causing full reprocessing on each request. These inefficiencies degrade performance and increase resource usage during multi-turn conversations.
• issues/25213, issues/25023, issues/25567, issues/25320

• Model Loading and Format Compatibility Bugs: Bugs include failures to load certain model architectures or formats such as Gemma4AssistantForCausalLM with GGUF, CPU repack buffer allocation failures for large MoE models, and aborts caused by empty metadata keys in GGUF files. These prevent successful model initialization or cause crashes.
• issues/23727, issues/23603, issues/24884

• Web UI and Browser Compatibility Issues: The built-in web UI is reported broken in recent versions, with scrolling disabled on iOS and Firefox due to CSS changes, and unsolicited third-party HTTP requests raising privacy concerns. Downgrading or reverting commits temporarily resolves these problems.
• issues/24808, issues/24617, issues/25509

• Tokenization and API Validation Bugs: The tokenizer produces divergent token sequences compared to HuggingFace when handling special tokens with leading spaces, and the Responses API incorrectly rejects valid client inputs due to strict type checks. Additionally, OpenAI-compatible endpoints reject nullable sampling parameters improperly.
• issues/23840, issues/23557, issues/25515

• Crash and Segmentation Faults in Various Contexts: Crashes occur in multiple scenarios including Metal backend on macOS with KV cache offload, grammar-constrained generation with specific tokens, illegal instruction errors during agentic workloads, and segmentation faults when loading models with Vulkan backend and parallel options.
• issues/23578, issues/23677, issues/23811, issues/23663

• Multi-GPU and Layer Assignment Imbalances: Issues include unexpected VRAM usage discrepancies when combining local GPUs with RPC servers and biased tensor split layer assignments causing uneven multi-GPU load distribution. Proposed fixes involve adjusting layer assignment heuristics for better balance.
• issues/23679, issues/23691

• Model Output Corruption on Multi-GPU SYCL Setups: Running certain models on Intel SYCL multi-GPU configurations causes heavily corrupted output with random tokens due to miscalculated memory offsets in sliding window attention and KV cache management, leading to unusable results.
• issues/23795

• Attention Mechanism Bugs in OpenCL Backend: A heuristic incorrectly applies causal masking to ViT full-attention layers in the OpenCL backend, causing vision transformers to produce incorrect outputs by converting bidirectional attention into unidirectional, fixed by disabling the heuristic.
• issues/23800

• Model Loading and Startup Failures on Specific Hardware: On Apple M4 hardware, the server exits immediately when serving certain models in CPU-only mode, blocking workflows and requiring stable binaries or build guides for resolution.
• issues/23631

• Miscellaneous Bugs and Requests: Other issues include extremely long model load times with SYCL tensor parallelism, missing release assets causing download failures, and a placeholder test issue with no description.
• issues/25423, issues/25394, issues/25600

2.5 Issue Discussion Insights

This section will analyze the tone and sentiment of discussions within this project's open and closed issues that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.

Based on our analysis, there are no instances of toxic discussions in the project's open or closed issues from the past week.

III. Pull Requests

3.1 Open Pull Requests

This section provides a summary of pull requests that were opened in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.

Pull Requests Opened This Week: 92

Key Open Pull Requests

1. ui: Agentic Content UX improvements: This pull request introduces multiple user experience improvements to the Agentic Content UI, including enhanced animations, redesigned collapsible content components, conditional setting displays, refined styling, new tool call features, improved auto-scroll behavior, and overall better layout and interaction for agentic chat messages.

• URL: pull/25450

• Associated Commits: f2e07, ea0b9, a7ee0, 0b990, 12f3f, 698c2, 07b3d, bc396, 1f93f, 6a815, 7f138, c5c60, 53991, 4638d, 44cfa, e33e7, 3969a, 2593d, b0451, 18427, bf5d6, dd35b, 7e87b, 4fa0f, dd31e, d46bf, 8e202, 18253, 81d11, 4a158, 7865c, 570bd, ed92d, c2161, e37fa

2. /slots related API fixes: This pull request fixes various issues related to the /slots API, including ensuring checkpoints are properly cleared on /slots/restore to prevent confusion in later computations, tightening error handling such as requiring filename to be a string and rejecting negative id_slot values, and expands the test suite with structured pytest tests to improve coverage and reliability of slot-related functionality.

• URL: pull/25462

• Associated Commits: 72210, 58d34, e1857, 73ee2, d0532, 9815d, dd7f4, dd5c9, f3e16, e5ba8, 66e10, 34076

3. spec: support DSpark for Gemma4: This pull request adds support for DSpark speculative decoding draft models specifically for the Gemma4 backbone in the llama.cpp project, enabling integration, conversion, building, and running of DSpark drafts with improved performance and preliminary functionality for testing.

• URL: pull/25549

• Associated Commits: 8006a, 3e048, 5bfd3, f2bd1, 57690, c52ba, 42865, 80d1d, dc900, a0dcc, d81b0

Other Open Pull Requests

• Mixture-of-Experts (MoE) Model Support: Multiple pull requests add comprehensive support for various MoE models including NVIDIA's Nemotron-3-Puzzle-75B-A9B, Tencent's Hy3 299B parameter model, and Qwen3-Next with MTP support. These include per-layer heterogeneous MoE configurations, multi-token-prediction speculative decoding heads, model registration, routing, conversion tooling, and performance benchmarking.
  • pull/25444, pull/25395, pull/25589

• Speculative Decoding and Sampling Enhancements: Pull requests extend backend sampling to support multiple outputs per sequence, add multiple end sequences in reasoning budget samplers, and optimize CUDA-based activation quantization for MoE gates. These changes improve deterministic sampling, support new termination tokens, and reduce redundant quantization work for better performance.
  • pull/25532, pull/25544, pull/25441

• Vulkan Backend Improvements: Several pull requests enhance Vulkan backend support by adding POOL_1D operation, optimizing q8_0 key-value tensor dequantization, and introducing reshaped shared memory layouts for Intel Xe GPUs. These updates include new compute shaders, pipeline creation, dispatch logic, and device-specific performance improvements.
  • pull/25431, pull/25494, pull/25380, pull/25465

• Quantization Format Support and Optimization: Pull requests add support for Q2_0 quantization in CUDA backend, optimize q8_0 quantization by expanding value range, and improve activation quantization deduplication for MoE. These changes enhance throughput, reduce quantization error, and maintain bit-identical outputs across multiple quantization types.
  • pull/25603, pull/25493, pull/25441

• Deepseek Graph and Template Fixes: Updates include adding fused hyper-connection sinkhorn operations to DeepseekV4 graph for performance gains and fixing the DS4 crafted template to retain all reasoning traces correctly. These changes reduce graph nodes significantly and ensure correct behavior based on reasoning flags and message history.
  • pull/25585, pull/25414

• Model Conversion and Indexer Support: Pull requests add support for forthcoming Granite models with interleaved Sliding Window Attention and full indexer support for GLM 5.2 by adapting Deepseek 3.2 indexing graphs. These include conversion infrastructure, per-layer RoPE/NoPE handling, and interleaved rotary positional embeddings.
  • pull/25505, pull/25407

• Backend Allocator and Memory Management Fixes: Fixes address VRAM overallocation during speculative context fitting and uninitialized view-only tensor tails in the ggml backend allocator. These changes improve memory footprint accuracy and prevent asserts caused by uninitialized data.
  • pull/25465, pull/25584

• Server and Error Handling Improvements: The server is modified to return HTTP 400 errors with detailed diagnostics for malformed tool-call arguments instead of HTTP 500 errors. This includes changing exception types, adjusting error message indexing, and adding regression tests for validation.
  • pull/25583

• User Interface Integration: Bodega One is added as a local-first IDE integrating llama.cpp for local inference with automatic hardware detection and configurable context size. It supports offline and private workflows via a bundled server and GGUF catalog for easy setup.
  • pull/25375

3.2 Closed Pull Requests

This section provides a summary of pull requests that were closed in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.

Pull Requests Closed This Week: 155

Key Closed Pull Requests

1. Initial ET backend : This pull request introduces the initial ET backend for llama.cpp, developed by AINekko and the AIFoundry community, to support the ET-SOC-1 processor originally created by Esperanto Technologies, enabling integration of this open-source hardware into the open-source inference ecosystem with support for various language model operations, kernel optimizations, and experimental Uberkernel features.

• URL: pull/24179

• Associated Commits: 4e2fc, 460ec, f6d4f, 726e3, 1392e, cbb2c, 54055, eae63, 19b88, 2bddc, 413a8, 3c145, 88c4e, 67f8d, 01d9b, 518e1, 1d371, 457b9, eaf31, 1581a, 6b367, 9ca96, 3a521, 7ad00, a8faf, 8df2f, 0428d, e2ab0, 0da8f, 438dc, afe58, cd25f, a2157, 9bb17, 21051, 316d3, adf4c, b4a52, fadbf, be469, 5f5a0, 2c381, cbc72, 004ed, ab00a, 7f364, 9db45, 22cde, e4c4f, 4d9a2, 39414, 1d768, 7f1e7, 1e472, f6bb4, 37fe3, 90f2d, ecef7, 05afd, 0291f, 87a1a, 16fc0, 1e5e8, eb52e, 75db6, 51309, 300c7, 9436d, 8ca13, 71049, e44c9, c67dc, f8216, d98d6, 6846d, b1165, 71a78, 18e41, 0aeec, 697fb, 42c6f, 212af, 15739, d148a, 238a7, c0fd7, b4888, aeab6, 92c45, 9a2d3, 32717, 2751b, 0b23b, 9b92e, 96161, 7a63a, 3f44f, bd0b2, 48eb3, cda34, 15ba0, 36d34, 402a3, 593ff, 1b9cc, 8eec7, fa50b, 5774d, a9221, 9d7f0, d28b1, f8f96, 4b172, 2c601, c6a46, 8a0f4, b3c20, 49167, 236df, 57136, 391ce, 2ec50, 70a34, df40e, 37b04, 3b9df, df5f5, ca421, c666d, 897c9, 5a78f, 3adfd, ecf2c, 5f6b7, ed12d, 510ec, 7f837, 43de7, b1281, e0d2e, 287fb, a6410, 51dbb, 0469f, 33e37, 91513, f9506, 5f0a1, 1ec96, 6ad38, 3d92c, 0a2f9, da6ca, fba80, 1d0a9, c4ffe, 7bef0, 04af4, f9de2, fccb8, f9c70, 47df9, f87f8, 35b0c, 3b711, 33c90, f400f, 5b58e, 3f885, d7e9f, 48293, a95ba, e040d, 6d4b0, a48a9, f810b, 1e5d3, 5439f, 57cbe, ef5b7, 28ac4, 8e08b, 7bf71, b06f3, ff5fa, 5fe0f, 10093, 4f5e5, 0c4f2, 20f59, 15465, 2e3b4, 671fb, ecfee, 3ab85, cb9d3, c61f8, 84c6c, d7af0, 6f9fa, c1d64, b1162, 0516a, 7c725, 285c1, 8c4c2, 45789, 044b3, 9f046, 3caa8, 87f03, 5197e, 0fdb6, 6ba47, 9c057, cfc58, 60a3e, 35ccc, 0a9b5, 2e8af, 381ce, 1457d, de752, 79a73, ae5cc, 6fff1, afd7b, bb788, e7ff5, 40891, 020aa, 8538a, 92f36, acc0b, 44029, 3cae6, e3e27, 02aab, 24292, 75f38, bcdd0, 4be93, fae55, 1928b, f867f, ac6ce, 0944c, 5a9f7, 4102e, c22c9

• Associated Commits: 4e2fc, 460ec, f6d4f, 726e3, 1392e, cbb2c, 54055, eae63, 19b88, 2bddc, 413a8, 3c145, 88c4e, 67f8d, 01d9b, 518e1, 1d371, 457b9, eaf31, 1581a, 6b367, 9ca96, 3a521, 7ad00, a8faf, 8df2f, 0428d, e2ab0, 0da8f, 438dc, afe58, cd25f, a2157, 9bb17, 21051, 316d3, adf4c, b4a52, fadbf, be469, 5f5a0, 2c381, cbc72, 004ed, ab00a, 7f364, 9db45, 22cde, e4c4f, 4d9a2, 39414, 1d768, 7f1e7, 1e472, f6bb4, 37fe3, 90f2d, ecef7, 05afd, 0291f, 87a1a, 16fc0, 1e5e8, eb52e, 75db6, 51309, 300c7, 9436d, 8ca13, 71049, e44c9, c67dc, f8216, d98d6, 6846d, b1165, 71a78, 18e41, 0aeec, 697fb, 42c6f, 212af, 15739, d148a, 238a7, c0fd7, b4888, aeab6, 92c45, 9a2d3, 32717, 2751b, 0b23b, 9b92e, 96161, 7a63a, 3f44f, bd0b2, 48eb3, cda34, 15ba0, 36d34, 402a3, 593ff, 1b9cc, 8eec7, fa50b, 5774d, a9221, 9d7f0, d28b1, f8f96, 4b172, 2c601, c6a46, 8a0f4, b3c20, 49167, 236df, 57136, 391ce, 2ec50, 70a34, df40e, 37b04, 3b9df, df5f5, ca421, c666d, 897c9, 5a78f, 3adfd, ecf2c, 5f6b7, ed12d, 510ec, 7f837, 43de7, b1281, e0d2e, 287fb, a6410, 51dbb, 0469f, 33e37, 91513, f9506, 5f0a1, 1ec96, 6ad38, 3d92c, 0a2f9, da6ca, fba80, 1d0a9, c4ffe, 7bef0, 04af4, f9de2, fccb8, f9c70, 47df9, f87f8, 35b0c, 3b711, 33c90, f400f, 5b58e, 3f885, d7e9f, 48293, a95ba, e040d, 6d4b0, a48a9, f810b, 1e5d3, 5439f, 57cbe, ef5b7, 28ac4, 8e08b, 7bf71, b06f3, ff5fa, 5fe0f, 10093, 4f5e5, 0c4f2, 20f59, 15465, 2e3b4, 671fb, ecfee, 3ab85, cb9d3, c61f8, 84c6c, d7af0, 6f9fa, c1d64, b1162, 0516a, 7c725, 285c1, 8c4c2, 45789, 044b3, 9f046, 3caa8, 87f03, 5197e, 0fdb6, 6ba47, 9c057, cfc58, 60a3e, 35ccc, 0a9b5, 2e8af, 381ce, 1457d, de752, 79a73, ae5cc, 6fff1, afd7b, bb788, e7ff5, 40891, 020aa, 8538a, 92f36, acc0b, 44029, 3cae6, e3e27, 02aab, 24292, 75f38, bcdd0, 4be93, fae55, 1928b, f867f, ac6ce, 0944c, 5a9f7, 4102e, c22c9

2. Turbo sycl opt: This pull request introduces and integrates TurboQuant weight quantization and KV-cache compression optimizations—including new TurboQuant kernels, asymmetric K/V compression policies, and backend support for CUDA, HIP/ROCm, Metal, Vulkan, and SYCL—enhancing model loading, memory efficiency, and inference speed with extensive testing, bug fixes, and upstream synchronization, while also adding support for new quantization types (TQ3_1S, TQ4_1S) and improving concurrency and device compatibility across multiple GPU architectures.

• URL: pull/25466

• Associated Commits: e3ce0, 71c7a, c29fa, 5bad8, e5ac9, 753f1, 65716, 51481, cc1ba, 941d4, 579db, 0bf1e, a4948, e596f, 3b5e1, 1df47, ff8bb, 88fcb, a4736, e53f8, 67755, 00093, 10736, 59798, fa4e8, f9ee9, 93bf2, 0198d, 8993d, 0757f, 7ca13, 627eb, 6112e, a1bcb, d3271, 90968, 6e063, 74450, 07595, f4ffe, 8ba9f, 4d24a, dad4e, b3401, 9e3fb, 70de2, 58bbe, d9adc, a4a09, 458c7, 4d754, d7b53, 15659, 67f07, 03704, f03d3, 99c2e, 99130, f64e8, b8b1d, 7f320, 67559, f2dc9, 11a24, db359, 817e9, 157f2, b6f8e, cde3e, 0fa61, e0954, d2d0d, 4f331, fec07, 60fc4, 170bd, 69d8e, e69af, 7e341, 5aeb2, d7420, 680d6, e89d8, 1e8fe, f3172, 08a68, 4306b, 8e894, 7101d, e30bb, 7f23a, bf590, 2191d, 4e351, c654c, eef2d, 2b61e, d6f7c, 2c514, 2a759, 99efa, c82ad, 1e4c0, 4a16c, 2cbfd, 7eb9e, a09ba, 112da, 253b0, 02f9a, 8cc38, 9682c, 8e12d, a6232, c9aa8, 7d971, 47106, 19c76, 67226, 41aef, 1c9f2, 697d1, 57cff, 676b3, 46cc1, ab11a, 337e0, 75606, 0c809, d55c8, 6d9a4, d1e70, 2f756, de389, 469c9, 3c0ef, 2e680, 73eb5, 0b8bd, f0d5c, 7985f, d75ef, ed81e, 7d5ac, f94c3, 36dc8, 35ac8, 8f3fb, 2f866, 58d40, 4595f, d6ae8, a33ef, 3faf3, 7eee2, de709, a7696, 9f8d0, fc358, 8b528, b1566, 90473, 4599a, 43505, ba7bb, b7adb, 8366c, ae1a0, dbf32, 0d518, db8f6, bd1f4, 01d98, 58a8c, 7f7af, b50ee, 014b5, db6af, c9357, 75231, 62705, f6793, 66c7f, 0ebbf, b6c79, 80153, bb905, b84e1, be52c, 52ddb, 6cbf1, 59ed2, b1725, 2fff3, 44567, 886a4, 77779, 94bd1, ae8c1, fca0c, 58f03, 023d7, 1bc5a, 7d734, 7929e, 99e61, d5bca, c6795, 85824, b9f06, b969b, f64aa, ba10b, 996c1, 7c60b, 17f24, a74b6, b2c71, dcdc5, 59cc8, 7a641, 4c79c, 535b2, b6667, fe0be, 4af3b, 67355, bd31b, e8ee2, 14d71, 62032, d4f2c, 1ca6b, 62af9, b4151, 27204, fb4ae, 6e4fa, 12351, 10b79, a318b, d4617, cd002, 60a8c, ba7f6, c437b, 75046, 8d204, fa88b, b1e4d

• Associated Commits: e3ce0, 71c7a, c29fa, 5bad8, e5ac9, 753f1, 65716, 51481, cc1ba, 941d4, 579db, 0bf1e, a4948, e596f, 3b5e1, 1df47, ff8bb, 88fcb, a4736, e53f8, 67755, 00093, 10736, 59798, fa4e8, f9ee9, 93bf2, 0198d, 8993d, 0757f, 7ca13, 627eb, 6112e, a1bcb, d3271, 90968, 6e063, 74450, 07595, f4ffe, 8ba9f, 4d24a, dad4e, b3401, 9e3fb, 70de2, 58bbe, d9adc, a4a09, 458c7, 4d754, d7b53, 15659, 67f07, 03704, f03d3, 99c2e, 99130, f64e8, b8b1d, 7f320, 67559, f2dc9, 11a24, db359, 817e9, 157f2, b6f8e, cde3e, 0fa61, e0954, d2d0d, 4f331, fec07, 60fc4, 170bd, 69d8e, e69af, 7e341, 5aeb2, d7420, 680d6, e89d8, 1e8fe, f3172, 08a68, 4306b, 8e894, 7101d, e30bb, 7f23a, bf590, 2191d, 4e351, c654c, eef2d, 2b61e, d6f7c, 2c514, 2a759, 99efa, c82ad, 1e4c0, 4a16c, 2cbfd, 7eb9e, a09ba, 112da, 253b0, 02f9a, 8cc38, 9682c, 8e12d, a6232, c9aa8, 7d971, 47106, 19c76, 67226, 41aef, 1c9f2, 697d1, 57cff, 676b3, 46cc1, ab11a, 337e0, 75606, 0c809, d55c8, 6d9a4, d1e70, 2f756, de389, 469c9, 3c0ef, 2e680, 73eb5, 0b8bd, f0d5c, 7985f, d75ef, ed81e, 7d5ac, f94c3, 36dc8, 35ac8, 8f3fb, 2f866, 58d40, 4595f, d6ae8, a33ef, 3faf3, 7eee2, de709, a7696, 9f8d0, fc358, 8b528, b1566, 90473, 4599a, 43505, ba7bb, b7adb, 8366c, ae1a0, dbf32, 0d518, db8f6, bd1f4, 01d98, 58a8c, 7f7af, b50ee, 014b5, db6af, c9357, 75231, 62705, f6793, 66c7f, 0ebbf, b6c79, 80153, bb905, b84e1, be52c, 52ddb, 6cbf1, 59ed2, b1725, 2fff3, 44567, 886a4, 77779, 94bd1, ae8c1, fca0c, 58f03, 023d7, 1bc5a, 7d734, 7929e, 99e61, d5bca, c6795, 85824, b9f06, b969b, f64aa, ba10b, 996c1, 7c60b, 17f24, a74b6, b2c71, dcdc5, 59cc8, 7a641, 4c79c, 535b2, b6667, fe0be, 4af3b, 67355, bd31b, e8ee2, 14d71, 62032, d4f2c, 1ca6b, 62af9, b4151, 27204, fb4ae, 6e4fa, 12351, 10b79, a318b, d4617, cd002, 60a8c, ba7f6, c437b, 75046, 8d204, fa88b, b1e4d

3. ui: Context usage gauge and panel: This pull request introduces a comprehensive UI enhancement featuring a new context usage gauge and panel with detailed token breakdowns and model info, restructures reasoning submenu defaults and layout, replaces the ChatScreenProcessingInfo component with the gauge, centralizes OpenAI wire format ownership in the toolsStore, adds a /tokenize endpoint with token count caching, and includes multiple refinements and cleanups to improve token tracking accuracy and interface usability.

• URL: pull/25340

• Associated Commits: 53cf3, ae1e1, 10e21, e979f, 3b88f, a6b2f, 5a2b9, 754dc, cc4a9, f4d0d, 5d019, 183dd, b288c, 3c856, 5030d, 83dc4, a2196, ae218, 22a8a, 601f2, 4ffb6, 1144a, 0f1ad, d262a, 1b23e, 9b96c, 43441, 7f0a8, 3c079, 7f015, f77b1, 2ca0b, c07cc, 5d887, 08909, 450fb, d9ebe, b81b5, 2fabc, f34a8, 6a1d5

• Associated Commits: 53cf3, ae1e1, 10e21, e979f, 3b88f, a6b2f, 5a2b9, 754dc, cc4a9, f4d0d, 5d019, 183dd, b288c, 3c856, 5030d, 83dc4, a2196, ae218, 22a8a, 601f2, 4ffb6, 1144a, 0f1ad, d262a, 1b23e, 9b96c, 43441, 7f0a8, 3c079, 7f015, f77b1, 2ca0b, c07cc, 5d887, 08909, 450fb, d9ebe, b81b5, 2fabc, f34a8, 6a1d5

Other Closed Pull Requests

3.3 Pull Request Discussion Insights

This section will analyze the tone and sentiment of discussions within this project's open and closed pull requests that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.

Based on our analysis, there are no instances of toxic discussions in the project's open or closed pull requests from the past week.

IV. Contributors

4.1 Contributors

Active Contributors:

We consider an active contributor in this project to be any contributor who has made at least 1 commit, opened at least 1 issue, created at least 1 pull request, or made more than 2 comments in the last month.

If there are more than 10 active contributors, the list is truncated to the top 10 based on contribution metrics for better clarity.