Weekly GitHub Report for Llama.cpp

Thank you for subscribing to our weekly newsletter! Each week, we deliver a comprehensive summary of your GitHub project's latest activity right to your inbox, including an overview of your project's issues, pull requests, contributors, and commit activity.

Table of Contents

• I. News
  • 1.1. Recent Version Releases
  • 1.2. Other Noteworthy Updates
• II. Issues
  • 2.1. Top 5 Active Issues
  • 2.2. Top 5 Stale Issues
  • 2.3. Open Issues
  • 2.4. Closed Issues
  • 2.5. Issue Discussion Insights

• III. Pull Requests
  • 3.1. Open Pull Requests
  • 3.2. Closed Pull Requests
  • 3.3. Pull Request Discussion Insights

• IV. Contributors
  • 4.1. Contributors

I. News

1.1 Recent Version Releases:

The current version of this repository is b4991

1.2 Version Information:

The version released on March 29, 2025, introduces key updates that enhance overall performance and stability, with notable improvements in user interface responsiveness and security features. These changes reflect a continued focus on optimizing user experience and safeguarding data.

II. Issues

2.1 Top 5 Active Issues:

We consider active issues to be issues that that have been commented on most frequently within the last week. Bot comments are omitted.

1. [BUG-UNCONFIRMED] Eval bug: Gemma 4 tg128 performance on RTX 5060 Ti (Blackwell) appears abnormally low compared to other architectures. Is this expected?: This issue reports abnormally low token generation performance for Gemma 4 models on an NVIDIA RTX 5060 Ti (Blackwell) GPU compared to other architectures like Qwen3.6, despite similar hardware and CUDA backend usage. The user observes high CPU utilization and low GPU utilization during generation, with attempts to force full GPU offload resulting in even slower speeds, and seeks to understand if this behavior is expected or indicative of a CUDA or model-specific performance problem.

   • The comments clarify that Gemma 4 is a dense model using all parameters per token, unlike the Mixture-of-Experts Qwen3.6 which uses fewer active parameters, explaining the speed difference; users confirm that high CPU and low GPU utilization is expected for large dense models on limited VRAM GPUs, and suggest trying smaller or MoE variants to improve performance, while also noting that the issue is not specific to Windows or Blackwell GPUs.
   • Number of comments this week: 13

2. [BUG] Eval bug: deepseek4 produces garbled output when parallel processing is enabled and speculation is in use: This issue describes a bug in the DeepSeek4 model where enabling both parallel processing with more than one slot and speculative decoding causes the model to produce garbled output over time. The root cause is identified as a flaw in the rollback restore logic within the compressed key-value cache handling, which corrupts long-range attention states when sequences are split unevenly across ubatches during parallel speculative decoding.

   • The comments confirm the issue is reproducible and discuss attempts to patch it, with one proposed fix causing model loading failures and another reducing but not eliminating the gibberish output. Further analysis reveals the bug stems from how rollback snapshots are applied across ubatches, leading to corrupted cache states, and ongoing efforts aim to refine the fix without introducing regressions or performance penalties.
   • Number of comments this week: 11

3. [BUG-UNCONFIRMED] Misc. bug: ROCm-7.14 - > 'error while loading shared libraries: libhipblas.so.3': This issue reports a problem encountered when building and running llama.cpp on ROCm 7.14, where binaries fail to load the shared library libhipblas.so.3 unless the LD_LIBRARY_PATH environment variable is manually set to include the ROCm library directory. The discussion reveals that this is due to ROCm 7.14 moving shared libraries without updating the system dynamic loader configuration, leading to proposed fixes either by modifying the build system to embed the library path or by addressing the root cause in the ROCm packaging itself.

   • Comments confirm the library loading failure and share workarounds involving setting LD_LIBRARY_PATH; a patch was proposed to add RUNPATH to the build but was later closed in favor of fixing the issue in the ROCm packaging; further discussion highlights the trade-offs between application-side fixes and system-level solutions, with ongoing debate about the best approach to provide a seamless user experience.
   • Number of comments this week: 6

4. Misc. bug: /slots save/restore silently loses all prompt reuse on hybrid/recurrent models — checkpoints are never persisted: This issue reports a bug in the llama-server where saving and restoring prompt slots to disk silently fails to preserve prompt reuse on hybrid/recurrent models, causing the system to recompute the entire prompt despite reporting a successful restore. The root cause is that the on-disk save format does not include necessary context checkpoints, which are cleared on restore, making the restored state unusable for caching and leading to full reprocessing of tokens on subsequent requests.

   • Commenters confirmed the issue and shared a patch implementing the proposed fix, which involves saving and restoring context checkpoints in a sidecar file; initial tests showed the patch working on some models. Further discussion clarified that the patch fixes the problem when correctly applied, and users shared logs and experiments confirming the bug and the fix’s effectiveness, while also noting related issues with checkpoint creation depending on model templates and recent changes.
   • Number of comments this week: 5

5. Eval bug: llama-server hard crash (cublasSgemm INVALID_VALUE) with --spec-type draft-mtp under KV-cache saturation: This issue reports a hard crash in the llama-server when running with the --spec-type draft-mtp option under KV-cache saturation, caused by a CUDA error (cublasSgemm INVALID_VALUE) that occurs despite all cuBLAS parameters being valid, indicating memory corruption rather than a configuration or context size problem. Extensive investigation traced the root cause to corruption of the CUDA stream pointer inside the cuBLAS handle due to unbounded CUDA graph cache growth in the MTP draft context, which leads to a GPU memory leak and ultimately causes the crash; disabling CUDA graph capture (GGML_CUDA_DISABLE_GRAPHS=1) prevents both the leak and the crash, serving as a practical workaround.

   • The comment discussion details a series of diagnostic tests that ruled out graph reuse and unsynchronized async reads as causes, identified the corrupted CUDA stream pointer as the root cause, and confirmed that disabling CUDA graph capture eliminates the memory leak and crash, with the crash consistently triggered by the first cuBLAS call after context-exceeded events under heavy parallel load in the MTP dual-context setup.
   • Number of comments this week: 5

2.2 Top 5 Stale Issues:

We consider stale issues to be issues that has had no activity within the last 30 days. The team should work together to get these issues resolved and closed as soon as possible.

As of our latest update, there are no stale issues for the project this week.

2.3 Open Issues

This section lists, groups, and then summarizes issues that were created within the last week in the repository.

Issues Opened This Week: 66

Summarized Issues:

• Tool call handling and parsing issues: Multiple issues describe problems with tool call behavior including silent fallback to default tools without warnings, failure to trigger XML tool calls on large prompts, parsing errors with Python code containing quotes, and grammar stack crashes due to deeply nested or improperly matched grammar causing server crashes or denial-of-service vulnerabilities. These issues highlight the need for stricter validation, better error reporting, and improved parsing logic to ensure reliable and secure tool call execution.
• issues/26530, issues/26535, issues/26658, issues/26600, issues/26787

• Backend GPU memory and kernel errors: Several issues report CUDA and HIP backend crashes and illegal memory accesses during evaluation, decoding, or graph replay, including out-of-bounds reads, invalid kernel launch configurations, and memory corruption caused by unbounded CUDA graph cache growth or kernel dispatch limits. These problems cause server aborts, GPU memory leaks, and require workarounds such as disabling CUDA graph capture or manual patches to maintain stability.
• issues/26546, issues/26558, issues/26738, issues/26746, issues/26782

• Performance regressions and inefficiencies on specific hardware: Multiple reports describe significant slowdowns or suboptimal performance on various GPUs and CPUs, including AMD Radeon GPUs with Vulkan backend, Intel Arc Pro B70 GPU memory latency issues, NVIDIA RTX 5060 Ti low GPU utilization, and Apple Silicon CPU backend silently limiting kernel dispatch. These issues often involve fallback to slower primitives, inefficient shader paths, or hardware-specific limitations causing degraded token generation throughput and increased latency.
• issues/26547, issues/26559, issues/26581, issues/26663, issues/26674, issues/26795

• Model loading, state restoration, and quantization bugs: Issues include failure to call encoder functions for encoder-decoder models causing crashes, silent desynchronization of KV cache state in multi-slot usage, fallback quantization warnings without fail-fast options, and requests for additional quantization format support. These bugs affect model correctness, stability, and user control over quantization behavior during loading and inference.
• issues/26565, issues/26616, issues/26777, issues/26587

• RPC and multi-node inference stability problems: Several issues describe crashes and invalid data pointer errors during multi-worker or multi-node RPC setups, including graph computation failures due to missing tensor buffers on workers and silent rpc-server crashes on relayed TCP connections. These problems prevent reliable distributed inference and cross-host RPC usage.
• issues/26583, issues/26680, [issues/26820](https://github.com/issues/26820]

• Vulkan and ROCm backend compatibility and correctness issues: Reports include garbled output due to mismatched fused operations between CUDA and Vulkan RPC nodes, flash-attention assert crashes on AMD GPUs requiring manual patches, and character substitution corruption on AMD GPUs affecting all tested models. These issues indicate incomplete or unstable support for Vulkan and ROCm backends impacting output correctness and stability.
• issues/26685, issues/26702, [issues/26754](https://github.com/issues/26754]

• Server crashes and stability vulnerabilities: Issues include crashes caused by deeply nested grammar recursion, assertion failures in sampler code on long contexts, heap-buffer-overflow vulnerabilities in RPC backend, and silent crashes on port conflicts or malformed requests. These stability problems expose the server to denial-of-service attacks and unexpected failures during normal or edge-case usage.
• issues/26600, issues/26648, issues/26825, [issues/26822](https://github.com/issues/26822]

• Feature requests for model and format support: Requests include adding support for new models like Ling-3.0-flash and Laguna S 2.1 DFlash, convrot support for int8 and fp4 quantization in GGUF format, and end-to-end CPU-only support for the Maple architecture MoE model. These enhancements aim to broaden the range of supported models and quantization formats for improved usability.
• issues/26590, issues/26669, issues/26587, [issues/26766](https://github.com/issues/26766]

• WebUI and user interface improvements: Suggestions include implementing micro-batching for streaming token updates to reduce DOM updates and resource usage, and adding a full-width chat layout option to improve readability of source code in chat bubbles. These improvements target better responsiveness and user experience in the WebUI.
• issues/26540, [issues/26721](https://github.com/issues/26721]

• Build and compilation issues: Problems include undefined CUDA identifiers causing build failures on CUDA 11.8, missing IEEE fp16 support on ARMv7a NEON builds, and CPU feature detection failures causing zero kernel compilation on Cortex-X925 CPUs. These issues hinder successful builds on certain platforms or configurations.
• issues/26591, issues/26677, [issues/26630](https://github.com/issues/26630]

• Token generation and decoding bugs: Issues include endless looping or malformed outputs in vision models during greedy decoding, garbled output due to rollback restore bugs in DeepSeek4 speculative decoding, and nondeterministic tool-calling flips at temperature 0 causing inconsistent outputs. These bugs affect output quality and reliability during generation.
• issues/26652, issues/26741, [issues/26817](https://github.com/issues/26817]

• Memory usage and VRAM optimization requests: Users report increasing VRAM usage during prompt processing and token generation in SYCL backend limiting batch sizes, and request host-pinned memory usage to reduce costly synchronous page-pinning and improve performance. These requests aim to optimize memory management for better scalability.
• issues/26747, [issues/26752](https://github.com/issues/26752]

• Security and supply-chain concerns: A trojan infection detected in a Windows DLL file causes installation errors, and a feature request proposes adding cooldown periods for pip package dependencies to mitigate supply-chain attacks by delaying installation of newly published packages. These issues highlight security risks in software distribution and dependency management.
• issues/26739, [issues/26850](https://github.com/issues/26850]

• Miscellaneous bugs and requests: Other issues include silent waiting on port conflicts instead of failing, requests for fail-fast flags in quantization tools, and workarounds to bypass hardware AI-throttling on specific NVIDIA GPUs by emulating instructions. These reflect various usability and performance edge cases encountered by users.
• issues/26616, issues/26810, issues/26822

2.4 Closed Issues

This section lists, groups, and then summarizes issues that were closed within the last week in the repository. This section also links the associated pull requests if applicable.

Issues Closed This Week: 86

Summarized Issues:

• Model conversion and architecture support issues: Several issues report failures or errors when converting or loading models due to unsupported architectures or missing features in conversion tools. These include requests for adding support for specific models like jinaai/jina-reranker-v3, BAAI/bge-reasoner-embed-qwen3-8b-0923, Meta Muse Glimmer 30B, and jina embeddings v5 omni, as well as bugs in conversion scripts causing assertion failures or crashes.
  • issues/16425, issues/16427, issues/24101, issues/24955, issues/26844

• Model loading and runtime crashes related to specific models or hardware: Multiple issues describe crashes or failures during model loading or inference caused by incompatibilities in model implementations, GPU backend bugs, or hardware-specific problems. Examples include crashes with DeepSeek-V4-Flash, Qwen 3.6 27B, PaliGemma-3B, and issues on AMD RDNA4 GPUs with ROCm or Vulkan backends, as well as Nvidia Blackwell GPU driver bugs.
  • issues/22746, issues/22927, issues/23311, issues/23385, issues/24672, issues/24961, issues/26521, issues/26537, issues/26576, issues/26588, issues/26768

• Router and request handling bugs causing premature aborts or interruptions: Issues report that the llama-server router unloads models or aborts ongoing requests prematurely when the maximum number of loaded models is reached or when switching models, disrupting long-running tasks. There are also requests for priority scheduling to prevent mid-request model swaps and improve request queuing.
  • issues/21678, issues/22560, issues/24849

• Backend and GPU support problems including compilation and runtime errors: Several issues describe compilation failures, runtime crashes, or incorrect behavior related to GPU backends such as ROCm/HIP, CUDA, Vulkan, and SYCL. Problems include conflicts in HIP runtime wrappers, driver bugs on Nvidia GPUs, ROCm device kernel errors, and performance regressions on Intel Arc GPUs.
  • issues/22570, issues/23209, issues/23382, issues/24858, issues/24964, issues/26119, issues/26396, issues/26413, issues/26655, [issues/26679](https://github.com/issues/26679]

• Memory management and multi-GPU model splitting issues: Problems include models failing to load on secondary GPUs, unexpected model splitting despite flags to prevent it, GPU memory faults caused by asynchronous copies, and requests for features to allocate KV-cache and MoE experts on separate GPUs or RAM to optimize hardware usage.
  • issues/23855, issues/24790, issues/26828, [issues/26831](https://github.com/issues/26831]

• Tokenization, embedding, and tool call parsing bugs: Issues report malformed tokenizer metadata causing crashes, silent failures returning zero embeddings when causal language models are served on embedding endpoints, and parsing errors in tool call arguments leading to corrupted inputs.
  • issues/23183, issues/23509, issues/24885, [issues/26763](https://github.com/issues/26763]

• User interface and Web UI problems: The Web UI is broken on Arch Linux causing 404 errors and non-functional interfaces, and bugs prevent image pasting when opening vision models via URL parameters. There are also requests for UI improvements such as live preview of shell command output and displaying default quantization types.
  • issues/23379, issues/24336, issues/24993, issues/25020, [issues/26569](https://github.com/issues/26569]

• Security concerns and feature requests: There are requests to add encryption for IndexedDB and localStorage data, warnings about remote code execution risks when enabling certain tools without API keys, and proposals for safer shell command execution tools to avoid command injection vulnerabilities.
  • issues/24654, issues/26639, [issues/26183](https://github.com/issues/26183]

• Performance regressions and numerical stability issues: Some issues describe significant slowdowns in prompt prefill and decode speeds on specific hardware due to recent changes, as well as numerical instability in standard deviation calculations causing inaccurate results.
  • issues/24951, issues/26396, issues/26413, [issues/26664](https://github.com/issues/26664]

• Model-specific bugs causing incorrect outputs or assertion failures: Bugs include LongRoPE models incorrectly encoding short sequences, DeepSeek-V4 CUDA flash attention producing corrupted output, and Qwen3-TTS repeating phrases and failing to stop correctly.
  • issues/24823, issues/26423, issues/26509, [issues/26699](https://github.com/issues/26699], issues/26700

• Build and compatibility issues: Problems include pre-built binaries crashing on macOS 15 due to missing symbols, container image build failures for ROCm/HIP, and incompatibilities between transformers library versions causing crashes during model conversion.
  • issues/26370, issues/24964, [issues/26735](https://github.com/issues/26735]

• API and server errors: Issues report HTTP 500 errors caused by malformed requests or grammar compilation failures, missing file error messages without details, and server crashes triggered by specific request parameters or model configurations.
  • issues/22948, issues/24279, issues/26737, [issues/26596](https://github.com/issues/26596]

• Miscellaneous bugs and feature requests: These include assertion failures in recurrent model rollback, CPU affinity mask not applying on Android, sidebar display inconsistencies, and requests for hybrid parallelism modes combining tensor and pipeline parallelism.
  • issues/26695, issues/26836, issues/24993, [issues/23568](https://github.com/issues/23568]

2.5 Issue Discussion Insights

This section will analyze the tone and sentiment of discussions within this project's open and closed issues that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.

Based on our analysis, there are no instances of toxic discussions in the project's open or closed issues from the past week.

III. Pull Requests

3.1 Open Pull Requests

This section provides a summary of pull requests that were opened in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.

Pull Requests Opened This Week: 99

Key Open Pull Requests

1. feat: MoE QLoRA training, QAT, quantized AdamW, Critical token SFT, and expert pruning: This pull request consolidates multiple advanced training and optimization features for Mixture-of-Experts (MoE) models, including QLoRA fine-tuning with backward pass improvements, quantization-aware training (QAT) with various quantized formats, new quantized AdamW optimizers, Critical token selective fine-tuning (SFT), enhanced learning rate scheduling with warmup, multithreaded data pipeline improvements, experimental ROCm support, and both soft and experimental hard pruning tools for MoE expert calibration and model size reduction, targeting architectures like Gemma 4 26B A4B and supporting models such as Mixtral and Qwen-MoE.

• URL: pull/26794

• Associated Commits: b290b, 46bfd, 193af, 6e1f8, 0025a, 14370, d2cce, 602cb, 2fc78, 4132b, 3dcfc, 39fac, 31ad0, 83830, c4c76, b6f48, 7d01d, bedd7, f8f49, ffd68, bd780, e3dcf, d828c, a4a73, 79efb, 3c481, 2b229, c4fc6, 8cc16, e6199, bea0b

2. sycl: fix multiple warnings in compiling sycl backend : This pull request fixes multiple compiler warnings in the SYCL backend of the project by cleaning up issues related to the icx/icpx compiler on Linux with various SYCL macro configurations and addresses some Windows-specific warnings, aiming to maintain a clean build environment for future development while using the 2026.1 oneAPI release and SYCL2020 standard features.

• URL: pull/26713

• Associated Commits: b1c8e, 2ce8d, 801ad, 4e808, e0ac2, 61908, 77522, f5854, 8946b, 9c04c, 8dd2c, 7a619, c748a, 2d984, dc66d

3. ggml-metal: add chunked SSD MMA for Mamba-2 prefill optimization: This pull request introduces a chunked SSD matrix multiplication path optimized for Metal devices using Mamba-2 style prefill, enabling parallel SIMD group matmuls for 64-token chunks to improve performance in batched token sequence processing while falling back to sequential scans for remaining tokens, resulting in up to approximately 12% throughput gains without affecting perplexity.

• URL: pull/26647

• Associated Commits: 21da5, 8597a, adfad, 7b8f0, 81825, 7563b, 1395e, 58b37, 3ae11, 08e42, 2f4fe

Other Open Pull Requests

• Metal backend improvements: Multiple pull requests enhance the Metal backend by splitting the FA-vector tuning into a standalone tool for better tuning workflows and implementing a per-operation source code split with parallel compilation to improve modularity and compilation efficiency. These changes streamline development and optimize performance on Metal devices.
  • pull/26620, pull/26561

• CUDA and GPU performance optimizations: Several pull requests introduce experimental CUDA SM120 CUTLASS-based MoE prefill paths, optimize HIP Q2_0 dot-product paths for AMD GPUs, and implement a static per-device workspace buffer for cuBLAS handles to improve stability and performance. These updates provide significant speedups and stability improvements across CUDA and HIP GPU backends.
  • pull/26704, pull/26753, pull/26574, pull/26592

• HIP backend enhancements: The HIP backend is improved by enabling CUB-based ARGSORT and TOP_K GPU sorting paths on HIP devices, adding missing stream-capture functions, and optimizing the dot-product path for gfx1201 GPUs using native AMD permutation instructions. These changes enhance performance and support for AMD GPUs.
  • pull/26592, pull/26753

• Server and API improvements: New server features include a POST /tts endpoint for text-to-speech with streaming audio and speaker references, comprehensive support for saving and restoring multimodal slots with media inputs, and improved error classification for mid-decode context exhaustion errors to better distinguish client and server issues. These updates enhance server functionality and robustness.
  • pull/26603, pull/26640, pull/26848

• Model and decoding fixes: Fixes address rollback bugs in DeepseekV4 and nemotron models by improving state management and adding test cases, while also resolving image sequence assembly issues in Granite4 Vision models by correcting tile interleaving and embedding logic. These fixes improve model correctness and decoding reliability.
  • pull/26756, pull/26623, pull/26653

• Vulkan backend optimizations: The Vulkan backend receives multiple enhancements including a tiled shared-memory transpose shader for faster DeepSeek-V4 indexing, Intel-specific kernel optimizations for ARC B70 Pro platform with flash attention and GEMM improvements, and support for large row counts in specialized f16 mat-vec shader paths to maintain performance on large contexts. These changes significantly boost Vulkan performance on various hardware.
  • pull/26585, pull/26574, pull/26595

• Quantization and model support: Pull requests add NVFP4 quantization support to the llama-quantize tool and enable BailingMoE 3 with Ling 3.0 flash model support including speculative decoding and safe gate metadata. These updates expand quantization options and model compatibility.
  • pull/26556, pull/26608

• Build and CI updates: Continuous integration improvements include fixing WebGPU backend errors by adjusting test paths and adding i32 support, updating Windows CI to use the official LLVM OpenMP release instead of a non-redistributable Microsoft version, and introducing semantic versioning via CMake configuration. These changes improve build reliability and version management.
  • pull/26566, pull/26678, pull/26839

• Miscellaneous fixes and enhancements: Additional updates include fixing sliding window attention activation for the EXAONE 4.5 model to prevent excessive memory use, updating gating logic for OneDNN features on Alchemist GPUs to improve prompt processing, adding a modular UI fix to mark API Key fields as private to prevent autofill, and implementing asynchronous graph computation and custom all-reduce for RDMA-connected dual Spark devices. These diverse improvements address stability, security, and distributed computation.
  • pull/26829, pull/26635, pull/26562, pull/26610

3.2 Closed Pull Requests

This section provides a summary of pull requests that were closed in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.

Pull Requests Closed This Week: 162

Key Closed Pull Requests

1. Feat/docs bootstrap: This pull request introduces foundational documentation and initial setup for the Tessera project’s bootstrap website, including eight core pages, CSS and JS assets for theming and layout, site configuration files, a custom domain setup, and a GitHub Pages deployment workflow, establishing a self-hosted, third-party-free, static site framework to present the project’s editorial content and changelog.

• URL: pull/26662

• Associated Commits: 07d36, 9d4ce, a9798, 9d43d, 91c89, 1beba, 3a25a, 292be, 8d97d, 8f364, b7442, 99bd0, 1c2d2, 204e9, ab68d, 47d59, 69124, 8646c, cae1f, 0b880, 7d315, 8741c, 1fd99, 9fc91, d3dad, d4f3b, 52870, a7055, 1dca5, 026db, f7997, 88b4c, 70779, 9d1d9, 41270, 94904, 8f8ff, 2bffb, db559, c1422, f1aee, 23433, ca686, 738be, c64e9, 8a9cb, 040de, e794a, fc876, bd656, c2a31, a05f2, 6e4b6, 883a9, 6e598, 89d30, 9e60f, 1901c, 01c41, f31f5, 471a2, 805f8, 1478f, e0645, 0a4bc, 269aa, d1cac, ae38f, def71, b9d49, 8568d, f5f7a, 98c82, 1e9cc, 4a0d7, d7f46, 3aa5f, 2a5ba, 41a11, fe2c3, 2e865, 1dfbe, 5772d, 716b4, e6f42, 69f01, 9fdcb, 08908, 5b4e2, afe69, 90397, d4d99, 8b369, 965ce, a738d, dafc7, bec03, 55d7e, 5d2f1, 5402d, 18b53, 81085, 3750c, a4fc2, d4678, 4bdc7, 1d8b9, 2a8c9, 1f8c9, 0090f, 0e59d, bc95d, c4b85, 5d2b0, 1e34b, 4ebec, 0dc51, 156f3, 7facb, b28d3, 8fe05, 5357a, 703df, c3dd8, 08696, dca43, fba0a, a259a, 86c63, ed5a0, 64b08, d37db, 9065d, 5c7db, 0384b, 5c254, 6112f, b50df, b8cd3, 1036f, a5811, fb213, 622d3, 1f7fc, 1d41f, 651c1, 01c83, b6d24, 3ba0c, bbdd8, ab875, 082f9, 913da, d3356, ead64, cd127, dada1, e21b5, 2aece, 5b55d, fff16, 45960, bbb8d, cae81, 7e5b0, 41a36, 48f03, f99bd, c4a04, 92611, 78f60, eaa86, f6275, 56659, f1e5f, 02e6a, 12796, a6322, 4f703, d73da, d9845, 03b5e, da6a6, 86266, 30dc2, d4ef9, 850bd, bc150, 425d2, 976f6, e5363, 95f6f, 756f0, f5d42, d0732, 92c9c, 365d8, 6f060, 40707, 320fb, 5b2ff, 41017, da307, 3a3d2, aa4b1, 79762, b79cb, f3248, ee32e, a08d7, 2b808, 2fe7f, 8a3f4, 4f278, 7f5bc, 2a6da, 217c9, 3377f, ecbdc, 64ed0, 38b81, 3e8f2, 24ef3, fc6b7, e7426, d1698, dc90b, 6e4d9, 994f9, ac7b3, 2d60a, 8e1d0, efcb7, 740ee, 50fa0, 2336b, 3d52b, 3932d, f16ca, c5afb, c45b9, a8d76, 6b6c9, 329db, ed0b6, 2e12d, dff51, 6a8e5, 232aa, 3db34

2. Granite-Switch Architecture: This pull request introduces the "granite-switch" architecture to llama.cpp, enabling support for a dense Granite-4.1 model with multiple embedded LoRA adapters that are dynamically selected per token via control tokens, including full conversion tooling, C++ backend integration, and a refined in-graph router attention mechanism to handle per-token adapter switching with concurrency and multi-turn considerations.

• URL: pull/25107

• Associated Commits: 89223, 7633e, 80a47, 30964, b4eb4, 5b4fd, f51c1, 8da8d, 205e1, 723db, a8ee7, 4e9ac, 42346, c1874, 3ad60, 3fdc1, 8ded2, 846e6, e5b15, 37a3f, 0173b, 9b6cb, 4a54c, 46753, ea397, 600ef, 83798, 010f2, 48fc2, 1870e, c9a28, dbfd7, 17ad1, 9fc68, fc5e0, 0cd43, f769d, aac9a, 1a0ac, f57f9, d9572, 6e734, af48d, 16c9a, 30ffe, 0ac7c, b17c0, 76263, e193e, 2b179, 3041b, 8c353, 36936, dba81, 26f28, 116c6, 6be5d, 05940, 8d71a, afa1b, 5b2cb, d385f, 768e6, 03e46, 24e06, 613bc, 472d1, c6014, 622eb, ad04c, 1d15e, e99c0, 46d88, d9929, c57b0, d2ca9, 11890, ae2ae, d8860, 22048, 56698, a7271, 64278, 34f56, 7cc8b, 118cf, 65cb2, 15e47

3. Expert Caching that can boost performance, heatmap, mmap pinning based on heatmap, real time cpu to device transfer and device to cpu transfer. Fully conditional on flags and self contained to not affect execution path outside of them: This pull request introduces an expert caching system with configurable heatmap-driven GPU memory management, including mmap pinning, real-time CPU-to-device and device-to-CPU expert tensor transfers, and adaptive expert slot allocation fully controlled by flags to optimize performance without affecting the default execution path.

• URL: pull/26824

• Associated Commits: fcaac, 61080, 21537, b27d2, 413e6, 3fd57, 23ff8, 289c4, 28421, 8e955, 08e65, 1aefb, cee17, a3c44, 90409, fb0b0, 8d1a6, 3c961, d1cdf, 8687e, 6e757, 5c502, 1a850, b18c4, f201e, ca624, fac40, 0b768, 7633d, 25558, 01705, 336be, 91916, d21fc, e2686, 08562, d3cba, 4ce95, 38976, d7345, 581cb, e0efc, e3004, 09b1e, 39641, 20f15, 6b752, cc0de, 8e3e8, d2949, ab875, 1550f, ef1af, a435e, 27a4e, bfe48, 8e340, be76d, 5097d, 7c08c, 75afc, f2b29, 81335, efa09, 5e5a4, 48f10, ea63c, b0baf, 86bef, 11855, 44010

Other Closed Pull Requests

3.3 Pull Request Discussion Insights

This section will analyze the tone and sentiment of discussions within this project's open and closed pull requests that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.

Based on our analysis, there are no instances of toxic discussions in the project's open or closed pull requests from the past week.

IV. Contributors

4.1 Contributors

Active Contributors:

We consider an active contributor in this project to be any contributor who has made at least 1 commit, opened at least 1 issue, created at least 1 pull request, or made more than 2 comments in the last month.

If there are more than 10 active contributors, the list is truncated to the top 10 based on contribution metrics for better clarity.