Weekly GitHub Report for Llama.cpp: July 20, 2026 - July 27, 2026 (22:04:15)
Weekly GitHub Report for Llama.cpp
Thank you for subscribing to our weekly newsletter! Each week, we deliver a comprehensive summary of your GitHub project's latest activity right to your inbox, including an overview of your project's issues, pull requests, contributors, and commit activity.
Table of Contents
I. News
1.1 Recent Version Releases:
The current version of this repository is b4991
1.2 Version Information:
The version released on March 29, 2025, introduces key updates that enhance overall performance and stability, with notable improvements in user interface responsiveness and security features. These changes reflect a continued focus on optimizing user experience and safeguarding data.
II. Issues
2.1 Top 5 Active Issues:
We consider active issues to be issues that that have been commented on most frequently within the last week. Bot comments are omitted.
-
[BUG-UNCONFIRMED] Misc. bug: SYCL: bad performance on newer oneAPI: This issue reports a significant performance regression when using newer versions of oneAPI with the SYCL backend in llama.cpp, negating previous performance improvements from a recent pull request. The user describes attempts to diagnose the problem, including rolling back dependencies and checking oneDNN logs, but the issue persists, with additional comments noting instability and degraded performance in related environments.
- Commenters discuss various troubleshooting steps such as rolling back Intel GPU compiler versions, enabling verbose logging for oneDNN operations, and testing different system configurations; ultimately, no definitive cause is identified, and some note instability and performance drops even with workarounds.
- Number of comments this week: 9
-
Eval bug: server -np 4 --kv-unified returns other requests' responses verbatim on integrated HIP GPU (gfx1151) — bisected to c7d87229: This issue reports a critical bug in
llama-serverrunning on integrated AMD HIP GPUs (gfx1151) under parallel mixed load with-np 4 --kv-unified, where responses from one request are incorrectly returned verbatim for different, concurrent requests, including chimeric outputs combining tokens from multiple requests. The problem was bisected to a specific commit that re-enabled integrated GPU support for HIP, causing cross-request response contamination that does not occur on discrete GPUs or CPU builds, and a temporary revert of this commit fixes the issue without throughput loss.- Commenters confirmed the bug on various integrated AMD GPUs and shared similar symptoms including garbled output and wrong tool calls; several users found switching to Vulkan or reverting the problematic commit as effective workarounds, and a fix is in progress with a pull request awaiting review.
- Number of comments this week: 7
-
[BUG-UNCONFIRMED] Eval bug: Warning Message - sched_reserve: layer 0 is assigned to device CPU but the fused Gated Delta Net tensor is assigned to device CUDA0 (usually due to missing support): This issue reports a warning message indicating that layer 0 of the model is assigned to the CPU while the fused Gated Delta Net tensor is assigned to the GPU, which leads to significantly slower performance when running the Qwen3.5-9B-Coder model with CUDA on a Windows system. The user suspects this is due to missing support in the release build despite using the latest version and CUDA 13.3, and the discussion explores context size limitations, VRAM usage, and potential code patches to resolve the warning and improve generation speed.
- The comments focus on troubleshooting the warning by adjusting the context window size, clarifying VRAM requirements beyond model size, sharing a code patch that fixes the warning and improves speed, and discussing hardware-specific performance issues and configuration tips to optimize GPU usage and avoid fallback to CPU.
- Number of comments this week: 6
-
[BUG-UNCONFIRMED] Compile bug: Build failure: CoopMat2 shaders fail with glslc -O / SPIR-V capability 5432: This issue describes a build failure when compiling CoopMat2 shaders with Vulkan enabled on a Linux system using an AMD Radeon RX 580 GPU, where the shader compiler (glslc) fails during optimization due to an invalid SPIR-V capability operand error (5432). The problem appears related to the optimization flags applied during shader compilation, and a workaround involving disabling optimization for CoopMat2 shaders allows the build to complete successfully.
- The comments suggest updating the glslc compiler as a potential solution, provide detailed environment information, share a local patch to avoid the failure, and note that the issue is resolved with the latest spirv-tools version, though the user plans to carefully manage updates due to their source-built system.
- Number of comments this week: 6
-
[BUG-UNCONFIRMED] Eval bug: [SYCL] Garbled output on 2+ GPUs: This issue reports a consistent problem where running the SYCL backend on two or more Intel B70 GPUs results in garbled output, while running on a single GPU or using the Vulkan backend works correctly. The user traced the problem to the
dev2dev_memcpyfunction inggml-sycl.cpp, suspecting synchronization issues during device-to-device memory copies, and found that a patched version of this function works but with poor performance.- The comments focused on gathering system information and environment variable settings to reproduce the issue, confirming it is a known problem with Intel driver version 26.xx, and discussing the internal implementation of
dev2dev_memcpywith plans to add environment variable support to enable a fallback method that currently cannot be directly activated by users. - Number of comments this week: 5
- The comments focused on gathering system information and environment variable settings to reproduce the issue, confirming it is a known problem with Intel driver version 26.xx, and discussing the internal implementation of
2.2 Top 5 Stale Issues:
We consider stale issues to be issues that has had no activity within the last 30 days. The team should work together to get these issues resolved and closed as soon as possible.
As of our latest update, there are no stale issues for the project this week.
2.3 Open Issues
This section lists, groups, and then summarizes issues that were created within the last week in the repository.
Issues Opened This Week: 61
Summarized Issues:
- Memory leaks and high memory pressure on GPU backends: Several issues report memory leaks or excessive memory usage on different GPU backends, including a macOS Metal memory leak when freeing models without prior GPU prompts, a suspected severe memory leak in llama-server after model switching, and excessive compute buffer reservation on ROCm HIP backend reducing usable context size. These problems cause high memory pressure, rapid RAM consumption, or drastically reduced available tokens, impacting stability and performance.
- issues/25937, issues/25953, issues/26038
- Server crashes due to unbounded recursion and parsing errors: Multiple issues describe server crashes caused by infinite recursion in RPC node creation and chat_parser JSON, unconditional aborts triggered by disabled features, and grammar parsing failures due to duplicate rules or complex chat templates. These lead to stack overflows, denial-of-service vulnerabilities, and unusable server responses, severely affecting server reliability.
- issues/25958, issues/25959, issues/25960, issues/25967, issues/25986
- GPU backend recognition and compatibility issues: Problems with GPU hardware recognition and backend compatibility are reported, including AMD GPU hardware not recognized when forcing Vulkan on ROCm HIP backend, missing Windows HIP support for gfx1152 GPUs causing invalid device function errors, and Vulkan backend failures on specific AMD GPUs due to shader compilation errors. These issues prevent proper GPU utilization and model loading.
- issues/25944, issues/25985, issues/26127
- Performance regressions and bottlenecks in various backends: Several issues highlight performance regressions or bottlenecks, such as SYCL backend regressions with newer oneAPI versions, token generation slowdowns on SYCL with Intel Arc GPUs, and compute-bound decode operations on Arm Neoverse-N2 CPUs. These degrade throughput and negate previous improvements, prompting requests for optimization efforts.
- issues/25973, issues/26010, issues/25976, issues/25977
- Crashes and failures in model loading and runtime on specific hardware and configurations: Issues report crashes during model load or runtime, including Gemma 4e4b model crashes on Windows CUDA backends, Gemma4-31B multi-GPU crashes with ISWA cache options, and CUDA cublasSgemm_v2 failures during speculative decoding on V100 GPUs. These failures cause aborts and prevent successful model execution.
- issues/26017, issues/26122, [issues/26119](https://github.com/issues/26119]
- Incorrect or corrupted output from concurrent or multi-GPU usage: Concurrency bugs cause corrupted or chimeric outputs in llama-server on AMD HIP GPUs under parallel load, and SYCL backend on multiple Intel GPUs produces garbled output due to synchronization issues. These bugs lead to invalid or mixed responses, undermining output correctness.
- issues/25992, [issues/26058](https://github.com/issues/26058]
- Bugs and crashes related to RPC backend and tensor operations: Crashes occur in RPC backend usage due to missing memset_tensor implementation, serialization failures of remote KV states, and excessive RPC node cache growth causing disk usage issues. These problems affect distributed inference stability and resource management.
- issues/26128, issues/26143, [issues/26151](https://github.com/issues/26151]
- Model-specific bugs causing hangs, infinite loops, or corrupted embeddings: Certain models exhibit unique issues such as Qwen 3.5 122B hanging on Vulkan backend, Gemma 4 31B entering infinite generation loops after idle, and Qwen3-VL producing NaN embeddings in multi-output decode. These bugs degrade model usability and require restarts or workarounds.
- issues/26087, issues/26088, [issues/26142](https://github.com/issues/26142]
- Requests for new model support and feature enhancements: Multiple issues request adding support for new models like Macaron-V1, Nanbeige, Motif-3-Beta, and Solar-Open2, as well as features like JSON argument for concatenation in completions, next-generation sampling methods, and Gigatoken Tokenizer integration. These aim to expand capabilities and improve user experience.
- issues/26066, issues/26086, issues/26115, issues/26008, issues/26020, issues/26139, [issues/26125](https://github.com/issues/26125]
- Build and compilation issues on various platforms: Problems include Linux HIP compile failures due to environment variable misconfiguration, missing Ubuntu CUDA 13 release binaries, and shader compilation errors on AMD GPUs. These hinder building and deploying the software across environments.
- issues/26060, issues/26019, [issues/25985](https://github.com/issues/25985]
- Security and stability concerns including malware detection and denial-of-service: A Trojan was detected in a Windows build executable, and unconditional aborts triggered by disabled features cause denial-of-service vulnerabilities. These raise concerns about software integrity and robustness.
- issues/26120, [issues/25960](https://github.com/issues/25960]
- API and monitoring improvements requested: There is a request for exposing detailed per-device memory usage metrics via Prometheus and JSON to replace brittle log parsing, and fixing non-deterministic timestamps in the models API to improve client caching. These improvements aim to enhance observability and API reliability.
- issues/26129, [issues/26169](https://github.com/issues/26169]
2.4 Closed Issues
This section lists, groups, and then summarizes issues that were closed within the last week in the repository. This section also links the associated pull requests if applicable.
Issues Closed This Week: 106
Summarized Issues:
- Model Support and Feature Requests: Several issues request adding support for new models or features, such as the Zyphra/Zamba2-2.7B hybrid model, MiMo V2.5 MTP specification, Nemotron-3-ultra MTP support, MiMo V2 Flash model behavior, and Metal backend support for TQ2_0 quantization. These requests aim to improve compatibility, performance, and functionality across various hardware and model types.
- Memory Management and Out-of-Memory Errors: Multiple issues describe memory consumption problems including excessive RAM usage with Gemma 4 models in llama-server, VRAM exhaustion when switching models, meta backend buffer over-allocation, and out-of-memory errors on devices like Jetson AGX Orin. These issues often cause crashes or system hangs and sometimes require workarounds like disabling checkpoints or manual unloading.
- Crashes and Illegal Memory Access: Several reports detail crashes caused by illegal memory access or assertion failures during model loading or inference, including Gemma 4 31B with Flash Attention, Qwen3.6 27B checkpoint restoration, CUDA illegal memory access on RTX 4080 SUPER, and segmentation faults with image prompts in llama4 scout. These crashes often occur under specific hardware or model configurations.
- Performance Regressions and Inefficiencies: Issues report significant slowdowns and inefficiencies such as slower token generation in llama-server compared to llama-cli, GPU power draw being lower than expected on Mac Studio M4 Max, generation speed drops with n-gram modulation, and a 36% throughput drop on AMD RX 9070 XT with large context sizes. These problems affect throughput and resource utilization.
- Backend and Hardware Compatibility Bugs: Various issues describe backend-specific bugs including Vulkan backend crashes and hangs, CUDA kernel exceptions on NVIDIA RTX 5090, Hexagon NPU vectorization bugs, Metal backend offloading failures, and OpenCL 2.0 build failures on Android. These problems limit hardware support or cause instability.
- Model Conversion and Loading Issues: Problems with model conversion scripts and loading include missing tensor exports due to prefix stripping errors, expert-selection bias tensor naming mismatches, and vector subscript errors during model initialization. These issues prevent proper model usage or require patches.
- Server Configuration and Preset Bugs: Issues highlight problems with server configuration files and presets, such as unexpected empty default model entries from INI files, incomplete reproduction of CLI arguments in models.ini, and router mode failing to load models with conflicting keys. These cause confusion and unexpected server behavior.
- issues/22364, issues/23502, [issues/24439](https://github.com/issues/24439]
- Web UI and User Interface Problems: Several issues describe UI bugs including poor visibility of reasoning controls, disappearing assistant replies in long conversations, limited system message editing window, redundant model selector dropdowns, and white screen failures due to missing assets. These degrade user experience and usability.
- Tool Calling and JSON Handling Bugs: Bugs include improper stripping of whitespace in tool call parameters causing JSON mismatches, MiniCPM5-1B failing to return proper tool_calls, and minja template engine errors when tool schemas include properties named "items." These issues affect tool integration and request processing.
- CUDA and GPU Performance and Stability Issues: Reports include CUDA performance regressions causing GPU clock speed throttling, kernel crashes due to memory access violations, and silent fallback to CPU attention when certain CUDA flags are not enabled. These impact stability and performance on NVIDIA GPUs.
- issues/26024, issues/26025, [issues/24485](https://github.com/issues/24485]
- Logging, API, and Protocol Issues: Problems include excessive logging in router mode, incorrect HTTP error responses on invalid chat completion requests, and missing generation settings in API responses. These issues affect debugging, client-server communication, and API usability.
- issues/24245, issues/25605, [issues/25803](https://github.com/issues/25803]
- Build and Compilation Problems: Issues describe build failures due to missing CUDA driver library propagation, compile warnings on Windows with CUDA, and OpenVINO backend incompatibility with dynamic loading. These hinder successful builds and backend functionality.
- Model Output and Generation Bugs: Several issues report generation problems such as infinite loops producing repeated tokens, garbage output from corrupted logits, and failure to produce reasoning output in the WebUI despite CLI success. These affect model correctness and output quality.
- issues/22230, issues/24311, [issues/26037](https://github.com/issues/26037]
- Resource Management and VRAM Usage: Issues include inconsistent VRAM usage reporting across backends, residual VRAM usage after repeated server restarts on ROCm, and VRAM enumeration failures on AMD APUs due to BIOS settings. These cause inaccurate resource tracking and potential performance degradation.
- issues/24159, issues/24330, [issues/26089](https://github.com/issues/26089]
- Security and Access Control Concerns: One issue reports that enabling
--tools allexposes the inference server's filesystem in the WebUI, which can be problematic when server and client run on different machines, raising security and privacy concerns.
2.5 Issue Discussion Insights
This section will analyze the tone and sentiment of discussions within this project's open and closed issues that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.
Based on our analysis, there are no instances of toxic discussions in the project's open or closed issues from the past week.
III. Pull Requests
3.1 Open Pull Requests
This section provides a summary of pull requests that were opened in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.
Pull Requests Opened This Week: 75
Key Open Pull Requests
1. ggml-webgpu: Fix some binding alias issues to support all archs, fix recurrent-state-rollback test: This pull request addresses WebGPU architecture compatibility by fixing binding aliasing issues in GLU and SSM scan, enabling broader test coverage while temporarily disabling deepseek32, and resolving related CI failures including adjustments for zero-sized concat inputs and recurrent-state-rollback test corrections.
- URL: pull/25931
- Associated Commits: 80b52, 2723f, 4e033, ffb30, e20f0, a429b, d4ac7, 36865, b4dda, 66337, ea3cd, c2710, a19e3, 80a5b, dc5a6, 13cad, 22d95, c720f, 40655, c2cdc, 68817, af6c4
2. reasoning-budget: Implementing thinking-budget mechanism to control thought process: This pull request implements an enhanced thinking-budget mechanism for the reasoning-budget sampler by adding opt-in intro and soft warning messages around the existing hard cutoff to help control and structure the model's thought process, preventing repetitive or nonsensical loops by deterministically rationing token usage and gracefully enforcing both soft and hard limits.
- URL: pull/25961
3. CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints: This pull request optimizes CUDA performance by fusing the weight scale multiplication directly into the matrix multiplication quantization (MMQ) epilogue for NVFP4 checkpoints, eliminating intermediate tensor writes and an extra CUDA kernel to improve efficiency in model layers such as Qwen without altering other model components like attention or gating mechanisms.
- URL: pull/26048
Other Open Pull Requests
- Model context and runtime state management: Multiple pull requests enhance model runtime flexibility by enabling server-side methods for reloading model context, mmproj, and speculative configurations without changing weights, and by preserving context checkpoints during slot save and restore operations to avoid full prompt re-processing. These improvements facilitate dynamic reinitialization and efficient state management for SWA and hybrid/recurrent models with backward and forward compatibility.
- Mixture of Experts (MoE) support and optimization: Several pull requests introduce and optimize MoE architectures, including initial support for Sparsetral sparse-adapter MoE with extensive testing, and the
--pin-hotexpertsfeature that pins frequently used experts in RAM to reduce latency spikes and improve throughput. These changes ensure compatibility with existing models and provide configurable memory management and real-time usage tracking for MoE models.
- Performance improvements and backend optimizations: Multiple pull requests focus on performance enhancements, including optimized ESIMD kernels for Qwen3-14B-Q4_K model on B70 and PVC hardware, HIP backend optimizations for RDNA 4 architecture with improved matrix operations and quantization, and a fix for a Metal backend memory leak by scheduling minimal GPU work. These updates deliver significant speedups and stability improvements validated through benchmarking and testing.
- Quantization and compression enhancements: New quantization types QFX16 and QFX32 are introduced, providing lossless compression of neural network weights up to 2.05x without quality loss, along with an opt-in dequant-on-load feature for faster Metal GPU inference. These additions improve model size efficiency and inference speed while maintaining accuracy.
- User interface and tooling improvements: Several pull requests improve the user experience by stabilizing tool invocation rendering in the
llama-serverUI to eliminate flicker and redundant renders, adding initial model management features in the web UI, and refactoring WGSL shader files in ggml-webgpu for better readability and maintainability. These UI-only changes enhance usability without modifying core C++ code.
- Model loading and memory management: Introductions include a new "auto" default load mode that disables mmap on integrated GPUs to prevent doubled memory usage and slow loading, and an unbuffered read file API for Windows to improve model loading performance by reducing system memory pressure. These changes optimize resource usage and loading times across platforms.
- Sampling and decoding methods: A new p-less sampling method is implemented, providing a hyperparameter-free decoding strategy based on dynamically thresholding token probabilities with support for CPU and GPU backends, CLI integration, and comprehensive testing. This method improves numerical stability and decoding performance.
- Build system and configuration enhancements: Updates include a configuration precedence system prioritizing CLI flags over environment variables and config files for flexible per-node settings, and a CMake build fix that prevents failures caused by stale UI assets when certain provisioning options are disabled, with added tests to verify behavior. These changes improve build robustness and configuration management.
- New model and architecture support: Native support is added for the Nanbeige 4.2 model with conversion and runtime updates, and for the Elbrus 2000 (e2k) CPU architecture with specific detection and compatible intrinsics replacements, enabling efficient inference on new hardware and architectures.
- Benchmarking and evaluation tools: Three new code-oriented benchmarks—ClassEval, HumanEval, and multi-turn agentic SACB suite—are added to llama-eval, enhancing evaluation of raw coding skills and agentic coding capabilities in sandboxed environments with updated tooling.
- Flash attention and WGSL improvements: The
flash_attn_vecfunction is improved for quantized key-value caches by applying a direct path approach and WGSL code refactoring, significantly enhancing performance at long context lengths. Additionally, WGSL shader files are refactored to remove unused parameters and improve maintainability.
- Distributed inference and network tools: LlamaNet is added as an open-source distributed inference network with an OpenAI-compatible API for serving and routing large language model inference across connected nodes, expanding the project's tooling ecosystem.
3.2 Closed Pull Requests
This section provides a summary of pull requests that were closed in the repository over the past week. The top three pull requests with the highest number of commits are highlighted as 'key' pull requests. Other pull requests are grouped based on similar characteristics for easier analysis. Up to 25 pull requests are displayed in this section, while any remaining pull requests beyond this limit are omitted for brevity.
Pull Requests Closed This Week: 131
Key Closed Pull Requests
1. P324 quality gate fixes: This pull request titled "P324 quality gate fixes" is focused on improving the correctness, robustness, and gating of the SYCL turbo flash-attention (FA) kernels and related test harnesses, including fixes to kernel dispatch, environment gating, innerq scale handling, capacity validation, and quality gate scripts to ensure stable, portable, and accurate validation of turbo quantization features across multiple models and hardware targets.
- URL: pull/26109
- Associated Commits: f14f0, e5062, 5bbd8, 64ecb, 9e43b, 1c3ea, a8c22, 0e016, 37f99, eb897, d4fb1, 9a3d8, 6a22d, 074db, a9e6c, ff268, edf95, 6a592, 98eb5, d4004, 54331, b4b2d, cb92a, 5d033, cfc85, 3af03, 9e77c, 17147, 058e5, 819d4, b13aa, 1a17f, 4e8a7, 075e8, 8397d, 20210, 9099a, d7ddf, 331d2, 1337c, 60c14, 0a55a, 39774, b4db1, 64a3b, fb227, b6589, a2acb, 5df5e, 9b714, a9b13, 33aff, 656e0, 7968c, 34d43, dcb96, 42ce3, 08ab1, c4848, 66eb7, 52ffc, 615a3, c13e5, 56d54, 23c4b, 57d07, 6a48a, c2452, 999fd, e7b4e, 8d5c5, 41306, 81c70, a3fc3, 3397b, 85b9e, 0075f, f9e4f, ba880, 64061, 85ce1, 24ce9, 03069, 551eb, 3fafb, 1772c, e93b7, 27f39, cd093, 45e87, 533c0, e2d26, 5b867, 0c703, 0e5ec, 3248c, e2a51, 6c623, e9b2e, 7e5a2, 42b30, f999c, e964d, 3c6c5, 8b11e, dcaf7, c3b20, 0c0da, 5b6a0, 11e5e, 1a022, 3fb02, 893ca, cc6dc, c2594, 39104, a8c87, 9d0d0, bd515, 67081, bd350, 70bff, 4e19c, 2b514, 59d91, f04d3, c82b9, 09bab, bbbb1, 2cc99, cc938, bf29a, 7f2a8, 3c778, 255a2, 8aff5, 1c261, 77d35, b5e6e, 9b132, e8d04, 22c41, 1baff, 528bf, 27ab6, 56af9, 39af6, 4f8b0, 57ff6, d01a3, 3ab98, bd59e, 5971b, 950a1, 492b5, b6410, 674ca, 3d1a1, 09004, 4ed65, 81c94, e8770, edf6f, 15bf3, 6e32d, a1456, 65bdb, 65500, aa745, fb421, 5e803, 64f53, 02808, 419ae, 45948, 7be8c, 94b24, 0a9b2, b8278, c95a8, 61eed, d2e8e, 44b72, be1d9, 18186, e7067, ddab7, 66439, 94d05, 19eeb, b289e, 7073b, 0ab4f
- Associated Commits: f14f0, e5062, 5bbd8, 64ecb, 9e43b, 1c3ea, a8c22, 0e016, 37f99, eb897, d4fb1, 9a3d8, 6a22d, 074db, a9e6c, ff268, edf95, 6a592, 98eb5, d4004, 54331, b4b2d, cb92a, 5d033, cfc85, 3af03, 9e77c, 17147, 058e5, 819d4, b13aa, 1a17f, 4e8a7, 075e8, 8397d, 20210, 9099a, d7ddf, 331d2, 1337c, 60c14, 0a55a, 39774, b4db1, 64a3b, fb227, b6589, a2acb, 5df5e, 9b714, a9b13, 33aff, 656e0, 7968c, 34d43, dcb96, 42ce3, 08ab1, c4848, 66eb7, 52ffc, 615a3, c13e5, 56d54, 23c4b, 57d07, 6a48a, c2452, 999fd, e7b4e, 8d5c5, 41306, 81c70, a3fc3, 3397b, 85b9e, 0075f, f9e4f, ba880, 64061, 85ce1, 24ce9, 03069, 551eb, 3fafb, 1772c, e93b7, 27f39, cd093, 45e87, 533c0, e2d26, 5b867, 0c703, 0e5ec, 3248c, e2a51, 6c623, e9b2e, 7e5a2, 42b30, f999c, e964d, 3c6c5, 8b11e, dcaf7, c3b20, 0c0da, 5b6a0, 11e5e, 1a022, 3fb02, 893ca, cc6dc, c2594, 39104, a8c87, 9d0d0, bd515, 67081, bd350, 70bff, 4e19c, 2b514, 59d91, f04d3, c82b9, 09bab, bbbb1, 2cc99, cc938, bf29a, 7f2a8, 3c778, 255a2, 8aff5, 1c261, 77d35, b5e6e, 9b132, e8d04, 22c41, 1baff, 528bf, 27ab6, 56af9, 39af6, 4f8b0, 57ff6, d01a3, 3ab98, bd59e, 5971b, 950a1, 492b5, b6410, 674ca, 3d1a1, 09004, 4ed65, 81c94, e8770, edf6f, 15bf3, 6e32d, a1456, 65bdb, 65500, aa745, fb421, 5e803, 64f53, 02808, 419ae, 45948, 7be8c, 94b24, 0a9b2, b8278, c95a8, 61eed, d2e8e, 44b72, be1d9, 18186, e7067, ddab7, 66439, 94d05, 19eeb, b289e, 7073b, 0ab4f
2. mtmd: Add Vision Support for Minimax-M3: This pull request implements vision support for the MiniMax-M3 model by integrating a Qwen2.5-VL style Vision Transformer with key architectural changes including a 3-axis rotary positional encoding, a gate-less GELU-erf feed-forward network, and a two-stage patch-merge projector, while ensuring compatibility with existing components and validating embedding parity against the Hugging Face reference.
- URL: pull/25113
- Associated Commits: 87cc2, 53c81, f07f1, 4a320, bbf1a, 09657, 8fe2e, 8c953, ea6fb, 2e827, 01522, d1a04, b1b17, cea71, 69c95, afc09, f40d0, 714bb, 8136a, 3ed9b, 79a6e, 35990, fa158, eae55, 2bb7e, d6f94, 0a7b2, 63614, 7b7ff, 1cd03, c70c8, 25199, d54b4, f57ef, 618e1, 5a247, 5dfe8, b99a8, 56ba5, 04439, c314f, 5015a, 76834, e819f, 492e3, 0d647, 98e15, 9693d, 06833, 296f9, 11395, 07e60, fce0d, 96b30, e56d2, ad353, 5cea3, 962d0, 498ef, 09644, f096b, 59262, c9955, e5695, 1209c, 0fbac, 2c1be, 3c53a, f1f0a, 522d5, ee373, a4c1c, dd0fe
- Associated Commits: 87cc2, 53c81, f07f1, 4a320, bbf1a, 09657, 8fe2e, 8c953, ea6fb, 2e827, 01522, d1a04, b1b17, cea71, 69c95, afc09, f40d0, 714bb, 8136a, 3ed9b, 79a6e, 35990, fa158, eae55, 2bb7e, d6f94, 0a7b2, 63614, 7b7ff, 1cd03, c70c8, 25199, d54b4, f57ef, 618e1, 5a247, 5dfe8, b99a8, 56ba5, 04439, c314f, 5015a, 76834, e819f, 492e3, 0d647, 98e15, 9693d, 06833, 296f9, 11395, 07e60, fce0d, 96b30, e56d2, ad353, 5cea3, 962d0, 498ef, 09644, f096b, 59262, c9955, e5695, 1209c, 0fbac, 2c1be, 3c53a, f1f0a, 522d5, ee373, a4c1c, dd0fe
3. vulkan: Refactor vk_queue to use per-instance mutexes and unique handles: This pull request refactors the Vulkan vk_queue implementation to assign each vk_queue a unique vk::Queue handle and replaces the previous static mutex with per-instance mutexes, enabling proper per-queue locking and laying the groundwork for per-pool command buffer locking, while also integrating support for VK_KHR_internally_synchronized_queues to optimize synchronization.
- URL: pull/23570
- Associated Commits: 2e7bd, c761d, fbd12, bd8f1, f5d59, 69431, ec542, bd600, 530b8, 97cae, 496d8, af45c, 10764, 174fa, ce342, 8f1a3, 7ade7, b18a6, 3db57, 55bee, 9c73c, b75a7, dbae7, 8852c, 06700, 3b118, d41bd, 1c0ca, 3d09a, fad1a
- Associated Commits: 2e7bd, c761d, fbd12, bd8f1, f5d59, 69431, ec542, bd600, 530b8, 97cae, 496d8, af45c, 10764, 174fa, ce342, 8f1a3, 7ade7, b18a6, 3db57, 55bee, 9c73c, b75a7, dbae7, 8852c, 06700, 3b118, d41bd, 1c0ca, 3d09a, fad1a
Other Closed Pull Requests
- Unified file loading modes: This pull request refactors the three mutually exclusive file loading modes—mlock, mmap, and direct I/O—into a single unified
--load-modeoption, deprecating the old individual flags and simplifying the codebase while ensuring feature parity through added debug logging and verification. It also redefines the-lm mlockargument to restore original behavior and introduces-lm mmap+mlockfor combined memory locking with mapping.
- MCP protocol support and integration: Multiple pull requests introduce and enhance support for the MCP protocol using stdio transport, including a minimal implementation for the
llama-clitool, a modular server architecture with improved subprocess handling, and integration with external MCP servers exposing tools via the /tools endpoint. These changes also include lifecycle management improvements, test coverage, and fixes for Windows command resolution and process termination.
- Performance and optimization improvements: Several pull requests focus on optimizing performance, including enhancements to the ggml-hexagon pipeline with better L2 cache flushing and Flash Attention optimizations, updates to Hexagon activation operations with a new fused GEGLU microkernel, and proposals for speeding up model loading in multi-GPU environments via parallel loading and thread safety.
- Model architecture support and enhancements: Support for new model architectures is added, including Poolside's Laguna XS.2 and M.1 models, and MiniMax-M3, a 60-layer mixture-of-experts model with sparse attention and multi-stream support. Preliminary MiniMax-M3 support includes chat parsing and compatibility fixes, while the full model integration provides quantization guidelines and fallback mechanisms.
- Reasoning and budget parameter fixes: This topic covers fixes and clarifications to the server's handling of reasoning and thinking budget parameters, establishing precedence rules between CLI flags and per-request values, correcting ignored budget tokens, and aligning defaults and documentation for improved usability. It also includes enhancements to the reasoning budget sampler to support multiple end sequences for flexible termination.
- Subprocess management and platform support: A common
subproc.hwrapper is introduced for subprocess management, with subprocess functionality disabled on Android, iOS, and WebAssembly due to sandboxing limitations. This change also disables MTMD_VIDEO support, which depends on spawning ffmpeg, ensuring platform compatibility and stability.
- UI and WebUI improvements: UI enhancements include bulk action capabilities for sidebar conversations, improved settings logic, and multi-selection support, alongside removal of problematic UI animations that caused performance issues and visual bugs. Additionally, fixes address model names with slashes causing failures in the WebUI Stop button and stream recovery by refactoring conversation ID handling.
- Symbolic math and sandbox tool enhancements: Symbolic math support is added to the JavaScript sandbox tool by integrating the nerdamer-prime engine, enabling advanced symbolic operations with user-toggleable settings and enhanced security. This includes a rigorous verification of a counterexample to the Jacobian Conjecture using the new capabilities.
- Repetition detection in server output: An optional repetition detection feature is introduced to the server, acting as a circuit breaker by identifying exact contiguous repetition patterns in generated tokens and terminating tasks early to prevent excessive output. This feature is opt-in and compatible with existing stop-reason protocols.
- DeepSeek V4 and DS4 template improvements: Improvements to DeepSeek V4 parsing and encoding align it with the official implementation, adding specialized handling for DSML tool formats and reasoning modes to enable better performance and structured responses. Additionally, fixes to the DeepSeek4 crafted template ensure correct retention of reasoning traces with added tests for verification.
3.3 Pull Request Discussion Insights
This section will analyze the tone and sentiment of discussions within this project's open and closed pull requests that occurred within the past week. It aims to identify potentially heated exchanges and to maintain a constructive project environment.
Based on our analysis, there are no instances of toxic discussions in the project's open or closed pull requests from the past week.
IV. Contributors
4.1 Contributors
Active Contributors:
We consider an active contributor in this project to be any contributor who has made at least 1 commit, opened at least 1 issue, created at least 1 pull request, or made more than 2 comments in the last month.
If there are more than 10 active contributors, the list is truncated to the top 10 based on contribution metrics for better clarity.
| Contributor | Commits | Pull Requests | Issues | Comments |
|---|---|---|---|---|
| Raudbjorn | 258 | 1 | 0 | 0 |
| allozaur | 149 | 3 | 0 | 10 |
| marty1885 | 150 | 0 | 0 | 0 |
| ngxson | 104 | 11 | 1 | 24 |
| ServeurpersoCom | 97 | 17 | 0 | 21 |
| max-krasnyansky | 121 | 1 | 0 | 6 |
| 125 | 0 | 0 | 0 | |
| pwilkin | 80 | 10 | 0 | 11 |
| am17an | 74 | 6 | 0 | 3 |
| timkhronos | 50 | 2 | 0 | 11 |