AISpot
所屬事件：llama.cpp v0.7.0 發布：MoE 專家快取最高提速 2.2 倍（1 個來源 · 2 則報導）

2026 年 10 月 15 日，llama.cpp 先後發布版本號升至 0.7.0（建置號 b11669）的公告與 v0.7.0 發布說明，預編譯包同步更新，新增 CUDA 13（基於 13.4 庫）與 ROCm 10.0 後端，覆蓋 macOS（Apple Silicon 與 Intel）、Linux、Window… 看完整事件

llama.cpp v0.7.0 發布：MoE 專家快取最高帶來 2.2 倍解碼提速

llama.cpp releases版本更新模型地端推論開源開發工具

llama.cpp 發布 v0.7.0，最核心的變化是新增主機記憶體 MoE 專家的 GPU 快取（--moe-cache-mib），專家改由裝置端 LRU 快取執行，專家解除安裝時解碼最高提速約 2.2 倍，且該快取可跨多張 GPU 共享。版本同時加入 LiliCorr 投機解碼與 GLM5-Next 的 MTP 投機解碼，並支援 K2 Horizon、Prism Bonsai 2 27B、EmbeddingGemma-2、MiniCPM-V 4.7 等一批新模型。

llama.cpp 大版本更新新增 MoE 專家 GPU 快取，官方給出最高 2.2 倍解碼提速的具體數字，並一次併入多種投機解碼與十餘個新模型架構，對在本地或消費級硬體上跑 MoE 與多模態模型的人最有參考價值。

llama.cpp推測解碼MoE本地推理v0.7.0

原標題：v0.7.0
閱讀原文

同一事件共有 2 則報導（1 個來源），看事件全貌

評分86 / 85（平均 85，門檻 60）
狀態精選

全文翻譯

概覽 llama.cpp v0.7.0 新增了 LiliCorr 推測解碼技術以及面向 GLM5-Next 的 MTP，新增了一個用於儲存在主機記憶體中的 MoE 專家的 GPU 快取（--moe-cache-mib，當專家被解除安裝時可帶來最高約 2.2 倍的解碼加速），並新增了一大批新模型：K2 Horizon（dense 和 MoVA）、Prism Bonsai 2 27B、EmbeddingGemma-2（文字+視覺+音訊）、LiquidAI d1/d1-omni 和 vLLM decision 2/d3 decision 模型、pplx-decider、MiniCPM-V 4.7、Nemotron Super 3.5 VL / Nemotron Bolt 以及 maion-coder。伺服器端獲得了重構後的聊天會話 API（common_chat_session，基於部件的格式化輸出）、大幅改進的槽位和提示快取恢復行為、對新架構的決策模型支援以及新的預設埠（9931），Web UI 新增了完整的模型管理器，ggml 已更新至 v0.27.0。 亮點 用於主機記憶體 MoE 專家的新 GPU 快取（--moe-cache-mib）：專家通過 LRU 快取從 GPU 上執行，解碼速度最高提升 2.2 倍 #29887 專家快取現在可在多個 GPU 之間共享 #30112 LiliCorr 推測解碼：在 DFlash 之上使用小型草稿 transformer 進行候選選擇 #30506 GLM5-Next：帶有最佳化稀疏注意力路徑的 MTP 推測解碼 #29928 同時支援 dense 和 MoVA 兩種變體的 K2 Horizon：轉換、tokenizer、聊天模板、-sm 張量拆分 #29535 支援 Prism Bonsai 2 27B 三值模型 #29600 EmbeddingGemma-2：文字 + 視覺 + 音訊嵌入模型 #30054 新的決策模型：LiquidAI d1-3B、d1-omni-600M、vLLM decision 2、vLLM d3 和 pplx-decider #30110 #30114 #30378 #30383 #30044 PLaMo-3 tokenizer 實現了參考預分詞，以實現精確的 tokenization 一致性 #30045 RPC 伺服器支援 -sm 張量（張量並行 RPC，RPC 協議 v8）#26610 伺服器現在預設監聽埠 9931，而不是 8080 #30159 API 變更 include/llama.h：新增 moe_cache_size 上下文引數，用於主機專家 GPU 快取 #29887 include/llama.h：新增 LLAMA_VOCAB_TYPE_PLAMO3 詞表型別 #30045 include/llama.h：llama_perf_context_data 報告推理期間的圖重分配 #30329 include/llama.h：當使用非 NONE 的池化型別時，embeddings 現在會跳過 LM head #30445 include/llama.h：當上下文沒有狀態嵌入時，llama_batch_ext_set_embd_state 返回 false，M-RoPE 順序已記錄 #30506 tools/server：聊天層圍繞 common_chat_session 進行了重構，使用 feed() / finish() #30210 tools/server：格式化聊天輸出現在是由部件組成的陣列，而不是單個字串 #30389 tools/server：預設監聽埠已改為 9931（Docker 映象在容器內仍保留 8080）#30159 新模型 - K2 Horizon：dense 與 MoVA 變體，支援轉換、tokenizer、chat template 和 -sm 張量 #29535 - Prism Bonsai 2 27B（ternary）#29600 - EmbeddingGemma-2：文字 + 視覺 + 音訊嵌入 #30054 - LiquidAI d1-3B 和 d1-omni-600M 決策模型 #30110 #30114 - vLLM decision 2 和 vLLM d3 模型 #30378 #30383 - pplx-decider 決策模型 #30044 - MiniCPM-V 4.7 多模態模型 #29416 - Nemotron Super 3.5 VL 和 Nemotron Bolt 74B A9B #30520 - maion-coder 架構 #29778 - Mellum：從 GGUF 讀取 MoE 門控引數，並統一 RoPE 處理 #30578 - LiliCorr 草稿模型轉換與圖 #30506 - Rerankers：支援 classifier_activation，為 ModernBERT 編碼器提供精確 GELU #29692 #30108 核心變更 - 推測解碼：為 GLM5-Next 支援 MTP、LiliCorr 草稿，修正 MTP 圖中的 mtmd 處理 #29928 #30506 #30257 - 推測解碼：共享 nextn 張量標誌，並整合行裁剪與標誌變更時的 sched 重新預留 #30097 #30017 #30020 - 推測解碼：修復草稿取樣與拒絕處理，DFlash 輸出頭共享 #30095 #30111 - 新增 --spec-draft-n-ubatch，用於單獨設定草稿上下文批次大小並節省 VRAM #30459 - MoE 專家快取：主機記憶體專家由裝置 LRU 快取提供服務，支援多 GPU #29887 #30112 - 嵌入模型：啟用池化時跳過 logits 緩衝區和 LM head（最多減少 10 倍 VRAM）#30255 #30445 - 嵌入圖：重排 get_rows，去掉冗餘的 t_embd expand 和遺留的 ggml_cont 呼叫 #30160 #30461 #30198 - 無快取注意力：填充 KV 維度與 KQ 偏置，避免圖重分配 #30416 #30446 #30439 - k-pool 模型：選擇建置在因果掩碼佈局中，修復共享序列上的 scatter 資料競爭 #30480 #29994 - GLM5-Next 稀疏注意力：移除了 gather 路徑 #30042 - KV 快取：遵循 LLAMA_STATE_SEQ_FLAGS_PARTIAL_ONLY，使草稿檢查點不再攜帶完整 KV #30514 - 預取：在發出系統預取呼叫前合併範圍，絕不將外部範圍對映到後端緩衝區 #30532 #30405 - 取樣：對符合條件的零溫度鏈使用貪心選擇，後端取樣圖在 ubatch 間保持靜態 #29797 #30223 - Tokenizers：PLaMo-3 預分詞與 FiM token，Unigram 可選啟用 NFC 歸一化，嚴格 UTF-8 校驗 #30045 #30090 #30390 #30485 - Jinja：修復了迴圈變數作用域，相容 Python 的 tojson 渲染，更快的上下文複製 #30533 #30483 #30362 #30562 - 模型拆分：當沒有裝置報告空閒記憶體時，均勻分配層 #29677 多模態變更 - 新增 mtmd_build_inp_attn_mask，用於建置非因果多模態注意力掩碼 #30259 - 新增 cohere2 視覺編碼器支援 #30062 - MTP 草稿現在處理 mtmd 輸入，Qwen3.8 Flash Next 接受影像嵌入 #30257 #30397 - 新增多模態架構：MiniCPM-V 4.7、Nemotron Super 3.5 VL、EmbeddingGemma-2（視覺 + 音訊） #29416 #30520 #30054 伺服器變更 - /v1/systemone 擴充套件到新的決策模型：d1-3B、d1-omni、vLLM decision 2/d3、pplx-decider #30110 #30114 #30378 #30383 #30044 - Chat 重構：基於會話的解析和基於 part 的格式化輸出 #30210 #30389 - 生成的文字和 token 現在會作為解析器輸入進行累積，為 token 感知解析做準備 #29876 - Prompt 快取：僅儲存可恢復的 slot 狀態，因此 rerank 和 embedding 請求不再填滿快取 #30498 - Prompt 快取：子任務優先選擇空 slot，並保留它們將要覆蓋的狀態 #30567 - 被固定的忙碌 slot 保持不動，直到請求可以在其上執行 #30295 - Slot 儲存/恢復現在會保留上下文檢查點和草稿模型狀態 #26004 #30574 - 修復了 SWA 複用閾值中一個會丟棄已恢復狀態的差二錯誤 #30580 - 修復了使用 std::regex 轉義特殊 token 時的棧溢位崩潰 #30243 - 為 --slot-save-path 提供 LLAMA_ARG_SLOT_SAVE_PATH 環境變數 #30272 - 新增按元件劃分的伺服器測試；內建的 cpp-httplib 更新至 0.61.0 #30312 #30494 #30134 #30081 UI 變更 - 新的 Models Manager：一個包含所有正在服務的模型的表格，支援可展開的量化、搜尋、篩選、排序和收藏 #29583 - 行操作：載入/解除安裝、從磁碟刪除、隱藏/取消隱藏、帶進度的斷點續傳下載 #29583 - 模型配置側邊抽屜，包含模型資訊、chat 模板以及“開始新聊天”/“在此聊天中使用”操作 #29583 - Models manager 後續改進：GGUF 訓練上下文、收藏使用者體驗、篩選和排序修復 #30228 - 修復了在 router 模式下首次訪問時未應用 ui_settings 的問題 #29668 - 修復了大型貼上文字的文字附件預覽溢位問題 #29898 - 修復了預設 agentic 輪次限制回退到 10 而不是 100 的問題 #30582 ggml 變更 ggml 更新至 v0.27.0：後端排程器獲得了用於拆分權重複制和計算緩衝區重新分配的使用者回撥，並且針對張量並行 RPC 伺服器的 RPC 協議升級至 v8。各後端獲得了：CUDA/HIP 上的分塊 gated-delta-net 核心，CPU 上的 KleidiAI SME2 k-quant 矩陣乘法與 AVX2 Q2_0 點積，Vulkan 上的 L2 感知 coopmat2 排程與 bf16 啟用，SYCL 上的重排佈局 MoE 核心（MXFP4、IQ3、Q5_K、分組 XMX GEMM），OpenCL 上的 dp4a MoE GEMM，Metal 上的 tensor-API 和更寬頭維度的 flash attention，以及 Hexagon 上的 DMA/HVX 複製重構（帶池化和 3D im2col 操作）。正確性工作集中在非連續和跨步檢視、網格與工作組大小限制、越界訪問以及針對重複專家 ID 的明確定義的 mul_mat_id 行為；符號可見性現在預設為隱藏 資源 每夜建置：b11669 更多資訊 ggml-org 專案的版本發布與版本管理 自 v0.6.0 以來的變更日誌 - 620d222 llama.cpp：將版本升級至 0.7.0（#30603） - cd0a5b6 models：新增 Nemotron Super 3.5 VL 和 Nemotron Bolt 74B A9B 支援（#30520） - a11b653 scripts：新增 ci-status.sh 以列出活躍的 CI 執行（#30586） - 61ad80b server：修復 SWA 重用閾值中的 off-by-two 錯誤（#30580） - a6b91d8 model：Mellum - 從 GGUF 讀取 MoE 門控引數，統一 RoPE（#30578） - ea736c1 server：將草稿狀態儲存在槽檔案中（#30574） - b0d2391 server：優先選擇空槽，並保留被子任務覆蓋的狀態（#30567） - c0cb53a sync：ggml - 4ed1c85 ggml：將版本升級至 0.27.0 - 711780f ci：將 ty 升級至 0.0.88（#30587） - fd55cb9 fix(ui)：從 DEFAULT_AGENTIC_CONFIG 獲取智慧體輪次預設值（#30582） - 3e4dfa8 models：新增 maion-coder（#29778） - 08698c8 jinja：改進上下文複製建構函式效能（#30562） - 82e1e2f jinja：像 json.dumps 那樣在 tojson 中寫入 nan 和 inf（#30483） - 47f5a64 vulkan：修復併發的首次裝置初始化（#29766） - e36df60 sycl：用 DLE+oneDNN 包替換 oneAPI 包（#30131） - 9166cea opencl：修復 adreno noshuffle gemv 核心的越界尾部訪問（#30421） - 2130ca8 jinja：修復迴圈體變數作用域（#30533） - 23f83e0 ggml-cpu：最佳化 pad_f32（#30526） - 2ce20d2 unicode：拒絕過長 UTF-8 和超出範圍的碼點（#30485） - 68ac765 llama：將 llama_prefetch() 拆分為合併和預取步驟（#30532） - 77c1f39 opencl：在掩碼之前應用 flash attn softcap（#30546） - 733f57a vulkan：在 AMD 上一次性複製帶步長的 f16 KV，使讀取分散到各記憶體通道（#27703） - 5a638ae cpu：新增 F16 SWIGLU_OAI 參考實現與驗證（#29200） - 6e66cb5 metal：為 tensor API flash attention 核心新增 BF16 變體（#30491） - e467e5b kv-cache：在普通 KV cache 中遵循 PARTIAL_ONLY（#30514） - 1ddc262 graph：在沒有 embeddings 時保持圖非空（#30522） - 2639a3b server：僅將可恢復的 slot 狀態儲存到 prompt cache（#30498） - 843e7dc spec：新增 lilicorr 支援（#30506） - 3e390af vulkan：為 cm2 mulmat 使用對 L2 友好的工作組排程（#30215） - a0e6dc7 opencl：在非連續檢視上支援 CLAMP（#30186） - 23deec0 devops：不正確的 COPY 指令（#30529） - 683bbc0 gguf-py：檢測 nfc normalization（#30540） - 529ac89 sycl：IQ3 程式碼重排（#29107） - 685f708 ggml-cpu：在帶 F16C 的 x86 上保留 FP16->FP32 查詢表（#30342）（#30343） - c114bd4 docs：更新 IBM zDNN 後端文件（#30527） - 5cdadfc ggml-cpu：為所有執行緒設定 set_rows 轉換緩衝區大小（#30454） - 62df181 cuda、hip：GDN 分塊核心（#29353） - 37d6d65 skill：為 UI 開發更新 code-review skill（#30509） - 6868a52 ggml：修復批處理輸入的 conv_1d 結果佈局（#28738） - 3fb4767 common：新增 --spec-draft-n-ubatch（#30459） - 318a471 docs：澄清 Intel oneAPI Toolkit 的安裝建議（#30407） - 8c2134a ggml：新增關於 mul_mat_id 的說明 + 停用 mul_mat_id_dup 測試（#30501） - df18bee model：當沒有裝置報告空閒記憶體時均勻拆分層（#29677） - 3eb1013 vendor：將 cpp-httplib 更新到 0.61.0（#30494） - af0579c ggml：改進 RPC 張量傳輸日誌（#30481） - 62bacfa cuda：在無 WMMA 的 AMD 上使用 SIMT 隱式 GEMM conv2d（#30325） - 77390ba jinja：像 Python 那樣在 tojson 中寫入浮點數（#30362） - c5a9976 opencl：對 adreno moe expert sort 使用 buf+offset 而不是 subbuffer（#30418） - 21a19cd opencl：修復 q8_0 dp4a mul_mat_id，並將 PREC_F32 mul_mat_id 標記為 adreno 不支援（#30365） - 9cf967a models：移除 embedding 圖的 t_embd 擴充套件（#30461） - 4dd6dd4 HIP：在 RDNA3.5 上對某些量化型別使用 64 行 MMQ 分塊（#28195） - 1431019 vulkan：修復編譯錯誤（#30495） - e7268bf llama：按 causal mask 佈局建置 k-pool 選擇（#30480） - cdce056 vulkan：為 Intel Windows 上裝置空閒記憶體大於總記憶體新增變通方案（#29835） - d524bb6 vulkan：為 Intel Arc Pro B65 + B70 定義著色器數量（#30431） - 7ec420b vulkan: 支援 mulmat bf16 啟用 ( #29069 ) - 10da8c4 unicode : 為 Unigram tokenizers 新增可選的 NFC 規範化 ( #30390 ) - eca5458 convert : 匯出 YaRN 截斷配置 ( #30288 ) - 091b6a7 在 FP32/FP16 SYCL 建置中啟用 OpenSSL 以恢復 HF HTTPS 下載 ( #30330 ) - fcac190 webgpu: (修復 CI) 修復具有重複專家 ID 的 mul_mat_id ( #30451 ) - 04f7226 server : 不使用 std::regex 轉義特殊令牌 ( #30243 ) - ff026f8 chat: 用 parts 替換輸出字串 ( #30389 ) - 1b70350 ggml : 簡化後端搜尋路徑錯誤處理 ( #28367 ) - d71e324 docs : 更新 zDNN 支援的操作 ( #30472 ) - 45451c3 kleidiai: 新增 SME2 Q4_K 和 Q6_K matmul 支援 ( #29219 ) - 77c4540 vulkan: 當池化視窗較大時最佳化 pool_2d ( #30119 ) - ecdd29e meta: 當 arena 重新分配時重新建立所有子圖 ( #30453 ) - ec0bfac hexagon: 新增可配置的 HTP DCVS/HMX 電源模式 ( #30423 ) - ace20f6 graph: 為無快取注意力的 KQ bias 填充 ( #30446 ) - b9f3599 llama : 不要將外部範圍對映到後端緩衝區 ( #30405 ) - f441d2d model: 為 laguna 模型載入 YaRN beta 後設資料 ( #30331 ) - cb55ffe graph: 對於池化嵌入跳過 LM head ( #30445 ) - a17e94d tests : 將上下文狀態測試合併到 test-llama-context ( #30023 ) - b476d9a llama : 修復日誌訊息 ( #30452 ) - e0bcacc vulkan: 允許多個裝置訪問 GPU 可見的主機記憶體 ( #29741 ) - 0a731cc ggml-cpu : 為 Q2_0 新增 AVX2 點積 ( #30447 ) - f851e24 cuda : 新增 GGML_CUDA_GRAPH_DEBUG 並只讀取一次環境變數 ( #30327 ) - bfe65e1 cuda : 修復 mmq-load-tiles.cuh 中的編譯器警告 ( #30443 ) - 411d548 vulkan : 當超過 maxComputeWorkGroupCount 時拆分 matmul 排程 ( #29533 ) - 296538a spec : 從 dsv4 graph_mtp 中移除令牌斷言 ( #30444 ) - 7f51953 model : 允許 Qwen3.8 Flash Next 接受用於 MTP 的影像嵌入 ( #30397 ) - 882d671 cuda : 修復死程式碼 break 產生的警告 ( #30441 ) - 30f7df1 cuda : 清理警告 ( #30420 ) - 9d7d504 convert : 修復使用 --outtype auto 的 LoRA 轉換 ( #30433 ) - ed019cd llama : 當圖不產生 logits 時跳過 logits 緩衝區分配 ( #30255 ) - 9880157 graph : 避免 "no cache" 路徑中的重新分配 ( #30439 ) - af50f2f ggml-zdnn : 引入 Q8_0 矩陣乘法 ( #29836 ) - 071b553 models : 移除現在被所有一元核心支援的冗餘 ggml_cont 呼叫 ( #30198 ) - 4413a19 hexagon: 擴充套件對二元運算的標量 DMA 廣播支援 ( #30245 ) - 8f574b0 ci：使用 GGML_NATIVE=OFF 和 CPU 後端變體建置 OpenVINO 發行版 ( #30324 ) - a49f58d ci：修復 OpenVINO 下載連結 ( #30401 ) - cbe28bb model：修復 vllm d3 多序列情況 ( #30408 ) - 358a420 graph：將無快取注意力的 KV 維度填充到 256 ( #30416 ) - 48e9857 metal：修復大張量的 metal 張量 kernel_mul_mm 錯誤 ( #28748 ) - 8bb3cf0 hexagon：最佳化 CPY/CONCAT — 第 3 輪，短行版本 ( #30341 ) - 0f701c8 vulkan：為 f16 累積打包 FMA，無點積操作 ( #29877 ) - efe3626 model：支援 vllm d3 模型 ( #30383 ) - 097cf33 model：支援 vllm decision 2 模型 ( #30378 ) - ccdbe29 tests：將每個測試僅連結到其使用的庫 ( #30299 ) - 073b740 metal：修復具有重複 id 的 mul_mat_id ( #30373 ) - 77757a3 examples：修復草稿取樣和拒絕處理 ( #30095 ) - 972b5fc llama：報告推理期間的計算圖重新分配 ( #30329 ) - 2f160ba ui：修復文字附件預覽溢位 ( #29898 ) - f9a95ae sycl：在行尾保護 DMMV 讀取 ( #29338 ) - 08ffba2 vulkan：使用三元表示式避免 CM2 量化塊大小的 switch lowering ( #30164 ) - a05e58c ggml：將符號的預設可見性更改為 hidden ( #30014 ) - b2f036c vulkan：修復原始碼拆分後 32 位平台上的緩衝區日誌記錄 ( #30326 ) - e937b17 hexagon：最佳化 q2_k bpw 和啟用 ( #30035 ) - b13a93a hexagon：新增 IM2COL_3D ( #30305 ) - f2cc728 opencl：改進 fa，允許 gemma-4 使用 dk512，改進 dk64 ( #30266 ) - c83f305 opencl：修復 q4_k 密集 bin 核心、q4_0 bcast 和 rms_norm 的影像限制 ( #30310 ) - 5c8854f rpc：將 GGML_RPC_DEBUG 轉為 int

由 AI 翻譯，以原文為準。

原文
Overview llama.cpp v0.7.0 adds the LiliCorr speculative decoding technique and MTP for GLM5-Next, a new GPU cache for MoE experts kept in host memory ( --moe-cache-mib , up to ~2.2x decode speedup when experts are offloaded), and a large batch of new models: K2 Horizon (dense and MoVA), Prism Bonsai 2 27B, EmbeddingGemma-2 (text+vision+audio), the LiquidAI d1/d1-omni and vLLM decision 2/d3 decision models, pplx-decider, MiniCPM-V 4.7, Nemotron Super 3.5 VL / Nemotron Bolt and maion-coder. The server gets a refactored chat session API ( common_chat_session , part-based formatted output), much better slot and prompt-cache restore behavior, decision-model support for the new architectures and a new default port (9931), the web UI gains a full models manager, and ggml is updated to v0.27.0. Highlights New GPU cache for host-memory MoE experts ( --moe-cache-mib ): experts run on the GPU from an LRU cache, up to 2.2x faster decode #29887 The expert cache is now shared across multiple GPUs #30112 LiliCorr speculative decoding: candidate selection with a small draft transformer on top of DFlash #30506 GLM5-Next: MTP speculative decoding with an optimized sparse attention path #29928 K2 Horizon support in both dense and MoVA variants: conversion, tokenizer, chat template, -sm tensor splitting #29535 Prism Bonsai 2 27B ternary model support #29600 EmbeddingGemma-2: text + vision + audio embedding model #30054 New decision models: LiquidAI d1-3B, d1-omni-600M, vLLM decision 2, vLLM d3 and pplx-decider #30110 #30114 #30378 #30383 #30044 PLaMo-3 tokenizer implements reference pre-segmentation for exact tokenization parity #30045 RPC server supports -sm tensor (tensor-parallel RPC, RPC protocol v8) #26610 Server now listens on port 9931 by default instead of 8080 #30159 API changes include/llama.h : new moe_cache_size context param for the host-expert GPU cache #29887 include/llama.h : new LLAMA_VOCAB_TYPE_PLAMO3 vocab type #30045 include/llama.h : llama_perf_context_data reports graph reallocations during inference #30329 include/llama.h : embeddings now skips the LM head when a pooling type other than NONE is used #30445 include/llama.h : llama_batch_ext_set_embd_state returns false when the context has no state embedding, M-RoPE order documented #30506 tools/server : chat layer refactored around common_chat_session with feed() / finish() #30210 tools/server : formatted chat output is now an array of parts instead of a single string #30389 tools/server : default listening port changed to 9931 (Docker images keep 8080 inside the container) #30159 New models K2 Horizon: dense and MoVA variants with conversion, tokenizer, chat template and -sm tensor support #29535 Prism Bonsai 2 27B (ternary) #29600 EmbeddingGemma-2: text + vision + audio embeddings #30054 LiquidAI d1-3B and d1-omni-600M decision models #30110 #30114 vLLM decision 2 and vLLM d3 models #30378 #30383 pplx-decider decision model #30044 MiniCPM-V 4.7 multimodal model #29416 Nemotron Super 3.5 VL and Nemotron Bolt 74B A9B #30520 maion-coder architecture #29778 Mellum: MoE gating parameters read from GGUF and unified RoPE handling #30578 LiliCorr draft model conversion and graph #30506 Rerankers: classifier_activation support, exact GELU for ModernBERT encoders #29692 #30108 Core changes Speculative decoding: MTP for GLM5-Next, LiliCorr drafts, correct mtmd handling in MTP graphs #29928 #30506 #30257 Speculative decoding: shared nextn tensor flags with consolidated row cropping and sched re-reserve on flag change #30097 #30017 #30020 Speculative decoding: fixed draft sampling and rejection handling, DFlash output head sharing #30095 #30111 New --spec-draft-n-ubatch to size the draft context batch separately and save VRAM #30459 MoE expert cache: host-memory experts served from a device LRU cache, supported over multiple GPUs #29887 #30112 Embedding models: skip the logits buffer and the LM head when pooling is enabled (up to 10x less VRAM) #30255 #30445 Embedding graphs: reordered get_rows , dropped the redundant t_embd expand and leftover ggml_cont calls #30160 #30461 #30198 No-cache attention: padded KV dimension and KQ bias, avoided graph reallocations #30416 #30446 #30439 k-pool models: selection built in the causal mask layout, fixed a scatter data race on shared sequences #30480 #29994 GLM5-Next sparse attention: removed the gather path #30042 KV cache: honor LLAMA_STATE_SEQ_FLAGS_PARTIAL_ONLY so draft checkpoints stop carrying the full KV #30514 Prefetching: merge ranges before issuing system prefetch calls, never map foreign ranges into a backend buffer #30532 #30405 Sampling: greedy selection for eligible temperature-zero chains, backend sampling graph kept static across ubatches #29797 #30223 Tokenizers: PLaMo-3 pre-segmentation and FiM tokens, opt-in NFC normalization for Unigram, strict UTF-8 validation #30045 #30090 #30390 #30485 Jinja: fixed loop variable scoping, Python-compatible tojson rendering, faster context copy #30533 #30483 #30362 #30562 Model split: distribute layers evenly when no device reports free memory #29677 Multi-modality changes Added mtmd_build_inp_attn_mask for building non-causal multimodal attention masks #30259 Added cohere2 vision encoder support #30062 MTP drafts now handle mtmd inputs, Qwen3.8 Flash Next accepts image embeddings #30257 #30397 New multimodal architectures: MiniCPM-V 4.7, Nemotron Super 3.5 VL, EmbeddingGemma-2 (vision + audio) #29416 #30520 #30054 Server changes /v1/systemone extended to the new decision models: d1-3B, d1-omni, vLLM decision 2/d3, pplx-decider #30110 #30114 #30378 #30383 #30044 Chat refactor: session-based parsing and part-based formatted output #30210 #30389 Generated text and tokens are now accumulated as parser input, preparing token-aware parsing #29876 Prompt cache: only restorable slot states are saved, so rerank and embedding requests no longer fill the cache #30498 Prompt cache: child tasks prefer empty slots and preserve the state they would overwrite #30567 A pinned busy slot is left untouched until the request can run on it #30295 Slot save/restore now keeps context checkpoints and the draft model state #26004 #30574 Fixed an off-by-two in the SWA reuse threshold that discarded restored states #30580 Fixed a stack-overflow crash when escaping special tokens with std::regex #30243 LLAMA_ARG_SLOT_SAVE_PATH environment variable for --slot-save-path #30272 New per-component server tests; vendored cpp-httplib updated to 0.61.0 #30312 #30494 #30134 #30081 UI changes New Models Manager: one table of all served models with expandable quants, search, filters, sorting and favorites #29583 Row actions: load/unload, delete from disk, hide/unhide, resumable downloads with progress #29583 Model configuration side drawer with model info, chat template and "start a new chat" / "use in this chat" actions #29583 Models manager follow-ups: GGUF training context, favorite UX, filter and sort fixes #30228 Fixed ui_settings not being applied on the first visit in router mode #29668 Fixed text attachment preview overflow for large pasted text #29898 Fixed the default agentic turns limit falling back to 10 instead of 100 #30582 ggml changes ggml updated to v0.27.0 : the backend scheduler gains user callbacks for split weight copying and compute-buffer reallocation, and the RPC protocol is bumped to v8 for tensor-parallel RPC servers. Backends gained chunked gated-delta-net kernels on CUDA/HIP, KleidiAI SME2 k-quant matmuls and AVX2 Q2_0 dot products on CPU, L2-aware coopmat2 scheduling and bf16 activations on Vulkan, reordered-layout MoE kernels (MXFP4, IQ3, Q5_K, grouped XMX GEMM) on SYCL, dp4a MoE GEMMs on OpenCL, tensor-API and wider-head-dim flash attention on Metal, and a DMA/HVX copy overhaul with pooling and 3D im2col ops on Hexagon. Correctness work focused on non-contiguous and strided views, grid and workgroup size limits, out-of-bounds accesses and well-defined mul_mat_id behavior for duplicate expert ids; symbol visibility now defaults to hidden Assets Nightly build: b11669 More info Releases and versioning of ggml-org projects Changelog since v0.6.0 620d222 llama.cpp : bump version to 0.7.0 ( #30603 ) cd0a5b6 models : add Nemotron Super 3.5 VL and Nemotron Bolt 74B A9B support ( #30520 ) a11b653 scripts : add ci-status.sh to list active CI runs ( #30586 ) 61ad80b server: fix the off-by-two in the SWA reuse threshold ( #30580 ) a6b91d8 model : Mellum - read MoE gating params from GGUF, unify RoPE ( #30578 ) ea736c1 server: save the draft state in the slot file ( #30574 ) b0d2391 server: prefer empty slots and keep states overwritten by child tasks ( #30567 ) c0cb53a sync : ggml 4ed1c85 ggml : bump version to 0.27.0 711780f ci : bump ty to 0.0.88 ( #30587 ) fd55cb9 fix(ui): take the agentic turns default from DEFAULT_AGENTIC_CONFIG ( #30582 ) 3e4dfa8 models : add maion-coder ( #29778 ) 08698c8 jinja : improve context copy constructor performance ( #30562 ) 82e1e2f jinja : write nan and inf in tojson as json.dumps does ( #30483 ) 47f5a64 vulkan : fix concurrent first device init ( #29766 ) e36df60 sycl : replace oneAPI package by DLE+oneDNN package ( #30131 ) 9166cea opencl: fix OOB tail access for adreno noshuffle gemv kernels ( #30421 ) 2130ca8 jinja : fix loop body variable scoping ( #30533 ) 23f83e0 ggml-cpu : optimize pad_f32 ( #30526 ) 2ce20d2 unicode : reject overlong UTF-8 and out-of-range codepoints ( #30485 ) 68ac765 llama : split llama_prefetch() into merge and prefetch steps ( #30532 ) 77c1f39 opencl: apply flash attn softcap before mask ( #30546 ) 733f57a vulkan : copy strided f16 KV once on AMD so reads spread across memory channels ( #27703 ) 5a638ae cpu: add F16 SWIGLU_OAI reference implementation and validation ( #29200 ) 6e66cb5 metal : add BF16 variant of the tensor API flash attention kernel ( #30491 ) e467e5b kv-cache: honor PARTIAL_ONLY in the plain KV cache ( #30514 ) 1ddc262 graph: keep the graph non empty without embeddings ( #30522 ) 2639a3b server: save only restorable slot states to the prompt cache ( #30498 ) 843e7dc spec : add lilicorr support ( #30506 ) 3e390af vulkan: Use L2-friendly workgroup scheduling for cm2 mulmat ( #30215 ) a0e6dc7 opencl: support CLAMP on non-contiguous views ( #30186 ) 23deec0 devops: incorrect COPY instruction ( #30529 ) 683bbc0 gguf-py : detect nfc normalization ( #30540 ) 529ac89 sycl: IQ3 code reorder ( #29107 ) 685f708 ggml-cpu : keep the FP16->FP32 lookup table on x86 with F16C ( #30342 ) ( #30343 ) c114bd4 docs: update IBM zDNN backend docs ( #30527 ) 5cdadfc ggml-cpu : size set_rows conversion buffer for all threads ( #30454 ) 62df181 cuda, hip: GDN chunked kernel ( #29353 ) 37d6d65 skill: Update code-review skill for UI development ( #30509 ) 6868a52 ggml: fix conv_1d result layout for batched input ( #28738 ) 3fb4767 common : add --spec-draft-n-ubatch ( #30459 ) 318a471 docs : clarify installation recommendation for Intel oneAPI Toolkit ( #30407 ) 8c2134a ggml : add note about mul_mat_id + disable mul_mat_id_dup tests ( #30501 ) df18bee model: split layers evenly when no device reports free memory ( #29677 ) 3eb1013 vendor : update cpp-httplib to 0.61.0 ( #30494 ) af0579c ggml : improve RPC tensor transfer logs ( #30481 ) 62bacfa cuda : use SIMT implicit GEMM conv2d on AMD without WMMA ( #30325 ) 77390ba jinja : write floats in tojson as Python does ( #30362 ) c5a9976 opencl: use buf+offset instead of subbuffer for adreno moe expert sort ( #30418 ) 21a19cd opencl: fix q8_0 dp4a mul_mat_id and mark PREC_F32 mul_mat_id unsupported for adreno ( #30365 ) 9cf967a models: drop the t_embd expand of the embedding graphs ( #30461 ) 4dd6dd4 HIP: use 64-row MMQ tiles for some quant types on RDNA3.5 ( #28195 ) 1431019 vulkan: fix compile error ( #30495 ) e7268bf llama : build the k-pool selection in the causal mask layout ( #30480 ) cdce056 vulkan: Add workaround for device free memory > total memory on Intel Windows ( #29835 ) d524bb6 vulkan: Define the shader count for Intel Arc Pro B65 + B70 ( #30431 ) 7ec420b vulkan: support mulmat bf16 activations ( #29069 ) 10da8c4 unicode : add opt-in NFC normalization for Unigram tokenizers ( #30390 ) eca5458 convert : export YaRN truncation config ( #30288 ) 091b6a7 Enable OpenSSL in FP32/FP16 SYCL builds to restore HF HTTPS downloads ( #30330 ) fcac190 webgpu: (Fix CI) fix mul_mat_id with duplicate expert ids ( #30451 ) 04f7226 server : escape the special tokens without std::regex ( #30243 ) ff026f8 chat: replace output string with parts ( #30389 ) 1b70350 ggml : simplify backend search path error handling ( #28367 ) d71e324 docs : update zDNN supported ops ( #30472 ) 45451c3 kleidiai: add SME2 Q4_K and Q6_K matmul support ( #29219 ) 77c4540 vulkan: Optimize pool_2d when the pooling window is large ( #30119 ) ecdd29e meta: recreate all subgraphs when the arena is reallocated ( #30453 ) ec0bfac hexagon: add configurable HTP DCVS/HMX power mode ( #30423 ) ace20f6 graph: pad the KQ bias of the no-cache attention ( #30446 ) b9f3599 llama : do not map foreign ranges into a backend buffer ( #30405 ) f441d2d model: load YaRN beta metadata for laguna models ( #30331 ) cb55ffe graph: skip the LM head for pooled embeddings ( #30445 ) a17e94d tests : consolidate context state tests into test-llama-context ( #30023 ) b476d9a llama : fix log message ( #30452 ) e0bcacc vulkan: allow multiple devices to access gpu-visible host memory ( #29741 ) 0a731cc ggml-cpu : add AVX2 dot product for Q2_0 ( #30447 ) f851e24 cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once ( #30327 ) bfe65e1 cuda : fix compiler warnings in mmq-load-tiles.cuh ( #30443 ) 411d548 vulkan : split matmul dispatch when it would exceed maxComputeWorkGroupCount ( #29533 ) 296538a spec : remove tokens assert from dsv4 graph_mtp ( #30444 ) 7f51953 model : allow Qwen3.8 Flash Next to accept image embds for MTP ( #30397 ) 882d671 cuda : fix warnings from dead breaks ( #30441 ) 30f7df1 cuda : warnings cleanup ( #30420 ) 9d7d504 convert : fix LoRA conversion with --outtype auto ( #30433 ) ed019cd llama : skip logits buffer allocation when graph produces no logits ( #30255 ) 9880157 graph : avoid reallocations in the "no cache" path ( #30439 ) af50f2f ggml-zdnn : introduce Q8_0 mat mul ( #29836 ) 071b553 models : remove redundant ggml_cont calls now supported by all unary kernels ( #30198 ) 4413a19 hexagon: extend scalar DMA broadcasting support for binary ops ( #30245 ) 8f574b0 ci : build OpenVINO releases with GGML_NATIVE=OFF and CPU backend variants ( #30324 ) a49f58d ci: fix OpenVINO download links ( #30401 ) cbe28bb model: fix vllm d3 multi seq case ( #30408 ) 358a420 graph: pad the KV dimension of the no-cache attention to 256 ( #30416 ) 48e9857 metal: fix metal tensor kernel_mul_mm error for large tensor ( #28748 ) 8bb3cf0 hexagon: optimize CPY/CONCAT — round 3, short-rows edition ( #30341 ) 0f701c8 vulkan: pack FMA for f16 accumulation, no dot product op ( #29877 ) efe3626 model: support vllm d3 models ( #30383 ) 097cf33 model: support vllm decision 2 model ( #30378 ) ccdbe29 tests : link each test against only the libraries it uses ( #30299 ) 073b740 metal : fix mul_mat_id with duplicate ids ( #30373 ) 77757a3 examples : fix draft sampling and rejection handling ( #30095 ) 972b5fc llama : report compute graph reallocations during inference ( #30329 ) 2f160ba ui: fix text attachment preview overflow ( #29898 ) f9a95ae sycl: guard DMMV reads at row tails ( #29338 ) 08ffba2 vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size ( #30164 ) a05e58c ggml: Change default visibility of symbols to hidden ( #30014 ) b2f036c vulkan: fix buffer logging on 32-bit platforms after source split ( #30326 ) e937b17 hexagon: optimize q2_k bpw and activation ( #30035 ) b13a93a hexagon: add IM2COL_3D ( #30305 ) f2cc728 opencl: improve fa, allow dk512 for gemma-4, improve dk64 ( #30266 ) c83f305 opencl: fix image limit for q4_k dense bin kernels, q4_0 bcast and rms_norm ( #30310 ) 5c8854f rpc : turn GGML_RPC_DEBUG into a verbosity level and add logs ( #29544 ) a9d27ac model : support for Prism Bonsai 2 27B ( #29600 ) 1cf7767 cpu: fix AVX2 CPUs not using F16C when available ( #30270 ) 966baae vulkan: handle mul_mat_id duplicates in the prepass rather than looping ( #29998 ) 62a6f74 server: add component tests ( #30312 ) 23b0202 server: leave a busy slot untouched when a request pins it ( #30295 ) 69f201a model : support MiniCPM-V 4.7 ( #29416 ) abee0c8 cmake(s390x): disable z17 target for unsupported compilers ( #30297 ) aa94f20 args: add LLAMA_ARG_SLOT_SAVE_PATH env for --slot-save-path ( #30272 ) 781dbc5 spec: properly handle mtmd input for mtp ( #30257 ) 0fd868c mtmd: add build_inp_attn_mask ( #30259 ) 1623d8c cuda: always use MMVQ for MUL_MAT_ID on sm_60 ( #27828 ) 1bb2b9f CUDA/HIP: fix race in flash_attn_ext_f16_process_tile when nbatch_combine != DKQ/2 ( #30103 ) 2bbca8f ggml-cpu: vectorize fp32 to fp16 conversion ( #30157 ) 404f557 vulkan: use 4 rows for NVIDIA MUL_MAT_ID MMVQ ( #29274 ) b797c82 ggml: fix s390x all cpu build ( #30140 ) f2918ca opencl: add bin kernels kernel_gemm_moe_q4_k_q8_1_dp4a_bin, kernel_gemm_moe_q6_k_q8_1_dp4a_bin ( #30187 ) 1e6f04a sycl : accelerate MXFP4 MoE with arithmetic decoding and weight reordering ( #29809 ) 10a60cf vendor: apply deep nested json patch from upstream ( #30253 ) 79e2e74 CUDA: fix round issue, under MSVC the CPU and GPU agree ( #30229 ) 8e2d31e graph : reorder get_rows for embeddings ( #30160 ) baef3ed ui: Models Manager Follow-up Improvements ( #30228 ) f39148a llama-bench: respect -fitc if bigger than required benchmark size ( #28331 ) 50e3e3e CUDA: Remove redundant CUDA copies after SSM_SCAN ( #29807 ) 64df918 opencl: fix kernel compilation for a6x GPUs ( #30176 ) 6184e92 model : use exact GELU for ModernBERT encoders ( #30108 ) 8b54361 chat : refactor API ( #30210 ) a518119 llama: keep the backend sampling graph static across ubatches ( #30223 ) 8ae3867 ggml: fix OOB write in ggml_acc with negative offset ( #30135 ) e60eff9 meta : handle host views ( #30217 ) ba6439a webgpu: use 2D workgroup dispatch for all the ops which use 1D dispatch (e.g., rms_norm) ( #30219 ) 5e4878e vulkan: fix rms_norm workgroup count overflow ( #30145 ) 609290b convert : support compressed-tensor mixed-precision NVFP4 checkpoint ( #28636 ) 86a2835 musa: drop mp_21 from the default architectures ( #30203 ) e94acad ci: fix Models Backend webgpu by supporting GGML_OP_DUP ( #30216 ) 5e1d740 ci : set run-name for publish release ( #30214 ) b42b7e6 server: use port 9931 by default ( #30159 ) b013e56 ui : add the models manager ( #29583 ) 2c564f4 ci: fix HIP quality check by ignoring the 320/256 FA kernel spill ( #30212 ) 1f8fa52 metal: add the 128/96 flash attention kernels, accept every compiled pair ( #30209 ) 8a1a9b5 CUDA: pass src1 precision to host MMQ config helpers ( #30168 ) d4d82d6 hexagon: fix IM2COL patch-embed DMA ring overflow ( #30189 ) 3d65c90 sycl : Q5_K reorder-layout MMVQ and fused GLU ( #29375 ) de7fa0a Musa FWHT fix ( #30167 ) 71ad059 CUDA: improve top-k algorithm selection ( #28713 ) a11f57b model : fix DFlash output head sharing ( #30111 ) fc9ce6b CUDA: fix MMQ out-of-bounds reads ( #29953 ) c35b667 CUDA : looped PAD kernel for more than 65535 rows or slices ( #30147 ) 1167d3f CUDA: fix CCCL version guard breaking on major version rollover ( #29453 ) 4f92965 ui: apply ui_settings on first visit in router mode ( #29668 ) c811cb8 llama: support MoE cache over multiple GPUs ( #30112 ) 033df86 server : preserve context checkpoints across slot save/restore ( #26004 ) ff5888f vulkan : fix TOP_K for +inf/NaN inputs and k = 1 on negative values ( #30107 ) dac3087 vulkan: extend sparse FA support to coopmat2 ( #30003 ) 03aa006 vendor : update cpp-httplib to 0.60.1 ( #30134 ) 08246a2 cuda : support arbitrary striding for unary ops on f16, f32, and bf16 ( #29781 ) 46baf1f sycl: FWHT optimizations ( #29605 ) 097f5b5 hex-mmadd: do not assume aligned read/write when bias-add is fused ( #30133 ) d888016 cuda : Use byte strides for roll to allow non-contiguous ROLL operations ( #29547 ) fda1866 CUDA: fix norm family kernels when ne[2]/ne[3] exceed grid dim limits ( #28175 ) ff30363 sycl: fuse the delta-net alpha gate (add + unary + mul) ( #29687 ) 000bee5 sycl: stage bulk uploads (model loading) through a pinned ring buffer ( #29608 ) 37ac634 model : support classifier_activation for rerankers ( #29692 ) 24e4183 ggml-cuda: assign four GDN state columns per warp ( #30087 ) 847f447 sycl: add grouped MoE XMX GEMM ( #29245 ) 75118a3 convert : support Qwen3.5 embedding models ( #27920 ) 9b4ed0c sycl: remove duplicate block-size defines from op headers ( #29507 ) 9c2e0e4 hexagon: enable alloc_buffer_n ( #30126 ) 06cad0b hexagon: Q6_K weight dequant speedup ( #30121 ) a657f7e model : add LiquidAI/d1-omni-600M decision model ( #30114 ) aa5e009 hexagon: support tiled Q4_K and Q6_K GET_ROWS ( #30115 ) 7081510 hexagon: improved GELU accuracy ( #30104 ) bd4eeaa chat: fix jinja parser for TranslateGemma ( #30096 ) 5de7334 chat : name tool and argument parser rules by index ( #30088 ) 88dcc46 model : add LiquidAI/d1-3B decision model ( #30110 ) b86d2f0 cuda: FWHT kernels for block widths above 512 ( #29100 ) 50a6c5c mtmd: add cohere2 vision support ( #30062 ) d6cf9ac llama : add a GPU cache for MoE experts kept in host memory ( #29887 ) 42c787e cuda: update uncoalesced memory reads in pool2d ( #29425 ) 18b5f8b server : accumulate generated text and tokens as parse input ( #29876 ) 448147d llama: share the nextn tensor flags between models ( #30097 ) 9881906 metal : few-row MMA mat-mul for the remaining src0 types ( #30065 ) 7e8324f metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT ( #30100 ) b9acf13 feat: add GLM5Next MTP, optimize ( #29928 ) 48499d2 qwen3tts : guard speaker_encoder_config patch for CustomVoice variant ( #29179 ) d0b490f sampling : use greedy selection for eligible temperature-zero chains ( #29797 ) 42b021b vocab : add plamo fim tokens ( #30090 ) 7481354 convert : fix token configuration for PLaMo-3 ( #29843 ) ad21565 musa: use the tile lightning indexer kernel ( #30080 ) b7dafa0 vendor : update cpp-httplib to 0.60.0 ( #30081 ) 2690873 imatrix : include clocale for std::setlocale ( #30079 ) 36a7391 ggml-webgpu: fix flash_attn supports_op check for overlapping KV ( #28205 ) fa3c2fa tests : retain the anchor when testing recurrent rollback ( #29923 ) 005a1e1 [SYCL] fix the issue in mixed different model GPUs in FA ( #29071 ) d2a79e6 sycl: accelerate GLM MLA prefill with MKL flash attention ( #29171 ) 5e5b628 sycl : fattn_kv_buffers cleanup ( #27689 ) 4d756bc vulkan: fix amd iGPU slow checkpoint read ( #30049 ) 78651c4 sycl: add IQ3_S multi-column MMVQ ( #29500 ) f498f86 ggml-webgpu: no dawn native features on wasi ( #27069 ) c479922 hexagon: CPY/CONCAT/CONT/DUP overhaul to use DMA/HVX for all cases ( #30067 ) 5ad1c5d cuda : add BF16 support for XIELU ( #29955 ) 51ce9c1 ggml-cuda: use per-thread stream for buffer-init padding memset ( #28782 ) abeada3 vocab : implement PLaMo-3 tokenizer pre-segmentation ( #30045 ) 4625240 model : add K2 Horizon dense and MoVA support ( #29535 ) 3109914 llama: remove the gather path of the glm5-next sparse attention ( #30042 ) 4fbc76d model: support embeddinggemma2 (text+vision+audio) ( #30054 ) 2207c8e opencl: fix OOB read in adreno xmem GEMM ( #30041 ) a46709b RPC: add -sm tensor ( #26610 ) 65840ed ggml: fix CLAMP on non-contiguous views (CPU, CUDA) ( #29517 ) ab09ea4 cuda: BF16/FP16 conversion to f32 chunking ( #29442 ) da263e7 models: support pplx-decider ( #30044 ) a043d38 metal : fix excess threadgroup memory in quantized flash attention ( #29340 ) 58cb913 vulkan : check for null vkEnumerateInstanceVersion ( #29872 ) 4f54067 HIP: use -O0 for host code in debug builds ( #29795 ) f0c41e0 models : consolidate nextn row cropping into shared helpers ( #30017 ) d7a695e scripts : limit apiabi checks to libllama and libmtmd ( #30038 ) 6c73b3e convert : add text_config as fallback [transformers 5.18] ( #30040 ) 1a3011c llama : re-reserve the sched when the nextn extraction flags change ( #30020 ) 6753a03 ggml: refactor selective expert copying to user code ( #29943 ) cbb7d52 test-llama-archs : initialize backends before generating models ( #30034 ) 63bef27 vendor : update LibreSSL to 4.3.3 [no ci] ( #30019 ) b9a5a00 ggml-openvino: fix CI tests; fix GPU regressions. ( #30037 ) 43fe9c6 llama: fix k-pool scatter data race on shared sequences ( #29994 ) 5e03bdd hexagon: ssm-conv updates ( #29971 ) 50569eb hexagon: add pool op support ( #29995 ) 7049ff0 ci : add 1accel label [no ci] ( #30016 ) c250304 ci : skip container re-tagging when Require Docker is disabled ( #30008 ) 8345f33 hexagon: matmul and flash-atten scalability updates ( #29974 )

相關報導

llama.cpp releases● 精選AI 評分69

llama.cpp b11507 支援 MoE 快取跨多 GPU

llama.cpp 發布 b11507 版本，新增 MoE 快取跨多張 GPU 的支援（PR #30112），可讓混合專家模型在多 GPU 環境下更充分地利用視訊記憶體。該版本同時更新了各平台預編譯產物，覆蓋 macOS Apple Silicon、iOS、Linux（CUDA 12/13、Vulkan、ROCm 10.0、SYCL、OpenVINO）、Windows、Android 以及 openEuler 等平台。

地端推論開源llama.cppMoE多 GPU本地推理版本更新
閱讀原文
llama.cpp releases● 精選AI 評分82

llama.cpp v0.6.0 發布：新增擴充套件 batch API 與 GLM-5.3-Flash 支援

llama.cpp 發布 v0.6.0，新增 llama_batch_ext 擴充套件 batch API，支援混合 token/embedding 輸入與 MTP、deepstack 狀態嵌入。本次加入 320B 的 GLM-5.3-Flash（GLM5-Next）文字+視覺混合模型、Clef 決策模型，以及 Qwen4Exp 的 MTP 投機解碼，官方稱在 DGX Spark 上解碼約快 1.5 倍。

模型開發工具llama.cppGLM-5.3-FlashMTP 投機解碼Metal多模態
2 個來源 · 4 則報導閱讀原文