AISpot
所屬事件：llama.cpp 服務端加入即時語音轉寫與對話，發布 v0.8.0（1 個來源 · 4 則報導）

10 月 21 日至 28 日，llama.cpp 服務端連續幾個建置加入即時語音轉寫與對話，v0.8.0 正式提供 /v1/realtime 端點。建置 b11780 按 GA Realtime 規範用 WebSocket 提供即時轉寫會話，會話物件為 transcription，輸入格式、轉寫模型、輪次檢測與降噪放… 看完整事件

llama.cpp v0.8.0 支援 LFM3 全模態與即時語音端點

llama.cpp releases版本更新地端推論開源開發工具

llama.cpp 發布 v0.8.0，新增 LiquidAI LFM3 模型家族，包括文字 LFM3-1.7B、視覺 LFM3-VL-2B、音訊 LFM3-Audio-2B 與 LFM3-Omni-2B，並配套實驗性流式音訊輸入 API。服務端新增 /v1/realtime 端點，實現 OpenAI realtime 協議，用於即時對話與轉寫，另有 POST /tts 文字轉語音端點，推理努力等級也可通過 API 選擇。

一次更新同時補上 LFM3 音訊/全模態支援與 OpenAI realtime 相容端點，使本地推理能直接對接即時語音應用；對自建語音服務、跟進新模型架構的開發者最有用。

llama.cppLFM3即時音訊GGML本地推理

原標題：v0.8.0
閱讀原文

同一事件共有 4 則報導（1 個來源），看事件全貌

評分82 / 85（平均 83，門檻 60）
狀態精選

全文翻譯

概覽 本版本新增了 LiquidAI LFM3 系列（文字、視覺、音訊和全模態），並在 mtmd 中加入了新的實驗性流式音訊 API，同時通過新的 /v1/realtime 和 /tts 端點將即時音訊的輸入輸出接入伺服器。一批新架構落地：DeepSeek V4.1 Flash、SenseNova U1、Beam、Kolibri-1、Flint、Muse Spark、Granite Switch Decision，以及來自 LiquidAI、百度和 NVIDIA 的決策/嵌入模型。核心部分將稀疏注意力（QSA/DSA）重構為融合掩碼操作，為主機駐留的 MoE 專家引入了 50/50 SLRU 快取，並最佳化了 no_alloc 記憶體處理；伺服器端改進了路由記憶體管理和工具呼叫；UI 現在可以直接從 HF Hub 瀏覽和下載模型、搜尋對話並對其進行壓縮。最後，ggml 升級至 v0.28.0，帶來了 GGML_OP_TOP_K_MASK、-1 專家索引語義以及大量後端效能工作（CUDA 分塊圖啟動、SYCL 圖重放、OpenCL Q2_K/Q3_K、Hexagon HTP 重構、KleidiAI SVE 核心）。 亮點 LFM3 系列：LFM3-1.7B（文字）和 LFM3-VL-2B（視覺），已對照參考 HF 實現驗證 llama/31252 LFM3-Audio-2B 和 LFM3-Omni-2B：單一模型實現 ASR、TTS 和語音聊天，支援流式音訊輸入與輸出 llama/31257 API 變更 DRY 取樣器：新增 LLAMA_DRY_MAX_SEQ_BREAKERS（64），限制不同序列斷點的數量，每個斷點在初始化時開銷為 O(n_vocab) llama/31035 mtmd：新的實驗性 mtmd_audio_stream API，將到達的 PCM 轉換為解碼器可用的嵌入行 llama/30629 mtmd：音訊生成現在是有狀態的——mtmd_gen_state 取代了音訊生成 API 中原始的 state_data 緩衝區 llama/30902 mtmd：mtmd_caps 新增 gen_audio；新增 TTS 型別 SOPRANO 和 LFM3A；對取樣碼引入重複懲罰 llama/28607 llama/31257 mtmd_helper_get_capabilities() 用於從輔助程式碼查詢 mmproj 檔案的能力 llama/31183 伺服器：新的 /v1/realtime 端點，實現 OpenAI 即時協議，用於對話和轉錄 llama/31269 llama/30636 伺服器：新的 POST /tts 端點，用於文字轉語音模型 llama/26603 伺服器：聊天模板的推理努力等級被暴露出來，並可通過 API 選擇 llama/31271 Chat API：支援延遲工具和 tool_reference 內容部分 llama/31256 CLI：--model 或 -hf 現在是必填項 llama/31242 新模型 LiquidAI LFM3-1.7B、LFM3-VL-2B、LFM3-Audio-2B、LFM3-Omni-2B llama/31252 llama/31257 LiquidAI d1-2B 和 d1-omni-2B 決策模型 llama/31275 DeepSeek V4.1 Flash llama/31086 SenseNova U1 llama/28919 Beam（反射） llama/30670 Kolibri-1（Aleph-Alpha） llama/30673 Flint llama/30928 Muse Spark Open（文字、視覺和音訊） llama/30616 Granite Switch Decision，包括 256 路決策頭 llama/30676 llama/30693 Nemotron-3(.5)-Embed，支援非標準 GQA 和 BPE 分詞器 llama/30877 llama/23343 H2O-Lightning-4B 決策模型 llama/30361 MiniCPM-OCR 1.0 和 MiniCPM-OCR-Jev llama/30979 llama/30980 Soprano TTS llama/28607 LFM2 短卷積和 FFN 塊移入共享的 LFM 基礎，供更新的 LFM 模型複用 llama/30874 核心變更 保留在主機記憶體中的 MoE 專家現在使用 50/50 的 SLRU 快取，而不是普通的 LRU 快取 llama/30637 稀疏注意力：融合的 QSA 掩碼操作，QSA/DSA 選擇在沒有 kq_mask 的情況下建置，k 池檢視在層之間共享 llama/30640 llama/31009 llama/30965 llama/31243 llama/30613 MoE：為 laguna 啟用 amax 重縮放，並且在使用它的架構上停用-sm 張量 llama/31147 llama/31081 記憶體：所有記憶體快取中都遵守 no_alloc，從 llama_hparams 中移出，廢棄的 n_pad 被移除 llama/31087 llama/31141 llama/31200 詞彙表載入最佳化；d2t 放置在其 set_rows 使用附近，以便可以留在 GPU 上 llama/30679 llama/30912 分詞器：新的 clean_spaces 和 metaspace 前置方案 GGUF 鍵，混合 DNA 分詞器修復 llama/30940 llama/30825 分詞器：用於 Pixtral 的 Tekken 預分詞器，Tekken/GPT-4o 前瞻中的\p{M}，Nemotron-3.5-Embed 分詞修復 llama/30960 llama/30845 llama/30974 語法：遞迴深度受限（並強制執行）以防止棧溢位；深度 JSON 在遍歷之前被拒絕 llama/29755 llama/31153 llama/30242 Qwen：轉換時合併張量，2D 輸出重塑，Qwen3 雙向注意力和 dense_2 輸出投影 llama/31028 llama/30823 llama/30607 推測解碼：持久化動態 ngram 快取，機率性 DFlash2 草稿，可選的 DFlash FC 輸入縮放 llama/22055 llama/30663 llama/30945 llama/30927 LoRA：支援 prism.hadamard 摺疊權重 llama/30910 張量並行：stepfun 和 laguna 現在支援 4-10 個 GPU llama/24554 GGUF 工具在 split、export-lora 和 merge 中拒絕截斷的張量資料 llama/30497 llama/31044 llama-quant：使用 ggml_nrows()修復 4D 張量量化，並基於名稱的專家檢測 llama/30820 convert-hf：更快的位元組序轉換，支援更多資料型別，pplx-decider 中的純文字檢查點 llama/30647 llama/30575 convert-hf：修復 Granite 語音張量、Granite Switch MultiSwitch 塊數量、Mistral 格式 DeepseekV2 llama/30628 llama/30621 llama/30669 通用：支援 HTTP_PROXY / HTTPS_PROXY，允許併發下載，在真實 HF 倉庫佈局上固定模型解析 llama/21752 llama/29312 llama/30548 多模態變更 通過 mtmd_audio_stream 進行流式音訊輸入，用於支援 voxtral-realtime llama/30629 支援 Soprano TTS 和 MiniCPM-OCR 1.0 llama/28607 llama/30979 影像預處理與 transformers/torchvision 對齊：LFM2 縮放、切片與佈局、Qwen-VL 縮放 llama/30943 llama/30261 llama/30736 LFM2 切片引數從 GGUF 後設資料讀取 llama/30840 lightonocr 預設使用原生解析度，而不是強制最長邊為 1540 llama/30600 build_inp_temporal_merge 移至 mtmd 基類；修復 Gemma-4 FFN 啟用 llama/31065 llama/30882 伺服器變更 在 /v1/realtime 上實現即時對話和轉錄，提交的音訊播放完畢後傳送 response.done llama/31269 llama/30636 llama/30994 為 TTS 模型提供 POST /tts 端點 llama/26603 暴露聊天模板的推理力度級別 llama/31271 路由器：可配置的裝置記憶體餘量，用於動態解除安裝模型 llama/21231 路由器：將客戶端斷開傳播到子任務，停止在休眠/恢復時重複新增 mmproj 餘量 llama/29707 llama/28233 重排序端點接受型別化內容輸入，並替換原始模板中的佔位符 llama/29625 llama/30864 Anthropic /v1/messages：轉發 llama.cpp 取樣引數，並在轉換過程中傳遞 id_slot llama/30856 llama/28554 支援 logprobs 與投機解碼同時使用 llama/27196 修復 n > 1 時 usage.completion_tokens 計數不足，以及 Responses API SSE 事件中缺少 output_index 的問題 llama/29486 llama/30863 修復 SWA 檢查點失效邏輯 llama/30666 新增 --connect 標誌，允許從區域網外部對伺服器進行 p2p 訪問 llama/28277 UI 變更 直接從 HF Hub 發現和下載模型，無組織模型優先列出 llama/29584 llama/31215 新增對話和訊息搜尋對話方塊 llama/31173 對話壓縮，使長聊天保持在上下文範圍內 llama/27401 模型管理器移至獨立路由，模型詳情在抽屜中開啟，無需離開當前頁面 llama/30988 llama/31176 推理選單根據聊天模板的力度級別建置，帶有更清晰的突出顯示切換開關 llama/31273 llama/31212 通過將其轉換為 WAV 來支援 OGG 音訊附件 llama/31237 新增“顯示進行中的工具呼叫”設定 llama/30681 僅在存在未儲存更改時顯示儲存和重置設定按鈕 llama/31248 側邊欄調整大小以及整體佈局/使用者體驗改進 llama/31119 llama/31030 修復：使用“新增系統訊息”後草稿丟失、WebKit 中系統行拉伸 llama/31214 llama/31246 修復：附件文字預覽被裁剪、Markdown 表格和列表在手機上被截斷 llama/30806 llama/30687 ggml 變更 從 v0.27.0 升級到 ggml v0.28.0 新增 GGML_OP_TOP_K_MASK 融合稀疏注意力掩碼，以及用於計算節點上所配置精度的訪問器 在 MUL_MAT_ID / ADD_ID 中，-1 現在是有效的專家索引：該槽位被跳過，輸出行置零 SCALE 接受非連續行檢視；SUM 在 CPU、CUDA、Vulkan 和 WebGPU 上接受非連續輸入 後端：在 CUDA 上通過分塊圖啟動實現融合 MoE top-k 和 gate/up+GLU，在 SYCL 上實現圖記錄/重放 後端：在 OpenCL 上支援 Q2_K/Q3_K 權重和 lightning indexer，在 Vulkan 上支援子組歸約和 src1 精度 更多：重構 Hexagon HTP 後端、在 ZDNN 上實現 flash attention、在 WebGPU 上支援 Q2_0、KleidiAI SVE GEMV/GEMM 核心 資源 每夜建置：b11960 更多資訊 ggml-org 專案的發布與版本管理 自 v0.7.0 以來的變更日誌 8642541 llama.cpp：將版本升級到 0.8.0（#31309） d4f1b18 mtmd：為 LFM3 模型系列啟用影片輸入支援（#31283） d1eb0de server：在 /v1/realtime 中支援工具呼叫（#31287） 5d7793d sync：ggml ec941ee ggml：將版本升級到 v.28.0（ggml/1676） e486a4b ui：根據模板 effort levels 建置推理選單（#31273） f1a7dea tests：調整常量以減少無關警告（#31254） a7af149 cuda：為 384 個專家新增 top-k MoE 融合（#31288） 2864937 vulkan：在 coopmat1 int8 matmul 中將 A 預取限制到 end_k（#30283） a71c00f chat：在模板分隔符之前停止 qwen3-coder 推理和內容（#30557） f4d8af6 ui：僅在相關時顯示儲存和重置設定按鈕（#31248） 02d502f ui：通過將 OGG 音訊附件轉換為 WAV 來支援它們（#31237） b1756b2 ui：修復使用“新增系統訊息”操作後新訊息草稿消失的問題（#31214） 6886628 ui：搜尋對話方塊（#31173） 8ca4060 server：在 /v1/realtime 上新增即時對話和轉錄（#31269） d05706b model：新增 LiquidAI/d1-2B 和 LiquidAI/d1-omni-2B（#31275） 1b4554e server：暴露聊天模板的推理強度級別（#31271） a919179 tests：修復 test-llama-context 中 CI riscv 超時問題（#31265） 338fce6 HIP/CUDA：移除 conv2d 中迭代次數無法在編譯時推斷的迴圈裡的 pragma unroll（#31255） 19ff817 model：新增 LiquidAI/LFM3-Audio-2B 和 LiquidAI/LFM3-Omni-2B（#31257） eb7818c common：支援延遲工具和 tool_reference 內容部分（#31256） 6c44591 snapdragon：將 QDC 製品上傳和日誌下載切換到 v2 API（#31240） 19cce69 model：新增 LiquidAI/LFM3-1.7B 和 LiquidAI/LFM3-VL-2B（#31252） e86f5fd cmake：WASI 不支援 spawn（#30751） 27b7a8a llama：獨立於 kq_mask 形狀建置 k-pool 稀疏選擇（#31243） 7ecb57f rpc：在 recv_msg 中捕獲 std::length_error（#31204） 8542e94 tests：加速張量資料生成和模型儲存（#31145） 922f53e vulkan：通過允許 MUL_MAT 中維度 3 廣播修復 CI llvmpipe（#31239） 0c8c59e mtmd-cli：縮小 batch 指標的作用域（#31155）（#31165） 3ab37ae ui：修復 WebKit 中系統訊息行被拉伸到整個提示高度的問題（#31246） 85cc432 server：為動態解除安裝新增路由器裝置記憶體餘量引數（#21231） 74a1fd3 common：--model 或 -hf 為必需項（#31242） 7eed6b7 model：新增 MiniCPM-OCR-Jev（#30980） 2763785 metal：新增 GGML_OP_TOP_K_MASK（#31230） d78f40d glm5-next：無 kq_mask 的 QSA（+ DSA）（#31009） fae9ac5 ci：為 ROCm 發布建置使用 ccache 依賴模式（#31231） a8276b7 ci：在 ubuntu 26.04 上強制使用 python 3.12（#31222） de5122d ggml：將 scale 放寬到非連續檢視（#31146） 6666a73 tests：修復已訓練模型的多序列拆分重放和部分狀態用例（#31216） 7a2e5ab llama：從記憶體例項中移除過時的 n_pad（#31200） 0d5ca60 ui：在模型選擇器中優先列出無組織模型（#31215） d930fe3 opencl：改進 mul mat 的 -1 索引（#31170） a2b3e12 opencl：初步支援 q3_k（#31172） 823df7d ui：懸停時高亮推理開關並使其與模型選擇器對齊（#31212） 1a97e38 ui：新增對話壓縮（#27401） bc262f3 ui：在不離開頁面的情況下於抽屜中開啟模型詳情（#31176） f5892ca ui：對對話刪除使用共享確認對話方塊（#31187） 7aca0d2 hexagon：新增 GGML_HEXAGON_VTCM_SIZE（#31221） 961c376 llama：為 laguna 啟用 moe amax 重縮放（#31147） 4244ad7 ci：新增 Windows 行為檢查（#31191） - 63b4b30 opencl：在 set_tensor 中寫入 KV-cache 檢視，以免狀態恢復被丟棄（#30471） - caa8191 hexagon：MUL_MAT_ID 行跳過最佳化與改進（#31169） - a469940 llama：將 no_alloc 移出 llama_hparams（#31141） - 4b851fb models：用輸出結構體替換 ggml_tensor**出參（#31142） - 4d71b42 ggml-cpu：將所有型別的 concat 重寫為單行 kernel（#31179） - 70c5f26 vulkan：使用子組歸約的 rms norm 最佳化（#29882） - 5cd62ff sycl：為普通 MMVQ 啟動三元組模板化（#31013） - 38645c2 sycl：為每種量化的 concat 實現模板化（#31008） - 02269b4 sycl：新增 set_tensor_2d 以加快張量分割載入（#30358） - 5320c87 sycl：將重排序臨時記憶體限制在 32 MiB（#31062） - 74b104c common：在真實 HF 倉庫佈局上固定模型解析（#30548） - c6fb3b5 mtmd：新增 mtmd_helper_get_capabilities（#31183） - 8b28c23 ggml-cpu：修復 arm 架構中的未使用變數（#31180） - fd8f845 tests：重構 test-llama-archs，使其不連結內部 libllama 符號（#30880） - 9125a2e ci：修復 webgpu memset 中的競態（#31162） - 9b8446b metal：阻止大矩陣輸出偏移中發生溢位（#30072） - dc9e3be ui：使用者體驗改進（#31030） - 69b3cc4 ui：側邊欄調整大小改進（#31119） - 224d94c ci：在啟動期間提高伺服器超時時間（#31174） - 48c3611 ci：移除 macos-latest-x64 任務（#31175） - a074215 deepseekv4.1：後續評審修復（#31137） - d854d98 cuda：使用 std::string 代替 VLA 作為 archName（#30463） - 48c9f08 common、grammar：新增遞迴深度限制，並將其作為一項要求（#31153） - 6050b46 tests：調整 test-llama-context 的預設執行緒數（#31157） - 2280f0f qwen：在轉換時合併張量（#31028） - 1a0b56c ci：在 webgpu 中停用 fa 關閉的 test-llama-context 測試（#30977） - 63412d8 llama：修復 dry 序列中斷器導致的 CPU 和記憶體耗盡問題（#31035） - 5c965fd llama-quant：使用 ggml_nrows()和基於名稱的專家檢測修復 4D 張量量化（#30820） - f1168ac grammar：限制遞迴深度以防止棧溢位（#29495）（#29755） - fa3645f llama：在所有記憶體快取中遵循 no_alloc（#31087） - 7790a92 tools：在合併時拒絕截斷的 GGUF 張量資料（#31044） - c49edf0 CUDA：在 MMQ 中融合 FFN gate/up 矩陣乘法與 GLU，使用交錯的 gate/up 行（#29948） - a55c16e ggml-cuda：融合 qwen MRoPE + set-rows（#31025） - e321bda vulkan：為 conv3d 新增更大的分塊大小（僅 coopmat2）（#29920） - 459a1ba vulkan：小 matmul 瓦片中每個 subgroup 一個 warp tile（#30106） - f0fb598 hexagon：用於一元、二元和 matmul 運算的通用維度摺疊/重塑（#31071） - 9a4580b opencl：新增 topk 和 lightning indexer（#31026） - b083c38 opencl：新增初始 q2_k 支援（#30999） - 344be0a ci：更新 CPU 作業分配（#31136） - b3fa699 chat：在遍歷 JSON 前拒絕巢狀超過 128 層的 JSON（#30242） - 74bca39 model：DSV4.1 flash（#31086） - d327bf1 jinja：修復 select 和 selectattr 引數處理（#31059） - 78a8ae1 hexagon：用於多核 HMX flash attention 的 KV-head 團隊拆分（#31055） - f3269ef server：重構 server 入口點（#31113） - 3f03b17 metal：移除 mul_mm_id amax 重新縮放（#30924） - 9a0fde9 rpc：在 rdma_recv 中新增邊界檢查（#30399） - e168e04 ui：將 Models Manager 作為 Route + UI 改進（#30988） - ea1d7d6 fix(ui)：使懸停卡片和上下文彈出框的樣式與下拉選單對齊（#30978） - 71255fe ci：將每個 runner 的結果目錄傳遞給 T4 runner 上的自託管作業（#31100） - 73adf89 minimax-m3：將量化縮放轉發到圖中（修復 #31063）（#31097） - df9beac tests：修復大 n_batch 下的 test-llama-context（#31083） - 4d32b29 ci：從 macos-latest-arm64 中排除 test-speculative-rejection-sampling（#31088） - 3032209 llama：對具有 moe amax 重新縮放的架構拒絕 -sm tensor（#31081） - 701fd25 ggml：允許 MUL_MAT_ID

由 AI 翻譯，以原文為準。

原文
Overview This release adds the LiquidAI LFM3 family (text, vision, audio and omni) together with a new experimental streaming-audio API in mtmd , and wires realtime audio in and out of the server through the new /v1/realtime and /tts endpoints. A batch of new architectures landed: DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, Granite Switch Decision, plus decision/embedding models from LiquidAI, Baidu and NVIDIA. The core got sparse-attention (QSA/DSA) reworked onto a fused masking op, a 50/50 SLRU cache for host-resident MoE experts, and cleaner no_alloc memory handling; the server got router memory-management and tool-calling improvements; the UI can now browse and download models straight from HF Hub, search conversations and compact them. Finally, ggml was bumped to v0.28.0, bringing GGML_OP_TOP_K_MASK , -1 expert-index semantics and large backend performance work (CUDA chunked graph launches, SYCL graph replay, OpenCL Q2_K/Q3_K, Hexagon HTP rework, KleidiAI SVE kernels). Highlights LFM3 family: LFM3-1.7B (text) and LFM3-VL-2B (vision), verified against the reference HF implementation llama/31252 LFM3-Audio-2B and LFM3-Omni-2B: ASR, TTS and voice chat from a single model, with streaming audio input and output llama/31257 API changes DRY sampler: new LLAMA_DRY_MAX_SEQ_BREAKERS (64) caps distinct sequence breakers, each costing O(n_vocab) at init llama/31035 mtmd : new experimental mtmd_audio_stream API turning PCM fed as it arrives into decoder-ready embedding rows llama/30629 mtmd : audio generation is now stateful - mtmd_gen_state replaces the raw state_data buffers of the audio gen API llama/30902 mtmd : mtmd_caps gained gen_audio ; new TTS types SOPRANO and LFM3A ; repetition penalty for sampled codes llama/28607 llama/31257 mtmd_helper_get_capabilities() to query the capabilities of an mmproj file from helper code llama/31183 Server: new /v1/realtime endpoint implementing the OpenAI realtime protocol for conversation and transcription llama/31269 llama/30636 Server: new POST /tts endpoint for text-to-speech models llama/26603 Server: reasoning effort levels of the chat template are exposed and selectable via the API llama/31271 Chat API: support for deferred tools and tool_reference content parts llama/31256 CLI: --model or -hf is now required llama/31242 New models LiquidAI LFM3-1.7B, LFM3-VL-2B, LFM3-Audio-2B, LFM3-Omni-2B llama/31252 llama/31257 LiquidAI d1-2B and d1-omni-2B decision models llama/31275 DeepSeek V4.1 Flash llama/31086 SenseNova U1 llama/28919 Beam (Reflection) llama/30670 Kolibri-1 (Aleph-Alpha) llama/30673 Flint llama/30928 Muse Spark Open (text, vision and audio) llama/30616 Granite Switch Decision, including the 256-way decision head llama/30676 llama/30693 Nemotron-3(.5)-Embed, with non-standard GQA and BPE tokenizer support llama/30877 llama/23343 H2O-Lightning-4B decision model llama/30361 MiniCPM-OCR 1.0 and MiniCPM-OCR-Jev llama/30979 llama/30980 Soprano TTS llama/28607 LFM2 short-conv and FFN blocks moved into a shared LFM base reused by newer LFM models llama/30874 Core changes MoE experts kept in host memory now use a 50/50 SLRU cache instead of a plain LRU cache llama/30637 Sparse attention: fused QSA mask op, QSA/DSA selection built without kq_mask , k-pool views shared between layers llama/30640 llama/31009 llama/30965 llama/31243 llama/30613 MoE: amax rescaling enabled for laguna , and -sm tensor denied on arches that use it llama/31147 llama/31081 Memory: no_alloc respected in all memory caches, moved out of llama_hparams , obsolete n_pad dropped llama/31087 llama/31141 llama/31200 Vocabulary loading optimized; d2t placed next to its set_rows use so it can stay on the GPU llama/30679 llama/30912 Tokenizer: new clean_spaces and metaspace prepend-scheme GGUF keys, hybrid DNA tokenizer fixes llama/30940 llama/30825 Tokenizer: Tekken pre-tokenizer for Pixtral, \p{M} in Tekken/GPT-4o lookaheads, Nemotron-3.5-Embed tokenization fix llama/30960 llama/30845 llama/30974 Grammar: recursion depth limited (and enforced) to prevent stack overflow; deep JSON rejected before walking it llama/29755 llama/31153 llama/30242 Qwen: merge tensors on convert, 2D output reshape, Qwen3 bidirectional attention and dense_2 output projection llama/31028 llama/30823 llama/30607 Speculative decoding: persist the dynamic ngram cache, probabilistic DFlash2 drafts, optional DFlash FC input scale llama/22055 llama/30663 llama/30945 llama/30927 LoRA: support for prism.hadamard folded weights llama/30910 Tensor parallel: stepfun and laguna now support 4-10 GPUs llama/24554 GGUF tools reject truncated tensor data in split , export-lora and merge llama/30497 llama/31044 llama-quant : fix 4D tensor quantization with ggml_nrows() and name-based expert detection llama/30820 convert-hf : faster endian conversion with more datatypes, text-only checkpoints in pplx-decider llama/30647 llama/30575 convert-hf : fixes for Granite speech tensors, Granite Switch MultiSwitch block count, Mistral-format DeepseekV2 llama/30628 llama/30621 llama/30669 Common: honor HTTP_PROXY / HTTPS_PROXY , allow concurrent downloads, pin model resolution on real HF repo layouts llama/21752 llama/29312 llama/30548 Multi-modality changes Streaming audio input via mtmd_audio_stream , used to support voxtral-realtime llama/30629 Soprano TTS support and MiniCPM-OCR 1.0 support llama/28607 llama/30979 Image preprocessing aligned with transformers/torchvision: LFM2 resize, tiles and layout, Qwen-VL resize llama/30943 llama/30261 llama/30736 LFM2 tiling parameters are read from the GGUF metadata llama/30840 lightonocr defaults to native resolution instead of forcing a 1540 longest edge llama/30600 build_inp_temporal_merge moved to the mtmd base class; fix Gemma-4 FFN activation llama/31065 llama/30882 Server changes Realtime conversation and transcription on /v1/realtime , with response.done sent after the committed audio drains llama/31269 llama/30636 llama/30994 POST /tts endpoint for TTS models llama/26603 Expose the reasoning effort levels of the chat template llama/31271 Router: configurable device memory margin for dynamic unloading of models llama/21231 Router: propagate client disconnects to the child task, stop re-adding the mmproj margin on sleep/resume llama/29707 llama/28233 Rerank endpoint accepts typed content input and substitutes the placeholder from the original template llama/29625 llama/30864 Anthropic /v1/messages : forward llama.cpp sampling params and pass id_slot through the conversion llama/30856 llama/28554 Support logprobs together with speculative decoding llama/27196 Fix usage.completion_tokens undercounting with n > 1 and missing output_index in Responses API SSE events llama/29486 llama/30863 Fix SWA checkpoint invalidation logic llama/30666 New --connect flag to allow p2p access to the server outside the LAN llama/28277 UI changes Discover and download models directly from HF Hub, with org-less models listed first llama/29584 llama/31215 New search dialog for conversations and messages llama/31173 Conversation compaction to keep long chats within the context llama/27401 Models manager moved to its own route, model details open in a drawer without leaving the page llama/30988 llama/31176 Reasoning menu built from the chat template's effort levels, with a clearer highlighted toggle llama/31273 llama/31212 Support OGG audio attachments by converting them to WAV llama/31237 New "Show tool call in progress" setting llama/30681 Save and reset settings buttons shown only when there are unsaved changes llama/31248 Sidebar resizing and general layout/UX improvements llama/31119 llama/31030 Fixes: draft lost after "Add System Message", system row stretching in WebKit llama/31214 llama/31246 Fixes: clipped attachment text preview, markdown tables and lists cut off on phones llama/30806 llama/30687 ggml changes Bumped from v0.27.0 to ggml v0.28.0 New GGML_OP_TOP_K_MASK fused sparse-attention mask, plus accessors for the precision configured on a compute node -1 is now a valid expert index in MUL_MAT_ID / ADD_ID : the slot is skipped and the output row zeroed SCALE accepts non-contiguous row views; SUM accepts non-contiguous inputs on CPU, CUDA, Vulkan and WebGPU Backends: fused MoE top-k and gate/up+GLU with chunked graph launches on CUDA, graph record/replay on SYCL Backends: Q2_K/Q3_K weights and a lightning indexer on OpenCL, subgroup reductions and src1 precision on Vulkan More: restructured Hexagon HTP backend, flash attention on ZDNN, Q2_0 on WebGPU, KleidiAI SVE GEMV/GEMM kernels Assets Nightly build: b11960 More info Releases and versioning of ggml-org projects Changelog since v0.7.0 8642541 llama.cpp : bump version to 0.8.0 ( #31309 ) d4f1b18 mtmd: enable video input support for LFM3 model family ( #31283 ) d1eb0de server : support tool calls in /v1/realtime ( #31287 ) 5d7793d sync : ggml ec941ee ggml : bump version to v.28.0 (ggml/1676) e486a4b ui: build the reasoning menu from the template effort levels ( #31273 ) f1a7dea tests : adjust constants to reduce irrelevant warnings ( #31254 ) a7af149 cuda : add top-k MoE fusion for 384 experts ( #31288 ) 2864937 vulkan: clamp A prefetch to end_k in coopmat1 int8 matmul ( #30283 ) a71c00f chat : stop qwen3-coder reasoning and content before template separators ( #30557 ) f4d8af6 ui : show save and reset settings buttons only when relevant ( #31248 ) 02d502f ui: Support OGG audio attachments by converting them to WAV ( #31237 ) b1756b2 ui: Fix draft for new message disappearing after using "Add System Message" action ( #31214 ) 6886628 ui : Search dialog ( #31173 ) 8ca4060 server : add realtime conversation and transcription on /v1/realtime ( #31269 ) d05706b model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B ( #31275 ) 1b4554e server: expose the reasoning effort levels of the chat template ( #31271 ) a919179 tests: fix CI riscv timeout in test-llama-context ( #31265 ) 338fce6 HIP/CUDA: remove pragma unroll from loops in conv2d where itteration count cant be infered at compile time ( #31255 ) 19ff817 model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B ( #31257 ) eb7818c common : support deferred tools and tool_reference content parts ( #31256 ) 6c44591 snapdragon: switch QDC artifact upload and log download to v2 API ( #31240 ) 19cce69 model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B ( #31252 ) e86f5fd cmake: WASI doesn't support spawning ( #30751 ) 27b7a8a llama : build the k-pool sparse selection independently of the kq_mask shape ( #31243 ) 7ecb57f rpc : catch std::length_error in recv_msg ( #31204 ) 8542e94 tests : speed up tensor data generation and model saving ( #31145 ) 922f53e vulkan: fix CI llvmpipe by allowing dim 3 broadcast in MUL_MAT ( #31239 ) 0c8c59e mtmd-cli: Reduced scope of batch pointer ( #31155 ) ( #31165 ) 3ab37ae ui: fix system message row stretching to full prompt height in WebKit ( #31246 ) 85cc432 server: add router device memory margin parameter for dynamic unloading ( #21231 ) 74a1fd3 common : --model or -hf is required ( #31242 ) 7eed6b7 model: Add MiniCPM-OCR-Jev ( #30980 ) 2763785 metal : add GGML_OP_TOP_K_MASK ( #31230 ) d78f40d glm5-next : QSA without kq_mask (+ DSA) ( #31009 ) fae9ac5 ci : use ccache depend mode for ROCm release builds ( #31231 ) a8276b7 ci : force python 3.12 on ubuntu 26.04 ( #31222 ) de5122d ggml : relax scale to non-contiguous views ( #31146 ) 6666a73 tests : fix multi-seq split replay and partial-state cases for trained models ( #31216 ) 7a2e5ab llama : remove obsolete n_pad from memory instances ( #31200 ) 0d5ca60 ui: list org-less models first in the model selector ( #31215 ) d930fe3 opencl: refine mul mat id -1 index ( #31170 ) a2b3e12 opencl: initial support for q3_k ( #31172 ) 823df7d ui: Highlight reasoning toggle on hover and align it with model selector ( #31212 ) 1a97e38 ui: add conversation compaction ( #27401 ) bc262f3 ui: Open model details in a drawer without leaving the page ( #31176 ) f5892ca ui: use shared confirmation dialog for conversation deletion ( #31187 ) 7aca0d2 hexagon: add GGML_HEXAGON_VTCM_SIZE ( #31221 ) 961c376 llama : enable moe amax rescaling for laguna ( #31147 ) 4244ad7 ci : add windows behavior check ( #31191 ) 63b4b30 opencl: write KV-cache views in set_tensor so a state restore is not dropped ( #30471 ) caa8191 hexagon: MUL_MAT_ID row skipping optimizations and improvements ( #31169 ) a469940 llama : move no_alloc out of llama_hparams ( #31141 ) 4b851fb models : replace ggml_tensor** out-params with output structs ( #31142 ) 4d71b42 ggml-cpu : rewrite concat as one row kernel for all types ( #31179 ) 70c5f26 vulkan: rms norm optimization using subgroup reductions ( #29882 ) 5cd62ff sycl: template the plain MMVQ launch triples ( #31013 ) 38645c2 sycl: template the per-quant concat impls ( #31008 ) 02269b4 sycl: add set_tensor_2d for faster tensor-split loading ( #30358 ) 5320c87 sycl : bound reorder temp memory to 32 MiB ( #31062 ) 74b104c common: pin model resolution on real HF repo layouts ( #30548 ) c6fb3b5 mtmd: add mtmd_helper_get_capabilities ( #31183 ) 8b28c23 ggml-cpu : fix unused variables in arm arch ( #31180 ) fd8f845 tests : refactor test-llama-archs to not link internal libllama symbols ( #30880 ) 9125a2e ci : fix race in webgpu memset ( #31162 ) 9b8446b metal : stop overflow from happening in large matrix output offsets ( #30072 ) dc9e3be ui: UX improvements ( #31030 ) 69b3cc4 ui: Sidebar resizing improvements ( #31119 ) 224d94c ci : bump server timeout during startup ( #31174 ) 48c3611 ci : drop the macos-latest-x64 job ( #31175 ) a074215 deepseekv4.1: follow-up review fixes ( #31137 ) d854d98 cuda : use std::string instead of VLA for archName ( #30463 ) 48c9f08 common, grammar: add recursive depth limit, make it a requirement ( #31153 ) 6050b46 tests : adjust default threads for test-llama-context ( #31157 ) 2280f0f qwen: merge tensors on convert ( #31028 ) 1a0b56c ci : disable test-llama-context with fa off in the webgpu ( #30977 ) 63412d8 llama: fix exhausting CPU and MEM caused by dry sequence breaker ( #31035 ) 5c965fd llama-quant : fix 4D tensor quantization using ggml_nrows() and name-based expert detection ( #30820 ) f1168ac grammar: limit recursion depth to prevent stack overflow ( #29495 ) ( #29755 ) fa3645f llama : respect no_alloc in all memory caches ( #31087 ) 7790a92 tools : reject truncated GGUF tensor data in merge ( #31044 ) c49edf0 CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows ( #29948 ) a55c16e ggml-cuda: fuse qwen MRoPE + set-rows ( #31025 ) e321bda vulkan: Add a larger tile size for conv3d (coopmat2 only) ( #29920 ) 459a1ba vulkan : one warp tile per subgroup in the small matmul tiles ( #30106 ) f0fb598 hexagon: generic dims collapse/reshape for unary, binary and matmul ops ( #31071 ) 9a4580b opencl: add topk and lightning indexer ( #31026 ) b083c38 opencl: add initial q2_k support ( #30999 ) 344be0a ci : update CPU job assignments ( #31136 ) b3fa699 chat : reject JSON nested more than 128 levels before walking it ( #30242 ) 74bca39 model: DSV4.1 flash ( #31086 ) d327bf1 jinja : fix select and selectattr argument handling ( #31059 ) 78a8ae1 hexagon: KV-head team split for multicore HMX flash attention ( #31055 ) f3269ef server: refactor server entrypoint ( #31113 ) 3f03b17 metal : remove mul_mm_id amax rescaling ( #30924 ) 9a0fde9 rpc : add bounds check in rdma_recv ( #30399 ) e168e04 ui: Models Manager as Route + UI improvements ( #30988 ) ea1d7d6 fix(ui): align hover card and context popup styling with dropdown menu ( #30978 ) 71255fe ci : pass per-runner results dir to self-hosted jobs on T4 runners ( #31100 ) 73adf89 minimax-m3 : forward quant scales to the graph (fixes #31063 ) ( #31097 ) df9beac tests : fix test-llama-context for large n_batch ( #31083 ) 4d32b29 ci : exclude test-speculative-rejection-sampling from macos-latest-arm64 ( #31088 ) 3032209 llama : deny -sm tensor for arches with moe amax rescaling ( #31081 ) 701fd25 ggml : allow MUL_MAT_ID -1 index to skip the computation ( #26631 ) 3642238 tests : add speculative rejection draft sampling test ( #30213 ) 6b9ba4f ci : fix data race between runners sharing the model cache ( #31046 ) 8e5a58d vulkan: ggml src1 precision support ( #30656 ) 1ddffe9 HIP: add byte permute fallback when __builtin_amdgcn_perm is unavailable ( #30915 ) 7894e8b mtmd: move build_inp_temporal_merge to base class ( #31065 ) edc3538 mtmd : lfm2 resize matches torchvision ( #30943 ) 9ae73e6 mtmd : add Soprano TTS support ( #28607 ) 0ae0e19 hexagon: add SGN op ( #30346 ) 21ec307 common : allow concurrent downloads across processes ( #29312 ) 9145ddb hip: fix bf16 WMMA operand vector type for gfx11/gfx12 ( #30885 ) 1a24df1 ggm-cuda: Enforce DP4A MMQ tile height multiple of warp size ( #30837 ) 9ff0d7e CUDA: fix build after chunked CUDA graphs ( #30804 ) ( #31053 ) 06230cb allows the usage of 4-10 gpus for stepfun and laguna in tensor parallel ( #24554 ) 5c31142 qwen4exp : build the QSA selection without the kq_mask ( #30965 ) da2ee1e vendor : update LibreSSL to 4.4.0 [no ci] ( #31049 ) 280141e CUDA: chunk graphs to amortize graph launch ( #30804 ) 36e36e6 hexagon: add CEIL op ( #31006 ) e1e3b2c ui : bump packages to clear npm audit ( #31038 ) 4231b23 ggml : fix time overflow in ggml_time_us ( #30969 ) f00c908 ci : add behavior check workflow ( #31036 ) f5027fb CANN: missing const qualifiers added for consistency ( #30933 ) f64a498 sycl : pass device into mul_mat_uses_onednn ( #31014 ) 1ab09d6 Extend probabilistic sampling for DFlash2 drafts ( #30663 ) f9599d0 vocab : fix Nemotron-3.5-Embed tokenization ( #30974 ) 643b0cd hexagon: Op Fusion refactoring, optimizations and new fusions like BINARY_NX (back-to-back ADDs, etc) ( #30843 ) 0a9ef00 sycl: Replace host waits in fattn-mkl with explicit dependencies ( #30684 ) 7377bad sycl: Add graph record and replay ( #28725 ) 184f653 server: Substitute rerank placeholder from the original template only. ( #30864 ) d814cbb server: realtime API: send complete after committed audio drains ( #30994 ) 970b51c opencl: add depthwise conv2d ( #30900 ) a08ab75 mtmd : add MiniCPM-OCR 1.0 support ( #30979 ) 19ab5a5 clef : include for std::find and std::copy ( #30981 ) 36cbf41 Fix !30793 5c19a5a glm5 : optimize inputs ( #30932 ) d9ee007 model : fix qwen4exp layer inputs for DFlash drafters ( #30927 ) d81d0d8 Multi-column on Intel XMX matrix engines refactored ( #29864 ) d772569 vulkan: replace std::aligned_alloc with posix_memalign for termux compatibility ( #30828 ) 0693dbf nemotron : support non-standard GQA, BPE tokenizer and domyn-small pre-type ( #23343 ) a5d2227 server : support typed content input for RERANK endpoint ( #29625 ) 5f5acfe ci : disable slow WebGPU tests ( #30963 ) 1f4c2ee ci : be able to reconfigure HTTP port for server pytest workers ( #30972 ) 7170cad sycl: fix CONV_2D wrong results for non-contiguous input ( #30363 ) d88040c vocab : add clean_spaces GGUF key ( #30940 ) e197cef vocab : use Tekken pre-tokenizer for pixtral rather than the Llama 3 one ( #30960 ) bf039c8 models : Record layer inputs for all granite models ( #30957 ) dfeea52 kleidiai : add Q8_0 SVE dotprod GEMV and SVE i8mm GEMM kernels ( #30808 ) e3797b1 sycl: merge the fp16 and fp32 dequant switches ( #30740 ) 0b1281a webgpu : add Q2_0 support ( #30914 ) 4912ac2 dflash : load optional FC input scale ( #30945 ) 84746c5 ui: Add "Show tool call in progress" setting ( #30681 ) 0682a57 lora : support prism.hadamard folded weights ( #30910 ) a6b83d4 ui : key draft sidecar badges by model instead of repo ( #30919 ) c7fc509 opencl: add F32-to-I32 ARGMAX ( #30897 ) a37cc46 models : add Flint model (FlintForCausalLM) ( #30928 ) 99fb997 jinja : fix namespace attributes shadowed by dict methods ( #30941 ) 7775ffb model: Add support for SenseNova U1 model ( #28919 ) d454a20 llama: create new fused op for QSA mask ( #30640 ) 0756e83 sycl: q8 fast dequant ( #30775 ) 9febe63 jinja: support hexadecimal string escapes ( #30929 ) 9b56ccf ggml-opencl : add IM2COL_3D support ( #30895 ) 0a1ef30 mtmd: add state object for mtmd_gen ( #30902 ) 261700b server : forward llama.cpp sampling params on /v1/messages ( #30856 ) f849095 vendor: update BoringSSL to 0.20261020.0 ( #30889 ) 824eaec llama : optimize vocabulary loading ( #30679 ) eb182d8 llama : place d2t by its set_rows use so it can stay on the GPU ( #30912 ) ea9e344 metal : fix memory leaks and cleanup logic in GPU capture path ( #30890 ) 0efe8e4 hexagon: neon-vectorized and threaded repack (tiling) ( #30901 ) b531e4c opencl: add dp4a GEMM variants to vector load activation ( #30753 ) 74b115c sycl : fix fa crash for Query head dimension mismatch (neq0 != HSK / D=2048/4096) ( #30286 ) 96caa80 sycl: collapse duplicate need_check launch bodies in mmq ( #30739 ) 479a527 ggml-cpu : compute f32 gelu_quick and geglu_quick without the fp16 table ( #30737 ) 8baabd5 llama : scale moe down projection per expert token ( #30861 ) 493253f Sycl iq3 crash fix ( #30604 ) 339f324 opencl: fix rope program build error with newer Clang ( #30893 ) 9a3b91c server : fix usage.completion_tokens undercount with n > 1 ( #29486 ) 5c0b49a server : add POST /tts endpoint for TTS models ( #26603 ) 1242be6 server: add realtime transcription API ( #30636 ) 84e66d0 refactor : move LFM2 shortconv and FFN blocks into a shared LFM base ( #30874 ) 52ffa02 model : support Nemotron-3(.5)-Embed ( #30877 ) 839a572 qwen: reshape output to 2d instead of 3d ( #30823 ) ead9f32 server: Support logprobs with speculative decoding ( #27196 ) b684b49 hexagon: fix VTCM size query on images that need layout out-params ( #30655 ) 4a45de4 mtmd : lfm2 tiles and layout match transformers ( #30261 ) 7e3eeee model : add H2O-Lightning-4B decision model ( #30361 ) 5ba3b9f mtmd: fix gemma 4 ffn act ( #30882 ) 5a9a34d server : avoid repeated mmproj memory margin addition on sleep/resume ( #28233 ) 39da130 ggml-cuda : support ssm_conv kernel size 7 ( #30873 ) 4e20138 server: Router wasn't propagating the client disconnects fully to the child, the response wait could cause a delayed cancellation ( #29707 ) 1ba568c server: fix missing output_index in Responses API SSE events for compatibility ( #30863 ) 36140db mtmd : read LFM2 tiling params from GGUF metadata ( #30840 ) df11702 metal : optimize argmax and pool_1d ( #30862 ) 9df0b73 model : drop the redundant precision overrides in the Kolibri-1 router ( #30865 ) b38960f ci : remove superfluous webgpu jobs ( #30868 ) a01ce9f opencl: add several gemm variants for small batches and some fixes ( #30850 ) 8115b4c sycl : replace memset due to not always ordered ( #30812 ) 119f68d ggml-cpu : select the NEON fp32->fp16 conversion ( #30505 ) b684498 HIP/CUDA: Fix reduction in mmf getting tile widths not devisiable by 64 ( #30836 ) 522f2b6 ggml-zdnn: introduce flash attention op ( #30512 ) 83c2d01 spec : save the dynamic ngram cache file ( #22055 ) 13786fb CUDA: Clear CUDA graphs after legacy memory pool flush ( #26771 ) d806193 server : pass id_slot through the Anthropic /v1/messages conversion ( #28554 ) 704366d metal : fix missing MoE kernel variant for 9 experts per token ( #30610 ) ( #30699 ) 707616f ci: Fix --strict mode for API check ( #30623 ) d8d9297 vocab : add \p{M} to lookaheads in Tekken and GPT-4o pre-tokenizer regexes ( #30845 ) 7879491 tests : make MoE router selection independent of the backend noise ( #30818 ) 8466395 hexagon: update to snapdragon-toolchain v0.8 and enable hexagon v85 ( #30854 ) dd002b9 sycl: fix compiler warnings - missing prototypes, dead code, extra semicolons ( #30271 ) 1d88cb0 ui: hide the native clear button in SearchInput ( #30719 ) 3bb63cb sycl : fix get_rows on reordered quantized weights ( #30430 ) f711c8f mtmd : match the transformers Qwen-VL resize ( #30736 ) 921257f ci, docs: add WASI CI and docs ( #30754 ) 3ffb629 ci : re-enable type/lint-check for skills ( #30713 ) 7adf160 server: add --connect, allow p2p access to server outside LAN ( #28277 ) ab7ed0d ui : fix attachment text preview clipped when a line is longer than the container ( #30806 ) 4c3c581 hybrid dna tokenizer fixes ( #30825 ) 1b29621 opencl: add cooperative-k dp4a gemm for q4_0, q4_k and q6_k ( #30741 ) 6fa4f6c opencl: add get_rows for i32, cpy for i32 and f32 ( #30756 ) 0ce8cd1 jinja : implement unique filter ( #30819 ) f174553 server : fix swa checkpoint invalidation logic ( #30666 ) 7951f60 vendor : update cpp-httplib to 0.62.0 ( #30785 ) 33d6135 moe-cache : replace MoE LRU cache with a 50/50 SLRU cache ( #30637 ) 098ce52 metal : fix moe_reduce fusion race when dst aliases its inputs ( #30795 ) f3fb2fc cann : refactor #ifdef blocks to avoid unreachable code after return ( #25273 ) fda00f1 tests: add big-endian conversion tests ( #30705 ) 4866db3 CUDA: fix warning for nonexistent extra semicolon ( #30774 ) 42b7003 ggml-cpu: vectorize ggml_v_silu and ggml_v_expf ( #30791 ) 4c1004c cuda: cap MMQ Stream-K blocks at available work ( #30553 ) 412a363 qwen3 : bidirectional attention and dense_2 output projection ( #30607 ) 2b57148 ui: Fix markdown tables and lists being cut off on phones ( #30687 ) 8f97d82 tools : reject truncated GGUF tensor data in split and export tools ( #30497 ) 27af581 tests : update Metal fusion baseline for beam ( #30764 ) 7db0d3d pi : turn .pi/gg into a system prompt addendum ( #30762 ) 99db719 CUDA: fix shared-memory WAR hazard in flash_attn_ext_vec ( #30573 ) 31288c9 hexagon: FLASH_ATTN_EXT improvements for longer context and SWA models ( #30731 ) fe8841a hexagon: allow multistream FA ( #30218 ) ff5a233 cuda : fuse qwen4exp's SCALE -> SIGMOID -> SCALE -> hc_post chain ( #30602 ) 0878f7c ggml: support non-contiguous SUM (CPU, CUDA, Vulkan, WebGPU) ( #28851 ) 5aeac37 ggml-cuda: fattn do not check oob if k_VKQ_max = (k_VKQ_max - nbatch_fa) ( #30572 ) 547208d Restore ESIMD kernels on FFN path ( #30239 ) 848bbda hexagon: add support for hadamard MUL_MAT and GET_ROW for flat and tiled Q1_0 and Q2_0 ( #30627 ) 797de63 hexagon: add support for Q1_0 and Q2_0 ( #30122 ) 9d1878a ggml-cuda: HIP: use __builtin_amdgcn_perm for q1_0 unpacking ( #29927 ) c0fd786 ui : Discover & Download Models from HF Hub ( #29584 ) 2744e48 CUDA: faster MMVQ for low-bit types ( #30517 ) 10b8b75 tests: give the random model generator a per-tensor filler table ( #30674 ) 9d1e036 cuda : fix -Wtautological-compare in fattn vec dispatch ( #30519 ) f2f667b cuda : add grouped experts top-k fusion ( #29181 ) 62318cd model : add Kolibri-1 (Aleph-Alpha) support ( #30673 ) 83f9389 ci : added ABI reports as github artifact ( #30619 ) 1268eda model : Granite switch decision 256 options ( #30693 ) f74e2fa sycl: fix prompt-processing regression after Q4_0/Q8_0 weight reorder ( #30374 ) ea0c4c2 skills: add slice-model ( #30677 ) 6420cf5 ci: update device matrix for Snapdragon jobs by removing SM8750 ( #30620 ) 27c7a76 hexagon: vectorize softplus with HVX ( #30435 ) 2f2763d opencl: Add MxFP4 MoE bin kernel for Adreno X1 devices ( #30691 ) 01fc3aa model: add Beam (Reflection) ( #30670 ) 23f7aab model : Granite Switch Decision ( #30676 ) 7182ef7 common: honor HTTP_PROXY/HTTPS_PROXY env vars in http client ( #21752 ) 2a66703 convert : respect Mistral format for DeepseekV2Model ( #30669 ) f336359 Add more benchmarks to llama-eval ( #26174 ) ceefd16 hexagon: correct HMX FA sinks ( #30581 ) 9a16f1d model : MTP support for Mellum2.1 ( #30612 ) b84ea5c convert : speed up model endian conversion and add more datatype support ( #30647 ) a30fb49 [SYCL]: fix memory errors in mul_mat, split buffer, host pool ( #29889 ) 495c580 gguf-py : add numpy Q1_0 and Q2_0 de/quantization support ( #30630 ) 623c163 CUDA: HIP: rms_norm: add variants that load float4 at a time and store x in registers ( #29720 ) 13b8048 common : read ngram cache parts by reference instead of copying ( #30061 ) 0f43762 convert : support text-only checkpoints in pplx-decider ( #30575 ) 3dbcb7f cpu : reduce AVX2 tiled accumulators together ( #30296 ) c804754 ggml-cpu: refactor s309x repack helper functions ( #30588 ) c0083e3 sycl : reduce the build threads to half ( #30236 ) 13de11e sycl: wide stores for the q4_K/q5_K weight dequant ( #29696 ) 88aaf32 tests : catch vanished rows in the fusion baseline ( #30618 ) a8d9a7e vulkan: fix gpu-visible host memory issues ( #30468 ) f12aaa3 Fix failing RocM tests ( #30611 ) 5e03a1f graph : add precision handling for MELLUM architecture ( #30584 ) b797b9c hexagon: tensor repacking rework and support for tiled and flat quantized and fp weights ( #30635 ) 6afa9a3 llama : share the k-pool input views between layers ( #30613 ) b6334e3 Limit max CUDA pinned host buffer size to 4 GB ( #30449 ) e707790 sycl: support reordered Q8_0 MoE weights ( #29337 ) 43d57fb [SYCL] Enable ssl in build script and CI ( #30456 ) 2851b3b mtmd: add mtmd_audio_stream + support voxtral-realtime ( #30629 ) 59d5fad convert : fix Granite Switch block_count for MultiSwitch checkpoints ( #30621 ) e287bd8 convert : fix regression in mapping of Granite speech tensors ( #30628 ) 2f729cf mtmd: lightonocr does not require longest edge to always be 1540, native resolution is the default ( #30600 ) ba3c05f model: add Muse Spark Open (text, vision and audio) ( #30616 ) b76b133 pi : expand AI disclosure with hardware and stats ( #30615 )

相關報導

llama.cpp releasesAI 評分46

llama.cpp 新增 LiquidAI d1-omni-600M 決策模型

llama.cpp 建置 b11488 新增 LiquidAI/d1-omni-600M 決策模型支援，服務端可通過 images 與 input_audio 直接接收音訊輸入，模型與多模態輸入打通。同時決策型別 d1omni 更名為 lfm2-d1-omni，images 成為 files 的別名，無記憶的 lfm2 被限定為非因果模式，CUDA 後端保留 conformer 的 GLU sigmoid。四筆改動均標註由 Claude Opus 5.5 協助完成；

地端推論開源llama.cppLiquidAId1-omni-600M多模態本地推理
2 個來源 · 2 則報導閱讀原文
Ollama releases● 精選AI 評分72

Ollama 上線 Liquid AI LFM3 小模型家族，1.7B 支援 256K 上下文

Ollama v0.40.12 上線 Liquid AI 的 LFM3 小模型家族，面向本地 agent、工具呼叫與結構化輸出。1.7B 的 lfm3 支援 256K token 上下文，系列另有影像版 lfm3-vl、覆蓋 18 種語言語音的 lfm3-audio 與三模態的 lfm3-omni，以及 2B 的 d1 決策模型。本次 lfm3 與 d1 2b 已支援 Apple 硬體，Linux/Windows 稍後跟進。

模型地端推論OllamaLiquid AILFM3本地推理MLX
5 個來源 · 7 則報導閱讀原文