AISpot

llama.cpp 修復 MiniMax-M3 量化 scale 未傳入計算圖導致輸出亂碼

llama.cpp releases版本更新地端推論開源

llama.cpp 修復了 MiniMax-M3 的量化 scale 未轉發到計算圖、導致輸出變成亂碼的問題。minimax-m3.cpp 此前沒有把逐專家的 NVFP4 scale2 張量,以及稠密 FFN 和共享專家的 MXFP8 逐張量 scale 傳入圖;

這條修復點明瞭量化 scale 漏傳導致 MoE 輸出亂碼的具體根因,並給出 M3.1 Flash 的 G≈15123 實測數字;對在 llama.cpp 上執行 MiniMax-M3 NVFP4/MXFP8 量化版的人,可據此判斷是否需要升級。

原標題:b11861
閱讀原文

評分52 / 55(平均 53,門檻 60)
狀態未入選

原文

minimax-m3 : forward quant scales to the graph (fixes #31063 ) ( #31097 ) The generic scale pass in llama-model.cpp loads the per-expert NVFP4 scale2 tensors (ffn_{up,gate,down}_exps_s) and the per-tensor MXFP8 scales for the dense FFN and the shared expert, but minimax-m3.cpp never forwarded any of them to the graph. build_moe_ffn() defaults these to nullptr, so build_lora_mm_id() skipped its w_s branch entirely and the routed-expert weights came out global_scale times too large, swamping the residual (reported G ~= 15123 for M3.1 Flash) and producing token soup. Forward all scale tensors at the three call sites, matching the pattern used by flint and the other NVFP4-capable MoE models. Co-authored-by: origin-labs-ai [email protected] Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/59462199 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64 (CUDA 13) - CUDA 13.4 libraries Ubuntu arm64 (CUDA 13) - CUDA 13.4 libraries Ubuntu x64 (ROCm 10.0) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide Android: Android arm64 (CPU) Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.4 DLLs Windows arm64 (CUDA 13) - CUDA 13.4 DLLs Windows x64 (Vulkan) Windows arm64 (Vulkan) Windows x64 (OpenVINO) Windows x64 (SYCL) Windows x64 (ROCm 10.0) openEuler: DISABLED openEuler x86 (310p) openEuler x86 (910b, ACL Graph) openEuler aarch64 (310p) openEuler aarch64 (910b, ACL Graph) UI: UI

相關報導

llama.cpp releasesAI 評分40

llama.cpp 修復 qkx3 量化縮放搜尋的非法舍入

llama.cpp 合併 PR #29817,修復 ggml-quants 中 qkx3 量化縮放搜尋的非法舍入問題:當 imatrix 擬合的最小值塌縮為最大值或範圍極小時,會產生無窮、NaN 或越界值,傳入 nearest_int 後可能觸發 Debug 建置斷言。修復方式是在舍入前把量化級別鉗制到 [0, nmax],並新增 q2_K、q4_K、q5_K、q4_1、q5_1 的退化 imatrix 組迴歸測試,對應 issue #29804。