AISpot

llama.cpp b11528 修復 tensor 拆分下 KV cache 檢視斷言崩潰

llama.cpp releases版本更新地端推論開源

llama.cpp 發布 b11528,修復 meta backend 處理主機記憶體 tensor 檢視時的斷言失敗。此前使用 --split-mode tensor 並配合部分 offload 時,KV cache 的 view 會被留在 meta split 中,導致 ggml_backend_meta_get_split_state() 解引用 tensor->buffer->context 崩潰;

原標題:b11528
閱讀原文

評分45 / 40(平均 42,門檻 60)
狀態未入選

原文

meta : handle host views ( #30217 ) meta : handle views of tensors allocated on the host A view shares the memory of its view_src, so ggml-alloc never allocates a view in the buffer of the split it lands in - the scheduler copies the source into the split and the ops that use the view read that copy. The view node itself is a noop and does not need a split of its own, but the meta backend asserted when one was left inside a meta split: ggml_backend_meta_get_split_state() dereferenced tensor->buffer->context the graph rebuild mapped every node with ggml_backend_meta_buffer_simple_tensor() Accept such nodes when they are views of host tensors, which also generalizes the previous s_copy_main workaround. This fixes the assert hit by KV cache views when using --split-mode tensor with partial offload. Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next archs : re-enable sm tensor for K2 Horizon cont : add TODO and reference Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54357448 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64 (CUDA 13) - CUDA 13.4 libraries Ubuntu arm64 (CUDA 13) - CUDA 13.4 libraries Ubuntu x64 (ROCm 10.0) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide Android: Android arm64 (CPU) Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.4 DLLs Windows arm64 (CUDA 13) - CUDA 13.4 DLLs Windows x64 (Vulkan) Windows arm64 (Vulkan) Windows x64 (OpenVINO) Windows x64 (SYCL) Windows x64 (ROCm 10.0) openEuler: DISABLED openEuler x86 (310p) openEuler x86 (910b, ACL Graph) openEuler aarch64 (310p) openEuler aarch64 (910b, ACL Graph) UI: UI

相關報導

llama.cpp releasesAI 評分43

llama.cpp 修復 Vulkan 主機記憶體問題,AMD 停用記憶體共享

llama.cpp 發布 b11683 版本,Vulkan 後端修復了 GPU 可見主機記憶體的相關問題,並減少主機指標匯入失敗時的日誌刷屏。該版本把 MoltenVK 排除在共享記憶體匯入之外,同時因 amdgpu 效能問題,在 AMD 上停用了主機記憶體緩衝區共享。

llama.cpp releasesAI 評分42

llama.cpp 發布 b11424,修復 Vulkan Flash Attention 共享記憶體越界寫

llama.cpp 發布 b11424 建置版本,修復 Vulkan 後端 Flash Attention 的共享記憶體越界寫問題(#29988)。該版本照例提供 macOS/iOS、Linux、Windows、Android 的預編譯包,涵蓋 Vulkan、CUDA 12/13、ROCm 10.0、OpenVINO、SYCL、OpenCL 等後端,並附驍龍 CPU/Adreno GPU/Hexagon NPU 的安裝指引。

llama.cpp releasesAI 評分40

llama.cpp b11693 修復 SYCL 後端 mul_mat 記憶體錯誤

llama.cpp 發布建置 b11693,核心改動是修復 SYCL 後端在 mul_mat、split buffer 和 host pool 中的記憶體錯誤(PR #29889)。該版本同時更新了預編譯產物清單,覆蓋 macOS/iOS、Linux、Windows、Android,包含 CUDA 12/13、Vulkan、ROCm 10.0、OpenVINO 以及 SYCL FP32/FP16 等後端。