llama.cpp 發布 b11472:零溫度取樣改用貪心選擇
llama.cpp 發布建置 b11472,對符合條件、溫度為零的取樣鏈改用貪心選擇,CPU 與 grammar/reasoning-budget 路徑共用同一套貪心判定;動態溫度和顯式請求機率的場景仍保留分佈取樣,最終 top-k 且 k=1 之後也走貪心。改動由 Georgi Gerganov 參與,OpenAI Codex 協助,並在現有 sampler 測試中覆蓋常見取樣器選擇與機率行為。
原標題:b11472
閱讀原文
| 評分 | 55 / 45(平均 50,門檻 60) |
| 狀態 | 未入選 |
|---|
原文
sampling : use greedy selection for eligible temperature-zero chains ( #29797 )
sampling : use greedy selection for eligible temperature-zero chains
Assisted-by: OpenAI Codex
Apply suggestion from @ggerganov
Co-authored-by: Georgi Gerganov ggerganov@gmail.com
sampling: simplify zero-temperature greedy eligibility
Allow the same greedy selection on CPU and grammar/reasoning-budget paths.
Keep distribution sampling for dynamic temperature and requested probabilities.
Cover the common sampler selection and probability behavior in the existing sampler tests.
Assisted-by: OpenAI Codex
sampling : use greedy selection after final top-k with k=1
Assisted-by: OpenAI Codex
cont : clean-up
Co-authored-by: Georgi Gerganov ggerganov@gmail.com
Website:
https://llama.app
Attestations:
https://github.com/ggml-org/llama.cpp/attestations/53574365
macOS/iOS:
macOS Apple Silicon (arm64)
macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED
macOS Intel (x64)
iOS XCFramework
Linux:
Ubuntu x64 (CPU)
Ubuntu arm64 (CPU)
Ubuntu s390x (CPU)
Ubuntu x64 (Vulkan)
Ubuntu arm64 (Vulkan)
Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries
Ubuntu x64 (CUDA 13) - CUDA 13.4 libraries
Ubuntu arm64 (CUDA 13) - CUDA 13.4 libraries
Ubuntu x64 (ROCm 10.0)
Ubuntu x64 (OpenVINO)
Ubuntu x64 (SYCL FP32)
Ubuntu x64 (SYCL FP16)
Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide
Android:
Android arm64 (CPU)
Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide
Windows:
Windows x64 (CPU)
Windows arm64 (CPU)
Windows arm64 (OpenCL Adreno)
Windows x64 (CUDA 12) - CUDA 12.4 DLLs
Windows x64 (CUDA 13) - CUDA 13.4 DLLs
Windows arm64 (CUDA 13) - CUDA 13.4 DLLs
Windows x64 (Vulkan)
Windows arm64 (Vulkan)
Windows x64 (OpenVINO)
Windows x64 (SYCL)
Windows x64 (ROCm 10.0)
openEuler:
DISABLED
openEuler x86 (310p)
openEuler x86 (910b, ACL Graph)
openEuler aarch64 (310p)
openEuler aarch64 (910b, ACL Graph)
UI:
UI
相關報導
llama.cpp releases● 精選10/2 22:38AI 評分69
llama.cpp 合併 PR #27694,為 simple draft 與 MTP 推測解碼引入機率取樣:草稿端按機率取樣,目標模型用拒絕取樣驗證,並修復草稿取樣器與目標模型共用 RNG 流等問題。新增開關控制機率草稿取樣,預設仍為貪心;語法約束請求回退到 argmax,同時支援在拒絕取樣中處理語法約束。改動由 NVIDIA 工程師參與提交,隨各平台建置一同發布。
llama.cpp releases10/4 06:22AI 評分42
llama.cpp 發布 b11387 版本,修復了在截斷後溫度大於 0 時 n-gram 草稿被拒絕的問題(PR #29924),由 NVIDIA 的 Pranesh Gonegandla 參與提交。
llama.cpp releases10/3 04:36AI 評分43
llama.cpp 合併 PR #29831,為 clef 決策模型加入初始支援,目前僅限純文字輸入。該改動同時更新了 gguf-py 常量並清理了靜態圖相關程式碼,隨附的建置產物覆蓋 macOS、Linux、Windows、Android 與 openEuler 等多個平台。
llama.cpp releases10/6 06:05AI 評分33
llama.cpp 發布 b11438,主要變更是 test-llama-archs 在生成模型前先初始化後端(#30034):開啟 GGML_BACKEND_DL 時,必須先顯式載入後端才能建立模型。
llama.cpp releases10/5 00:35AI 評分37
llama.cpp 發布 b11402 版本,CUDA 後端改為優先採用整塊(whole-tile)FlashAttention 排程,以提升兩階段核心效率。