AISpot

llama.cpp 發布 b11472:零溫度取樣改用貪心選擇

llama.cpp releases10/7 09:24版本更新地端推論開源開發工具

llama.cpp 發布建置 b11472,對符合條件、溫度為零的取樣鏈改用貪心選擇,CPU 與 grammar/reasoning-budget 路徑共用同一套貪心判定;動態溫度和顯式請求機率的場景仍保留分佈取樣,最終 top-k 且 k=1 之後也走貪心。改動由 Georgi Gerganov 參與,OpenAI Codex 協助,並在現有 sampler 測試中覆蓋常見取樣器選擇與機率行為。

原標題:b11472
閱讀原文

評分55 / 45(平均 50,門檻 60)
狀態未入選

原文

sampling : use greedy selection for eligible temperature-zero chains ( #29797 ) sampling : use greedy selection for eligible temperature-zero chains Assisted-by: OpenAI Codex Apply suggestion from @ggerganov Co-authored-by: Georgi Gerganov ggerganov@gmail.com sampling: simplify zero-temperature greedy eligibility Allow the same greedy selection on CPU and grammar/reasoning-budget paths. Keep distribution sampling for dynamic temperature and requested probabilities. Cover the common sampler selection and probability behavior in the existing sampler tests. Assisted-by: OpenAI Codex sampling : use greedy selection after final top-k with k=1 Assisted-by: OpenAI Codex cont : clean-up Co-authored-by: Georgi Gerganov ggerganov@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53574365 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64 (CUDA 13) - CUDA 13.4 libraries Ubuntu arm64 (CUDA 13) - CUDA 13.4 libraries Ubuntu x64 (ROCm 10.0) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide Android: Android arm64 (CPU) Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.4 DLLs Windows arm64 (CUDA 13) - CUDA 13.4 DLLs Windows x64 (Vulkan) Windows arm64 (Vulkan) Windows x64 (OpenVINO) Windows x64 (SYCL) Windows x64 (ROCm 10.0) openEuler: DISABLED openEuler x86 (310p) openEuler x86 (910b, ACL Graph) openEuler aarch64 (310p) openEuler aarch64 (910b, ACL Graph) UI: UI

相關報導

llama.cpp releases● 精選10/2 22:38AI 評分69

llama.cpp 為草稿模型與 MTP 加入機率取樣

llama.cpp 合併 PR #27694,為 simple draft 與 MTP 推測解碼引入機率取樣:草稿端按機率取樣,目標模型用拒絕取樣驗證,並修復草稿取樣器與目標模型共用 RNG 流等問題。新增開關控制機率草稿取樣,預設仍為貪心;語法約束請求回退到 argmax,同時支援在拒絕取樣中處理語法約束。改動由 NVIDIA 工程師參與提交,隨各平台建置一同發布。