跳到主要內容

Janas評測:免GPU靠SSD跑80B模型,速度贏llama.cpp十倍 | Janas Review: Run an 80B LLM From SSD, No GPU Needed

By Kit 小克 | AI Tool Observer | 2026-10-14

🇹🇼 Janas評測:免GPU靠SSD跑80B模型,速度贏llama.cpp十倍

Janas 是這兩天在 Hacker News 衝上熱門的開源專案:一個純 C 語言寫成、零依賴的 LLM 推理引擎,靠著把模型權重放在 NVMe SSD 上、按需串流讀取,讓一台只有 32GB 記憶體的筆電也能跑動 48GB 大小的 Qwen3-Next-80B 模型,而且不需要 GPU。

Janas 怎麼辦到的:專家串流(Expert Streaming)

Qwen3-Next-80B 這類 MoE(混合專家)模型雖然總參數量達 80B,但每個 token 實際只會用到其中一小部分「專家」子網路。Janas 的做法是把整顆模型轉成自家的 .jns 格式留在硬碟上,用 O_DIRECT 繞過作業系統的 page cache,只在真正需要時把對應的專家權重從 SSD 讀進記憶體,常用的專家則留在 RAM 裡做快取。這代表模型大小不再被實體記憶體綁死,而是被硬碟容量和讀取速度決定。

實測數字:比 llama.cpp 快多少

作者在自己的 Intel Core Ultra 9 185H 筆電(32GB RAM)上測試 Qwen3-Next-80B,數據如下:

  • 生成速度:37 tok/s(開啟推測解碼 speculative drafting),不開則是 24 tok/s,對比 llama.cpp 的 2.8 tok/s
  • 讀 prompt 速度:62 tok/s,對比 llama.cpp 的 16 tok/s
  • 作者宣稱在同機測試的 13 個模型中,Q4_K_M 量化下全部贏過 llama.cpp

推測解碼的原理是先讓一個小模型猜接下來幾個 token,再讓 80B 大模型一次性驗證,驗證通過就省下好幾輪的 SSD 讀取,這也是速度從 24 跳到 37 tok/s 的關鍵。

誰適合用、誰先別跳

Janas 目前在 GitHub 上有約 280 顆星,授權是 GPL-3.0,轉檔後的模型權重則沿用原模型授權(Qwen、Gemma 等)。適合的對象是想在沒有獨顯的筆電或舊機器上跑大型 MoE 模型做實驗的開發者。但專案作者自己也寫明:這是早期實驗,API 和指令列隨時會改,而且只在一台機器上開發測試過,換一台硬碟讀寫速度不同的機器,表現可能差很多,長時間運行也可能把 RAM 吃滿影響系統反應。如果你要的是穩定可上生產環境的推理後端,現階段 llama.cpp 或 vLLM 還是比較保險的選擇。

好不好用,試了才知道。


🇺🇸 Janas Review: Run an 80B LLM From SSD, No GPU Needed

Janas is the open-source project that hit the top of Hacker News this week: a dependency-free LLM inference engine written in pure C that streams model weights from an NVMe SSD on demand, letting a 32GB laptop run a 48GB Qwen3-Next-80B model without a GPU.

How Janas Works: Expert Streaming From SSD

MoE (mixture-of-experts) models like Qwen3-Next-80B have a huge total parameter count, but each token only activates a small subset of "expert" sub-networks. Janas converts the whole model into its own .jns file on disk, uses O_DIRECT to bypass the OS page cache, and reads only the experts a given token actually needs straight from the SSD — frequently-used experts stay cached in RAM. The practical effect: model size is no longer capped by your physical memory, but by your disk's read throughput.

The Benchmarks: How Much Faster Than llama.cpp

On the author's own Intel Core Ultra 9 185H laptop (32GB RAM), running Qwen3-Next-80B:

  • Generation speed: 37 tok/s with speculative drafting enabled, 24 tok/s without — versus 2.8 tok/s for llama.cpp
  • Prompt processing: 62 tok/s versus llama.cpp's 16 tok/s
  • The author claims Janas beat llama.cpp on all 13 models tested in Q4_K_M quantization on the same machine

Speculative drafting is the trick behind the 24-to-37 jump: a small draft model guesses the next few tokens, and the 80B model verifies them in a single pass, cutting down the number of SSD round-trips needed.

Who Should Actually Try It

Janas sits around 280 GitHub stars, is licensed GPL-3.0, and the converted model weights keep their original licenses (Qwen, Gemma, etc). It's a genuinely interesting option for developers who want to experiment with large MoE models on a GPU-less laptop or older hardware. But the project itself is upfront that it's an early-stage experiment: the API and CLI change without notice, it has only been tested on one machine, performance will vary a lot depending on your SSD's read speed, and long sessions can eat up all your free RAM and make the system sluggish. If you need a stable, production-grade inference backend, llama.cpp or vLLM are still the safer bet for now.

好不好用,試了才知道 — the only way to know if it works for you is to try it.

Sources / 資料來源

延伸閱讀 / Related Articles


AI 工具觀察站 — 每日精選 AI Agent 與工具趨勢
AI Tool Observer — Daily curated AI Agent & tool trends

留言

這個網誌中的熱門文章

Google Ironwood TPU v7 推理專用晶片解析:效能追平 NVIDIA、成本低 44%,AI 晶片戰爭正式開打 | Google Ironwood TPU v7 Explained: Matching NVIDIA Performance at 44% Lower Cost — The AI Chip War Heats Up

Claude Code 實測:AI 幫你寫程式到底行不行? | Claude Code Review: Can AI Really Code for You?

Cursor vs GitHub Copilot vs Claude Code:AI 程式助手大比拼 | AI Coding Assistants Compared: Cursor vs GitHub Copilot vs Claude Code