Janas評測:免GPU靠SSD跑80B模型,速度贏llama.cpp十倍 | Janas Review: Run an 80B LLM From SSD, No GPU Needed
By Kit 小克 | AI Tool Observer | 2026-10-14
🇹🇼 Janas評測:免GPU靠SSD跑80B模型,速度贏llama.cpp十倍
Janas 是這兩天在 Hacker News 衝上熱門的開源專案:一個純 C 語言寫成、零依賴的 LLM 推理引擎,靠著把模型權重放在 NVMe SSD 上、按需串流讀取,讓一台只有 32GB 記憶體的筆電也能跑動 48GB 大小的 Qwen3-Next-80B 模型,而且不需要 GPU。
Janas 怎麼辦到的:專家串流(Expert Streaming)
Qwen3-Next-80B 這類 MoE(混合專家)模型雖然總參數量達 80B,但每個 token 實際只會用到其中一小部分「專家」子網路。Janas 的做法是把整顆模型轉成自家的 .jns 格式留在硬碟上,用 O_DIRECT 繞過作業系統的 page cache,只在真正需要時把對應的專家權重從 SSD 讀進記憶體,常用的專家則留在 RAM 裡做快取。這代表模型大小不再被實體記憶體綁死,而是被硬碟容量和讀取速度決定。
實測數字:比 llama.cpp 快多少
作者在自己的 Intel Core Ultra 9 185H 筆電(32GB RAM)上測試 Qwen3-Next-80B,數據如下:
- 生成速度:37 tok/s(開啟推測解碼 speculative drafting),不開則是 24 tok/s,對比 llama.cpp 的 2.8 tok/s
- 讀 prompt 速度:62 tok/s,對比 llama.cpp 的 16 tok/s
- 作者宣稱在同機測試的 13 個模型中,Q4_K_M 量化下全部贏過 llama.cpp
推測解碼的原理是先讓一個小模型猜接下來幾個 token,再讓 80B 大模型一次性驗證,驗證通過就省下好幾輪的 SSD 讀取,這也是速度從 24 跳到 37 tok/s 的關鍵。
誰適合用、誰先別跳
Janas 目前在 GitHub 上有約 280 顆星,授權是 GPL-3.0,轉檔後的模型權重則沿用原模型授權(Qwen、Gemma 等)。適合的對象是想在沒有獨顯的筆電或舊機器上跑大型 MoE 模型做實驗的開發者。但專案作者自己也寫明:這是早期實驗,API 和指令列隨時會改,而且只在一台機器上開發測試過,換一台硬碟讀寫速度不同的機器,表現可能差很多,長時間運行也可能把 RAM 吃滿影響系統反應。如果你要的是穩定可上生產環境的推理後端,現階段 llama.cpp 或 vLLM 還是比較保險的選擇。
好不好用,試了才知道。
🇺🇸 Janas Review: Run an 80B LLM From SSD, No GPU Needed
Janas is the open-source project that hit the top of Hacker News this week: a dependency-free LLM inference engine written in pure C that streams model weights from an NVMe SSD on demand, letting a 32GB laptop run a 48GB Qwen3-Next-80B model without a GPU.
How Janas Works: Expert Streaming From SSD
MoE (mixture-of-experts) models like Qwen3-Next-80B have a huge total parameter count, but each token only activates a small subset of "expert" sub-networks. Janas converts the whole model into its own .jns file on disk, uses O_DIRECT to bypass the OS page cache, and reads only the experts a given token actually needs straight from the SSD — frequently-used experts stay cached in RAM. The practical effect: model size is no longer capped by your physical memory, but by your disk's read throughput.
The Benchmarks: How Much Faster Than llama.cpp
On the author's own Intel Core Ultra 9 185H laptop (32GB RAM), running Qwen3-Next-80B:
- Generation speed: 37 tok/s with speculative drafting enabled, 24 tok/s without — versus 2.8 tok/s for llama.cpp
- Prompt processing: 62 tok/s versus llama.cpp's 16 tok/s
- The author claims Janas beat llama.cpp on all 13 models tested in Q4_K_M quantization on the same machine
Speculative drafting is the trick behind the 24-to-37 jump: a small draft model guesses the next few tokens, and the 80B model verifies them in a single pass, cutting down the number of SSD round-trips needed.
Who Should Actually Try It
Janas sits around 280 GitHub stars, is licensed GPL-3.0, and the converted model weights keep their original licenses (Qwen, Gemma, etc). It's a genuinely interesting option for developers who want to experiment with large MoE models on a GPU-less laptop or older hardware. But the project itself is upfront that it's an early-stage experiment: the API and CLI change without notice, it has only been tested on one machine, performance will vary a lot depending on your SSD's read speed, and long sessions can eat up all your free RAM and make the system sluggish. If you need a stable, production-grade inference backend, llama.cpp or vLLM are still the safer bet for now.
好不好用,試了才知道 — the only way to know if it works for you is to try it.
Sources / 資料來源
- Janas on GitHub (prabanta-dev)
- AI Weekly: Janas Pure-C Engine Streams Qwen3-80B From NVMe at 37 tok/s
- dev.to: Run a 48GB AI Model on a 32GB Laptop
延伸閱讀 / Related Articles
- Claude Code評測:Torvalds用AI改寫Linux核心效能 | Claude Code Review: Torvalds Uses AI on Linux Kernel
- Mistral Large 4評測:歐洲最大模型,開源權重卻跳票 | Mistral Large 4 Review: Big Model, Delayed Open Weights
- Muse Wonder評測:Meta首款前沿模型,最強版鎖訂閱 | Muse Wonder Review: Meta's Frontier Model, Locked Tier
AI 工具觀察站 — 每日精選 AI Agent 與工具趨勢
AI Tool Observer — Daily curated AI Agent & tool trends
留言
張貼留言