跳到主要內容

GPT-6.1 Sol評測:COBOL遷移飆97.9%,測試黑箱藏陷阱 | GPT-6.1 Sol Review: 97.9% COBOL Score, Hidden Test Catch

By Kit 小克 | AI Tool Observer | 2026-10-22

🇹🇼 GPT-6.1 Sol評測:COBOL遷移飆97.9%,測試黑箱藏陷阱

OpenAI本週發布GPT-6.1 Sol,宣稱在COBOL遷移測試上準確率從前代GPT-5.6 Sol的5.4%,一口氣衝到86.6%(低推理量)甚至97.9%(拉高推理量),同時API成本只要前代的61%。COBOL撐著全球四成以上的網路銀行系統、八成的實體刷卡交易、九成五的ATM提款,這數字乍看像是遺留系統轉型的福音——但細看測試方法,事情沒那麼簡單。

數字很猛:5.4%飆到97.9%是怎麼回事

這個COBOL遷移評測會把老COBOL程式碼轉成Python、Java、Kotlin、C++或Rust,前代模型幾乎全軍覆沒,新模型卻能衝到近滿分。OpenAI把它歸功於推理量拉高加上架構優化,同時強調低推理量模式下成本直接砍四成。

但測試題目是OpenAI自家的黑箱

開發者社群馬上戳破一個關鍵:OpenAI同時公開的另一個基準MultiLangBench放在GitHub上可以自己重跑驗證,但COBOL遷移評測用的是OpenAI自己設計、沒公開的隱藏測試,外部完全無法獨立複現。換句話說,97.9%這個分數,目前只有OpenAI自己說了算。

真正的地雷:四捨五入誤差

有工程師指出COBOL的定點數運算(fixed-point arithmetic)跟Python、Java常見的float運算,四捨五入規則不一樣——老系統小數點第三位直接捨去,新程式碼卻拿去做四捨五入,兩邊算出來的金額對不上。如果驗證只檢查「數字差不多」,這種誤差完全抓不到,放到銀行或ATM系統裡就是真金白銀的帳務問題。

該不該直接拿來用在生產系統?

  • 先用自己的平行驗證(parity harness)比對新舊程式輸出,不要只信OpenAI公布的分數
  • 拿真實生產資料測試,不要只測開心路徑(happy path)
  • 遷移完的程式碼照樣要人工審查,GPT-6.1 Sol目前還不是全自動方案

GPT-6.1 Sol在COBOL遷移上的進步是真的,但「97.9%準確率」跟「可以直接上線」中間還隔著一段距離。對銀行、保險這類系統,省下來的人力成本可能都不夠補一次對帳出錯的損失。

好不好用,試了才知道。


🇺🇸 GPT-6.1 Sol Review: 97.9% COBOL Score, Hidden Test Catch

GPT-6.1 Sol, OpenAI's latest model, claims a dramatic leap on COBOL migration: accuracy jumped from 5.4% (GPT-5.6 Sol) to 86.6% at low reasoning effort, and 97.9% at high reasoning effort — while costing about 61% less per API call. COBOL still runs over 40% of online banking, 80% of in-person card transactions, and 95% of ATM withdrawals worldwide, so a real jump here matters. But the benchmark details tell a more cautious story.

The Numbers: 5.4% to 97.9%

The COBOL migration eval translates legacy COBOL into Python, Java, Kotlin, C++, or Rust. Earlier models mostly failed; GPT-6.1 Sol clears nearly the whole test set at higher reasoning effort, while the low-effort mode cuts API cost by roughly 61% compared to GPT-5.6 Sol.

The Catch: It's OpenAI's Own Hidden Test Suite

Developers quickly flagged a problem: OpenAI also published MultiLangBench, a public benchmark anyone can rerun on GitHub — but the COBOL migration score comes from OpenAI-designed hidden tests that nobody outside the company can independently verify. The 97.9% figure, for now, is OpenAI grading its own homework.

The Real Risk: Rounding Errors

Engineers reviewing the claim point to a subtler issue: COBOL's fixed-point arithmetic rounds differently than the float operations common in Python or Java. Old code might truncate a third decimal place; migrated code might round it instead. Any test that only checks "close enough" output will miss this — and in a bank or ATM system, that's real money going missing.

Should You Trust It in Production?

  • Build your own parity harness to compare old vs. new output byte-for-byte, rather than trusting OpenAI's score alone
  • Test against real production data, not just happy-path cases
  • Keep human review in the loop — GPT-6.1 Sol is not a fully automated migration solution yet

The COBOL migration improvement is real, but there's still a gap between "97.9% accuracy" and "safe to ship." For banks and insurers, the labor savings may not cover the cost of a single bad reconciliation.

You won't know until you try it.

Sources / 資料來源

延伸閱讀 / Related Articles


AI 工具觀察站 — 每日精選 AI Agent 與工具趨勢
AI Tool Observer — Daily curated AI Agent & tool trends

留言

這個網誌中的熱門文章

Google Ironwood TPU v7 推理專用晶片解析:效能追平 NVIDIA、成本低 44%,AI 晶片戰爭正式開打 | Google Ironwood TPU v7 Explained: Matching NVIDIA Performance at 44% Lower Cost — The AI Chip War Heats Up

Claude Code 實測:AI 幫你寫程式到底行不行? | Claude Code Review: Can AI Really Code for You?

Cursor vs GitHub Copilot vs Claude Code:AI 程式助手大比拼 | AI Coding Assistants Compared: Cursor vs GitHub Copilot vs Claude Code