Truncate or Round
OpenAI’s COBOL test counts a migration as correct only if it passes every hidden test. Here is a tiny migration and a set of tests. You choose which tests are hidden, and see what a pass can and cannot tell you.
1 · A one-line rule nobody wrote down
The original program works out monthly interest. In COBOL, if a result has more decimal places than its field can hold, the extra digits are dropped unless the line says ROUNDED. The migrated version does what most people would write today.
05 WS-INTEREST PIC 9(7)V99.
COMPUTE WS-INTEREST =
WS-BALANCE * WS-RATE / 100.
* no ROUNDED: extra digits droppedinterest = balance.multiply(rate)
.divide(HUNDRED)
.setScale(2, RoundingMode.HALF_UP);
// rounds to the nearest centSimplified for illustration. OpenAI’s post uses this behaviour as its example: a rewrite that rounds “can run without errors while producing different financial results.”
2 · You choose the hidden tests
Tick the accounts that go into the test suite. A migration passes only if every chosen test gives exactly the same output as the original.
| Test? | Balance | Rate | Exact | Original (drops) | Migrated (rounds) |
|---|
3 · One cent, many times
Each difference is a single cent. For amounts with arbitrary fractions, rounding and dropping disagree about half the time. Move the volume.
calculations a day → about a day moving to the wrong side of the ledger.
Illustrative arithmetic, not data from any bank: half of the calculations differ by one cent. The point is direction, not size — the migrated system and the old one no longer agree, and every test that was chosen still passes.
4 · Three companies, one hard part
“You’re reverse engineering business logic from systems built when Nixon was president.”
“Translation captures almost none of the actual complexity.”
“Important business rules may only exist in code.”
5 · What the market did
IBM had already partnered with both model makers: Anthropic in October 2025, OpenAI in August 2026.