OpenAI reports a large improvement in its models’ ability to work in languages such as COBOL, Fortran and OCaml. On MultiLangBench, GPT-6 Astra passed an average 87.4% of tests across 20 languages, up from 26.8% for GPT-5.6 Sol. A separate COBOL migration evaluation tested a different requirement: whether rewritten programs preserve exactly what the originals do.[1]
OpenAI released two evaluations on 19 October 2026 aimed at this difficult territory. MultiLangBench asks models to build a SQLite-compatible database engine from scratch in 20 programming languages; a separate migration evaluation tests whether COBOL programs can be moved into another language while preserving their behaviour. The attraction of these tests is that they require sustained engineering and observable outputs, not just the completion of small textbook functions.[1]
Two tasks, two performance measures
On the database-building benchmark, OpenAI reports GPT-6 Astra reaching an average test pass rate of 87.4% across the 20 languages, compared with 26.8% for GPT-5.6 Sol. COBOL, historically a difficult language for code-generating models, improved from 10.9% to 72.1%. The reported Fortran performance rose from 21.4% to 83.5%, while OCaml increased from 25.6% to 85.8%. These are pass percentages on OpenAI’s chosen benchmark and tasks, not success rates for software projects in the real world.[1]
The COBOL migration evaluation uses a different model comparison. OpenAI reports GPT-6.1 Sol scoring 86.6% accuracy at low reasoning effort versus 5.4% for GPT-5.6 Sol, with the newer model costing 39% less under the evaluation setup. At higher reasoning effort, the new model reached 97.9%. It would be a mistake to combine that 97.9% with MultiLangBench’s 87.4% as if they measured the same thing: one concerns semantic preservation during migration, the other construction of database functionality across languages.[1]
A database file is a useful referee
The SQLite-compatible task gives the model an initial database file and a SQL query and asks it to produce the expected modified database state. A reference SQLite implementation checks the results. That shared interface makes it possible to compare work written in otherwise very different languages. It also tests the ability to manage state and data formats across an extended coding session, not merely to produce a plausible-looking function.[1]
The design has boundaries. Passing many database tests is a real achievement, but even 87.4% leaves failed cases. The benchmark cannot represent every concurrency problem, security property, operational dependency or institution-specific rule in a live system. OpenAI says it has released the MultiLangBench test setup on GitHub, allowing outside developers to inspect which behaviours are checked. Independent reproduction and a longer maintenance record will be more informative than a single performance chart.[1]
The dangerous part of translating financial history
A small arithmetic choice can expose the difficulty. COBOL fixed-point fields discard excess decimal digits unless rounding is requested. A rewrite that rounds instead can execute successfully while moving a different amount of money. The migration evaluation therefore uses hidden tests requiring byte-for-byte parity on observable outputs.[1]
That makes the practical migration method more important than the choice of model. Preserve a reproducible baseline, map input and output contracts, compare the old and new programs on both routine and pathological cases, document each changed behaviour, and roll out behind reversible controls. Where a transaction carries legal or financial consequences, human owners need to approve semantic changes even if the AI confidently calls them improvements.[1]
The practical gain is better access to code that has received less attention from mainstream AI tools. The database and migration tests make different parts of that gain measurable: building working functionality, and preserving behaviour that an existing organisation already depends on.[1]
Sources & further reading
- Advancing coding in low-resource programming languages OpenAI · 19 Oct 2026 · Accessed 24 Oct 2026
- SQLite testing and file format documentation SQLite · Accessed 24 Oct 2026