AI researchers became much more likely to share both code and data over the decade to 2024, according to a study of 56,800 papers from five leading conferences. The proportion rose from 11% in 2014 to 64% in 2024. That is a large change in what another researcher can inspect and attempt to use.[1]
The paper, published in Communications AI & Computing on 22 October, also estimates that reproducibility rose from 28% to 64%. The matching endpoints conceal different measurements. One records whether authors supplied code and data. The other uses a model, informed by earlier empirical reproducibility research, to estimate the likelihood of repeating results.[1]
Two measures, two kinds of evidence
| Measure | Value | Context |
|---|---|---|
| Code and data both shared | 11% of papers in 2014; 64% in 2024 | |
| Estimated reproducibility | 28% in 2014; 64% in 2024 |
What an open repository does—and does not—supply
A published codebase turns a method description into something another team can examine. The dataset gives that team a route toward the same inputs. Yet an experiment may still depend on undocumented settings, unavailable computing resources or software versions that no longer behave as they did. Sharing removes substantial obstacles without guaranteeing a successful rerun.[1]
That distinction matters when reading the study’s trend. It shows a research community making more of its work available, accompanied by a modelled improvement in reproducibility. It does not identify a set of papers that independent teams have definitively reproduced. The estimate is useful precisely when its statistical status stays attached to it.[1]
The trend predates the checklist
The authors find that improvement was already under way before conferences introduced formal reproducibility checklists. The timing weakens a simple explanation in which checklists alone produced the change. It does not make those policies pointless: asking authors to document resources and settings can still standardise expectations and make omissions easier to spot.[1]
For someone deciding whether to build on a result, openness provides a starting point for inspection. The next step is an actual attempt to reproduce the relevant finding under documented conditions. This study measures progress toward that possibility at the scale of an entire research field, while leaving the experiment-by-experiment work distinct.[1]
Sources & further reading
- The shift toward open and reproducible AI research Communications AI & Computing · 22 Oct 2026 · Accessed 26 Oct 2026