🔍 Read the full analysis: Can AI’s Review Process Scale Alongside Its Output? on ThorstenMeyerAI.com
Get monitors, keyboards and dev gear delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
TL;DR
OpenAI said its model produced 722 mathematics manuscripts this week, while software-industry data cited in a recent analysis suggests AI-assisted code is also increasing review workloads. The figures point to a growing mismatch between the low cost of producing AI output and the human time needed to verify, judge and take responsibility for it.
OpenAI said this week its model produced 722 mathematical manuscripts from a programme that posed about 4,000 problems, intensifying debate over whether human review can keep pace with AI output. The development matters beyond mathematics: software-industry data cited by ThorstenMeyerAI.com also points to rising review time and uneven checks as AI-generated changes become more common.
The manuscripts were grouped into 372 families, and the source estimates that the average result took about three hours of compute to produce. OpenAI has said some results have formal checks in Lean, a proof-assistant system, while warning that unformalized results could have issues. The source does not provide a complete independent audit of all 722 manuscripts.
The difference between producing work and establishing its value is central to the concern. A formal system can test whether a proof follows from its stated assumptions; experts still need to judge whether the assumptions and theorem address the intended problem, whether the result is significant, and whether it withstands scrutiny. The source contrasts the manuscript volume with the careful verification by five leading mathematicians of an earlier result from the same programme, described there as a counterexample to an old Erdős conjecture.
Software metrics cited in the source offer a second example, though their methods and periods are not fully detailed in the supplied material. Faros AI reported that teams merged 98% more pull requests across low- and high-AI-adoption periods while review time rose 91%. LinearB said its analysis of 8.1 million pull requests across 4,800 organisations found AI-generated changes took 4.6 times longer for review to begin and were accepted 32.7% of the time, compared with 84.4% for human-written changes. These are reported findings, not a controlled demonstration that AI use alone caused the differences.
The referee shortage: AI made doing cheap and checking expensive
OpenAI’s model produced a maths result in about three hours of compute. Verifying one earlier result took five of the world’s leading mathematicians. That ratio is the next decade of work: producing is cheap and abundant; trusting is slow, human and scarce.
Some Lean-checked; OpenAI warns unformalized ones “could have issues.” Verification abundance, adjudication scarcity.
Faros AI. LinearB (8.1M PRs): AI changes wait 4.6× longer, accepted 32.7% vs 84.4%.
Real progress. Someone still has to find the other 45% before the work can be used.
Machine output arrives without reasoning a reviewer can interrogate. It looks locally clean and gives no clue where it’s wrong.
A prover confirms the proof proves its statement; tests confirm what tests check. Neither confirms it’s what was needed.
Contracts are signed, designs stamped, papers defended. Responsibility is institutional — you can’t hold a model to it.
of AI-agent pull requests got no human review at all (EASE 2026). Zero-review merges up 31.3% (Faros).
of reviewers deliberately deprioritise AI changes (LinearB). Good machine work waits behind bad.
OpenAI chose which maths families were significant. When referees can’t keep up, the producer’s filter becomes the review.
Budget review hours next to model spend.
Provers, types, tests, policy engines.
Experts only where consequences are high.
Who profits from generation pays for checking.
Keep some production human for learners.
The first automation question was which jobs AI would do. The better one is which jobs AI makes more necessary: the ones that check, adjudicate and take responsibility. Expect a referee premium — senior engineers, auditors, specialist lawyers, reviewing scientists become the binding constraint on how much AI output anyone can use.Accountability — standing behind a result — may be the most durable form of human work there is.
Review Time Is Becoming a Constraint
If generation grows faster than review capacity, organisations face a practical limit on how much AI-produced work they can safely use. More drafts, proofs or code changes do not automatically create more usable work; each still needs checks suited to its risks, and some decisions require a person able to explain and stand behind them.
The cited software figures describe several possible consequences: longer queues, lower review rates and reviewers giving AI-generated submissions extra scrutiny. The source also reports that a 2026 peer-reviewed study found 61% of AI-agent pull requests received no human review before merging or closing. Without details about the study’s sample and definition of review, that figure should be read as a finding within its reported scope, not a universal rate.
There is also a workforce implication. The source argues that experienced reviewers usually gain judgment through years of doing the underlying work. If AI changes reduce opportunities for junior staff to write code, draft contracts or develop proofs themselves, organisations may weaken the route by which future experts acquire that judgment. That is a risk raised by the analysis, not an established outcome across industries.
As an affiliate, we earn on qualifying purchases.
Three Sectors, Similar Bottlenecks
Mathematics makes the distinction between checking and judging especially visible. A proof assistant can check a formal proof against a specified statement. It cannot by itself establish that the statement is the right one to pursue or explain what the result means. OpenAI’s warning about unformalized manuscripts also means the full collection should not be treated as uniformly verified.
In software, automated tests and review tools can catch some problems, but their coverage depends on what was tested and what reviewers inspect. The source cites Faros AI, LinearB and a peer-reviewed 2026 study, while cautioning that several of the data providers sell code-review tools. Their commercial interests do not invalidate the measurements, but readers need study methods and independent replication to assess how broadly the findings apply.
The source also describes an OpenAI partnership with contract-software company Ironclad involving a model called GPT-6 Astra, evaluated on 11 contracting tasks. It reportedly met 55% of evaluation criteria on average, an improvement over a previous model. That result is a reported benchmark, not proof that the model can safely handle contracts without legal review; the supplied material gives no task-by-task results or full evaluation methodology.
AI manuscript verification software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
How Much Work Was Verified?
The supplied source does not establish how many of the 722 manuscripts were independently checked, how many were formally verified, or how the published selection was made from the roughly 4,000 problems. It also does not provide enough detail to compare the mathematics programme’s review effort directly with the software studies.
The software statistics come from sources with differing methods and potential commercial interests. The material does not give their complete definitions, sampling procedures or measurement windows, so the numbers should not be combined into a single industry-wide estimate. The Ironclad evaluation likewise lacks enough methodological detail here to show how its criteria map to real contract outcomes.
More broadly, it remains unclear whether new review tools and workflows can expand expert capacity fast enough, or whether organisations will respond by accepting more unchecked work, delaying it, or narrowing what they use. The source presents these as possible patterns and risks, not settled outcomes.
As an affiliate, we earn on qualifying purchases.
More Audits and Workflow Evidence
The next useful evidence will be independent audits of AI-generated mathematics, including clear counts of formally checked work, corrections and failures. For software, comparable studies should report how review time, defect rates and review coverage change under clearly defined levels of AI adoption, with methods that readers can examine.
For contract workflows, the reported 11-task result needs fuller evaluation details and evidence about how outputs perform when reviewed in actual practice. Across fields, organisations will also need to track who is accountable for approving work and whether junior staff still gain the experience required to become reliable reviewers.
For now, the confirmed development is the scale of OpenAI’s reported mathematics output, alongside separate studies and industry measurements pointing to review pressures in software. Whether verification can scale through better tools, more training or different institutional rules remains an open question.
As an affiliate, we earn on qualifying purchases.
Key Questions
What did OpenAI report this week?
OpenAI’s model produced 722 mathematical manuscripts in 372 families after being posed about 4,000 problems, according to the source. The source estimates an average of about three hours of compute per result.
Were all 722 manuscripts verified?
No. The source says some results were formally checked in Lean and quotes OpenAI warning that unformalized results could have issues. It does not state how many manuscripts were independently verified.
What do the software figures show?
The cited figures report rising review time and longer waits before review for AI-generated code changes. They come from different analyses with distinct methods; the supplied material does not establish that AI use alone caused the reported patterns.
Why can’t AI simply review AI output?
Automated checks can test specific properties, such as whether code passes tests or a formal proof follows stated rules. They may not establish whether the requirements, tests or original question were correct, and people or institutions often remain responsible for approving consequential work.
What remains unknown?
The total share of AI output receiving meaningful human review, the reliability of the cited findings across organisations, and whether review tools can increase expert capacity are all unclear from the available material.
Source: ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
