AI made the work better. Did it make the worker better?

A beautifully improved document can conceal an unchanged author. That is the uncomfortable question raised by a patent-law experiment discussed by Google Research on 7 October: using an assistant to deliver better work and learning to exercise better judgement are not necessarily the same achievement.[2]

The distinction matters beyond a debate about productivity. A business can buy an immediate improvement in output while quietly changing the process through which its next generation of experts develops. Whether that change is beneficial cannot be established by counting finished documents alone.

A trial with two different finish lines

The September NBER working paper describes a preregistered, three-month randomised trial involving 133 practising patent lawyers at 11 US firms. Expert attorneys scored the work without knowing treatment status. AI-assisted drafting was tested during the experiment; everyone then completed an unaided review of an existing application. Google funded the direct experimental costs, and the paper includes author employment and financial disclosures.[1]

The results do not say that every junior became dependent or that every senior learned. An average can conceal diverging trajectories. Nor does a finding on a review task directly measure everything a professional knows. Those are important limits on the inference, even when random assignment makes the short-run comparison much more informative than a customer testimonial.

The missing apprenticeship can be hidden in the workflow

Google’s discussion offers a possible mechanism: less experienced participants sometimes concentrated on surface edits, while experienced practitioners engaged more deeply with substantive choices. These observations help generate explanations; they do not isolate the causal effect of a particular training method. The broader lesson remains a hypothesis to test, not a licence to sort workers into permanently capable and incapable groups.[2]

Consider a hypothetical analyst who must explain why a supplier’s margin fell. Without assistance, the analyst might discover that volume, price and product mix lead to different conclusions. With assistance, a polished explanation may arrive immediately. The task is completed, but the learner may never confront the competing explanations. The lost activity is not typing: it is discriminating between plausible accounts.

Now change the workflow. Before displaying the assistant’s analysis, ask the employee to state the two most likely causes and the evidence that would distinguish them. Afterwards, require one correction or a reasoned defence of the original view. This is our proposed training design, not an intervention tested in the patent study. It preserves a small, purposeful piece of effort while still using AI to accelerate the larger job.

Make the learning visible without turning every task into an exam

The important word is consequential. Asking someone to rewrite every AI sentence by hand would preserve labour without necessarily preserving learning. A useful checkpoint targets a decision whose logic will recur: why a claim is too broad, why an observation supports one interpretation rather than another, or what evidence would reverse the recommendation.

Managers also need two dashboards. One tracks the quality, turnaround time and correction burden of delivered work. The other samples what people can still do when the usual assistance is unavailable or wrong. Combining those into a single productivity score would erase the distinction the experiment was designed to reveal.

A firm should not make its employees feel that admitting uncertainty jeopardises their job. In our proposed design, a disagreement log is a training record, not a leaderboard. Otherwise the rational response could be to conceal dependence, accept the assistant’s answers and optimise for appearances—the exact behaviour the assessment needs to detect.

Three months is evidence, not a lifetime verdict

There is a legitimate counterargument: professionals have always learned with tools, and unaided performance is not the sole purpose of work. A calculator user need not become faster at mental arithmetic for the calculator to be worthwhile. But that argument does not settle which underlying skills are dispensable. If an employee must recognise an implausible result, handle an unfamiliar case or supervise an automated system, some independent judgement still has a job to do.

The trial’s duration, professional setting and particular assistant bound what can be concluded. A different interface, coaching programme or allocation of tasks could produce a different learning pattern. The study is a working paper, and its sponsor disclosures belong beside its strengths rather than being hidden or treated as automatic grounds for dismissal.[1]

The practical implication is neither “ban the assistant” nor “let the assistant teach everything”. It is to stop assuming that excellent output proves excellent learning. The strongest AI workplace will deliberately preserve the moments that build judgement—and automate the rest where doing so genuinely improves the work.

Report an error or suggest a correction