TL;DR: The controlled evidence on AI and knowledge work looks contradictory until you sort it. A Science experiment with 453 professionals found average task time fell 40 percent. A Quarterly Journal of Economics study of 5,172 support agents found 15 percent more issues resolved per hour, with the most experienced agents gaining almost nothing in speed and losing a little quality. An Organization Science field experiment with 758 Boston Consulting Group consultants found 12.2 percent more tasks completed and 25.1 percent faster on 18 in-frontier tasks, but 19 percent lower odds of a correct answer on one task deliberately placed outside the frontier. A 2025 randomized trial of 16 experienced open-source developers working in repositories they knew for about five years found AI made them 19 percent slower, while they predicted it would make them 24 percent faster. Ordered by the user’s task-specific expertise, those five results form a clean gradient: the deeper your existing mastery of the exact task, the smaller and eventually the more negative the measured effect. A day that scales is therefore a day that is layered, with AI concentrated in the layers where expertise is shallow and deliberately restrained in the layers where it is deep.
Most advice about working with AI is written as if the tool has one effect size. Adopt it, get faster. The controlled evidence does not support that, and the disagreement between studies is not noise. It is the finding.
Four research groups have now run credible experiments on generative AI and professional work, using different populations, different tasks and different designs. Their headline numbers span a range of roughly sixty percentage points, from a 40 percent reduction in time taken to a 19 percent increase. Any framework that cannot explain that spread is not a framework. It is a preference.
What five controlled results actually measured
Every figure below is taken from the published abstract of the study named. Nothing here is inferred or rounded from secondary coverage.
| Study | Population and design | Measured effect |
|---|---|---|
| Noy and Zhang, Science, 2023 | 453 college-educated professionals; preregistered online experiment; occupation-specific incentivized mid-level writing tasks; half randomly exposed to ChatGPT | Average time taken decreased by 40 percent; output quality rose 18 percent; inequality between workers decreased |
| Brynjolfsson, Li and Raymond, Quarterly Journal of Economics, 2025 | 5,172 customer-support agents; staggered introduction of a generative AI conversational assistant | Issues resolved per hour rose 15 percent on average; less experienced and lower-skilled workers improved both speed and quality; the most experienced and highest-skilled workers saw small gains in speed and small declines in quality |
| Dell’Acqua and colleagues, Organization Science | 758 knowledge workers at Boston Consulting Group; preregistered; three arms; 18 realistic consulting tasks inside the AI capability frontier | Subjects using AI completed 12.2 percent more tasks and completed them 25.1 percent more quickly on average, with significantly improved quality |
| Dell’Acqua and colleagues, same experiment, out-of-frontier task | One complex managerial task selected to sit outside the frontier of AI capability | Subjects using AI were 19 percent less likely to produce correct solutions than those without AI |
| METR randomized controlled trial, 2025 | 16 experienced open-source developers with moderate AI experience; 246 tasks in mature projects on which they averaged five years of prior experience | Allowing AI increased completion time by 19 percent; the same developers forecast a 24 percent reduction beforehand and estimated a 20 percent reduction afterwards |
Two details in that table deserve to be read twice.
The first is the last row’s forecast error. The developers were not naive. They had used the tools, they were working in code they knew intimately, and they were asked to predict the effect before and estimate it after. They were wrong in the same direction both times, by roughly forty percentage points relative to the measured outcome. Economists asked to predict the same experiment expected a 39 percent reduction, and machine-learning researchers expected 38 percent. The intuition that AI is helping is not reliable evidence that it is, and it is least reliable exactly where people feel most confident.
The second is the Brynjolfsson row’s internal split. The same tool, in the same job, at the same firm, helped the least experienced agents on both speed and quality while producing small speed gains and small quality declines for the most experienced. The average of 15 percent is a real number that describes almost nobody.
The gradient nobody names
Put the five results in one order: how much task-specific expertise the user already had in the exact work being measured. Not general seniority, not years in the profession, but depth in that particular task.
Table: Measured AI effect ordered by the user’s task-specific expertise (CEOtudent editorial framework, derived from the five studies cited above)
| Rank | Who was doing the work | Depth in that exact task | Measured direction |
|---|---|---|---|
| 1 | Professionals writing mid-level documents of a kind they produce occasionally, not as their core craft | Shallow | Time down 40 percent, quality up 18 percent |
| 2 | Support agents with least experience and lowest baseline skill | Shallow | Speed and quality both improved |
| 3 | Consultants on 18 standard analytical and creative tasks | Moderate | Time down 25.1 percent, throughput up 12.2 percent |
| 4 | Support agents with most experience and highest baseline skill | Deep | Small speed gain, small quality decline |
| 5 | Consultants on a managerial task outside the model’s competence | Deep, and the task is off-frontier | 19 percent less likely to be correct |
| 6 | Developers in repositories they had worked in for about five years | Very deep | Time up 19 percent |
The direction is monotone. As task-specific depth rises, the measured effect falls, passes through roughly zero somewhere around rank four, and turns negative. This ordering is CEOtudent’s; the underlying results are the researchers’.
It is worth being precise about what this does and does not establish. These are five different experiments with different populations, tasks and outcome measures, so the ordering is an interpretive synthesis rather than a single estimated curve. No study has yet varied task-specific expertise experimentally while holding everything else constant. What can be said is that the ranking is consistent across all five, that no result violates it, and that each research team independently reported a skill- or expertise-related split running in the same direction inside its own data.
The mechanism is not mysterious. AI assistance substitutes for the part of the work you would otherwise have to construct from scratch. When you have little existing structure for a task, that substitution is enormous. When you have deep structure already, the model’s output arrives as something you must read, evaluate, reconcile against what you already know, and usually repair. Evaluation is not free. Past a certain depth of expertise, the cost of checking exceeds the cost of doing. That is the same reason judging AI output is itself a distinct skill rather than a by-product of using the tools.
The five layers
If the payoff varies by depth, then the unit of design is not the tool and not the day. It is the layer. A knowledge worker’s day decomposes into five kinds of work with structurally different expertise profiles, and each one deserves a different standing policy.
Table: The 5-Layer AI Workflow (CEOtudent editorial framework)
| Layer | What happens here | Standing AI policy | Why, in evidence terms |
|---|---|---|---|
| 1. Intake | Reading, searching, meetings, inbound messages, gathering raw material | Maximum delegation. Summarize, retrieve, cluster, triage | Depth is shallow by definition; you have not formed a view yet. This is the rank-1 and rank-2 zone |
| 2. Drafting | Producing first versions: documents, plans, code skeletons, replies, outlines | High delegation, always as a first draft never a final one | The Noy and Zhang and in-frontier consulting results live here. Large, reliable, well-replicated gains |
| 3. Judgment | Choosing between options, setting direction, accepting trade-offs, deciding what not to do | Human owns the decision. Use AI adversarially: ask it to argue the case against your choice | The out-of-frontier result is a judgment failure that looked like an analysis task. The model gave no signal that it had crossed the line |
| 4. Craft | The narrow work you are genuinely expert in, where your reputation actually lives | Restrained. Delegate mechanical sub-steps only; keep the substantive work manual by default | The METR result. This is the one layer where the measured effect was negative, and the users could not feel it |
| 5. Verification | Checking, reconciling, catching what the earlier layers got wrong | Cannot be delegated to the same system that produced the output. Budget real time for it | Every gain in layers 1 and 2 increases the volume flowing into layer 5. Skipping it is how the frontier eats you |
The layers are not a schedule. They are a policy set. Any given hour may contain several of them, and the point is that the correct AI posture changes between them even when the tool on your screen does not.
The reallocation rule
Here is where most adoption goes wrong, and it is a structural error rather than a discipline error.
Layers 1 and 2 compress dramatically under AI. That is the whole promise, and the evidence supports it. But layer 5 does not compress. It expands, because it now has more output to check, produced faster, by a system whose failure modes are not visually distinct from its successes. The Organization Science finding is precisely this: the task that sat outside the frontier looked no harder than the eighteen inside it, and the model gave no indication of having crossed over.
So the rule is simple and almost nobody follows it. The time saved in intake and drafting is not free. A defined share of it belongs to verification, permanently.
A practical starting split: return roughly a third of measured time savings to layer 5 for the first quarter of any new AI-assisted workflow, then adjust based on how many errors verification actually catches. If verification is catching nothing over several weeks, the budget is too large and the work has probably drifted into layers where AI was already safe. If verification is catching things you would have shipped, the budget is too small. This allocation is a working rule rather than a measured optimum, and it should be treated as a starting hypothesis to be tuned against your own error log.
The second half of the rule concerns layer 4. Protecting your craft layer from casual AI use is not nostalgia and it is not a productivity choice. It is a measurement problem. The METR developers experienced a speed-up while being slowed down, and their estimate was still wrong after the work was finished. If you cannot perceive the effect in your own expert work, you cannot manage it by feel, and the only correct default is restraint until you have measured it. That is also the honest reason to run a structured workflow audit rather than trusting your sense of where the time goes.
Running it for one week
The framework is only useful if it changes a calendar. A minimal implementation takes a week and produces evidence rather than opinion.
Days one and two: label, do not change. Go through your existing day and tag each block with one of the five layers. Most people discover two things. Layer 3 has almost no dedicated time and happens in the gaps between other work. Layer 5 has none at all, because verification is currently assumed rather than scheduled.
Day three: set the standing policies. Write one line per layer stating what AI is allowed to do there. The value is in writing it down, because an unwritten policy defaults to whatever the tool makes easiest, and what the tool makes easiest is using it everywhere.
Days four and five: run the policies and log every correction. Every time verification catches something, record which layer produced it. This log is the only reliable instrument you have, and it is what deliberate delegation boundaries should be built on rather than intuition.
End of week: reallocate. Compare where time was saved against where errors were found. Move calendar time accordingly. Then repeat monthly, because the frontier moves and your policies should be dated.
This is the same underlying method as auditing which tasks to automate first and it sits naturally alongside a structured briefing framework for delegating to agents. The distinction the layer model adds is that it stops treating your whole day as one delegation decision.
Why this is a CEO and student problem at once
The CEO half of the job is allocation. Given five layers with different measured returns, the executive question is where capacity goes, and the answer is not “everywhere the tool works”. It is: maximum delegation where depth is shallow, restraint where depth is deep, and a permanent standing budget for the checking function that scales with everything you automated.
The student half is harder, because it requires accepting that your perception of your own performance is the least reliable instrument you own in exactly the domain you know best. The developers in the METR trial were experts, they were measured, and they still could not feel a 19 percent slowdown. The only defence against that is to keep measuring rather than to keep believing.
A day that scales is not a day with more AI in it. It is a day where AI is placed by layer, where the savings are consciously reallocated, and where the expert work is defended until evidence, not enthusiasm, says otherwise.
Frequently asked questions
Does this mean experts should not use AI at all?
No. It means experts should not use it by default in their narrow area of mastery, and should keep using it heavily in the other four layers. An expert’s intake and drafting layers are not expert layers. A senior developer summarizing a specification is at rank one on the gradient, not rank six.
How do I know which of my work is layer 4?
The practical test: it is the work people come to you specifically for, and the work where a subtle error would be visible to someone who knows the field. If a wrong answer would be caught by any competent reader, it is probably layer 2. If it would only be caught by a peer, it is layer 4.
Is the 19 percent slowdown result generalizable?
It is one trial with 16 developers and 246 tasks, so it should be read as a strong signal rather than a settled parameter. The authors collected and evaluated evidence on 20 properties of their setting that could a priori have contributed to the slowdown, and reported that the effect was robust across their analyses, while noting that experimental artifacts cannot be entirely ruled out. Its main value is that it is the only entry in the table where careful measurement contradicted every prediction, including the participants’ own.
Why is verification its own layer rather than part of drafting?
Because it has a different owner and a different failure mode. Drafting fails visibly, by producing nothing. Verification fails invisibly, by producing confidence. Layers with invisible failure modes need their own calendar time, or they get absorbed into whatever is louder.
What if my organisation measures only output volume?
Then the gradient predicts you will be rewarded for the first two quarters and exposed in the third, because layers 1 and 2 move volume immediately while layer 5 failures surface later. The defensible position is to keep your own error log, so that when quality questions arrive you have evidence about where errors originated rather than an argument about effort.
Sources
Noy, S. and Zhang, W. Experimental evidence on the productivity effects of generative artificial intelligence. Science, 2023.
Brynjolfsson, E., Li, D. and Raymond, L. Generative AI at Work. The Quarterly Journal of Economics, 2025.
Dell’Acqua, F. and colleagues. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. Organization Science.
METR. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. Randomized controlled trial report, 2025.
Harvard Business School Digital Data Design Institute, Division of Research and Faculty Development, acknowledged funder of the Boston Consulting Group field experiment.
This content was compiled with the support of AI following in-depth research, then written and prepared for publication by the CEOtudent editorial team.
This post is also available in:
