Tag: The Unfolding Thought Podcast

  • AI Doesn’t Transform Work. Management Systems Do.

    AI Doesn’t Transform Work. Management Systems Do.

    AI can turn knowledge work from craft into process. Whether that produces a golden age of work or an AI slop factory depends on what leaders redesign around it.

    About an hour into my conversation with Neel Doshi, he described a chief technology officer who had a puzzle.

    The company’s engineers were coding faster with AI. The expected business improvement never appeared.

    Neel asked the question that should have come before the rollout: If accelerating engineering did not accelerate the company, what made anyone think engineering was the bottleneck?

    This is question about AI at work separates output from performance.

    AI can help a person write more code, produce more slides, generate more campaign concepts, summarize more meetings, and answer more email. Every one of those activities can increase while the organization stays exactly where it was or gets worse. The code waits for a decision. The slides create another review cycle. The campaign concepts converge on the same familiar ideas. The meeting summary gives ten people ten more things to read.

    The problem is not that the tool failed. The problem is that we accelerated one part of a system and called it transformation.

    Faster work is not necessarily a faster organization

    The early evidence on AI productivity is real, but it is not uniform.

    In a study of 5,179 customer-support agents, researchers found that a generative AI assistant increased issues resolved per hour by 14 percent on average and 34 percent among novice and lower-skilled workers. The tool appears to have helped less-experienced workers apply practices that stronger workers had already learned.

    A separate experiment with 758 consultants found similarly large gains on tasks that sat within AI’s capabilities. People with AI completed 12.2 percent more tasks, worked 25.1 percent faster, and produced higher-quality results. But on a task outside that capability frontier, the people using AI were 19 percent less likely to reach the correct answer.

    Then, there is the uncomfortable counterexample. In a randomized trial involving experienced open-source developers working in codebases they knew well, AI use made them 19 percent slower. Before the work, they expected AI to make them 24 percent faster. Afterward, they still believed it had made them 20 percent faster. The tools have improved since that 2025 study, and METR’s 2026 follow-up says newer systems probably produce more benefit, but selection effects made the size of that benefit impossible to estimate reliably.

    These findings do not cancel one another out. Together, they tell us something more important: “Does AI improve productivity?” is an underspecified question.

    Whose productivity? On what task? At what level of experience? Measured at what point in the workflow? Over what period? At what cost to the next person in the system?

    The practical unit of analysis cannot be the prompt or even the worker. It has to be the system that turns work into an outcome.

    If AI helps a copywriter produce twice as many drafts but the client still approves one campaign a month, draft production was not the constraint. If an engineer produces code faster but product decisions, security reviews, and customer adoption do not move, coding was not the constraint. If managers use AI to generate more feedback but employees need more time to interpret generic advice, the organization has transferred effort rather than eliminated it.

    Activity is easy to accelerate. Throughput is harder.

    From craft to process: why that phrase is dangerous

    Neel’s most provocative claim was this:

    “What AI can do for knowledge work is what mechanized factories did for manufacturing.”

    Manufacturing moved from craft toward process. Neel believes AI can do the same for knowledge work.

    It is an illuminating analogy, but only if we resist the easiest interpretation of it. Process does not mean that knowledge work becomes perfectly repeatable. It means we become more explicit about how work moves, where variation enters, who evaluates it, and how the system learns.

    Modern manufacturing was never as deterministic as it looks from a distance. Materials vary. Machines drift. Conditions change. People discover better methods. Systems such as the Toyota Production System and Six Sigma were built to notice and manage variation, not to pretend it had disappeared.

    Generative AI makes the variation more obvious. The same instruction can produce different answers. A model can perform brilliantly on one task and fail on another that looks almost identical. Its capabilities change. The surrounding data change. The meaning of “good” often depends on context that exists only in the heads of customers, operators, and subject-matter experts.

    That means expert judgment is not a final inspection step tacked onto an AI process. Judgment is part of the production system.

    This is where the factory analogy can mislead us. The worst version of industrialization reduces the person to a monitor of the machine: wait for output, check a box, fix the occasional defect. That would turn a great deal of knowledge work into the cognitive equivalent of watching a conveyor belt.

    The better analogy is a production system in which the people closest to the work can stop the line, diagnose a problem, change the method, and teach the improvement to everyone else.

    AI can make knowledge work more process-like without making people more machine-like. But only if leaders distribute the authority to question and improve the process. Otherwise, we will not get continuous improvement. We will get continuous generation.

    Four modes of AI augmentation, organized by individual versus collective use and unstructured versus structured use.

    Neel described four broad modes of augmentation. Most organizations begin in the lower-left corner and mistake that starting point for the whole field.

    The performance dip is real. It is not a blank check.

    Neel offered a useful definition of transformational change: performance gets worse before it gets better.

    People need to learn new tools, but that is the smaller part of the problem. They also have to learn how to delegate to a system that does not understand responsibility, how to evaluate fluent output that may be subtly wrong, how to break work into parts, how to give useful feedback, and how to decide when a human conversation is the shortest route.

    In the middle of that learning curve, an experienced person can reasonably think, “I could have done this faster myself.” Sometimes they are correct.

    Economists have described a related pattern as the Productivity J-Curve. General-purpose technologies often require investments that ordinary productivity measures do not capture well: redesigned processes, new skills, new organizational structures, and new business models. During the investment period, measured productivity can stagnate or decline. The gains arrive later, if the complementary changes actually work.

    That gives leaders a reason to tolerate a dip. It does not give them permission to romanticize failure.

    Not every valley leads somewhere. A team can get worse because it is learning, or because the use case is poor, the tool is unreliable, the work is badly designed, or the underlying problem was never worth solving. Calling every disappointment “transformation” is how a temporary experiment becomes a permanent drain.

    A responsible performance dip has boundaries:

    • A specific capability the team is trying to develop.
    • A real work outcome, not an adoption target.
    • A baseline against which the change can be judged.
    • A protected period for practice.
    • Clear risks that require human review.
    • A date when the team will decide whether to continue, change course, or stop.

    “Use AI more” satisfies none of these conditions. It is not a strategy. It is not even a complete instruction.

    Individual acceleration can create a collective tax

    One of the people Neel spoke with during an AI transformation was an engineer who had not talked to another person in four days. She was producing work, but the way the company had introduced AI was removing the collaborative part of the job, the part she enjoyed.

    It is tempting to dismiss this as a preference. Some people like quiet, independent work. Not every task needs a meeting, and collaboration can become its own form of bureaucracy.

    But knowledge work depends on more than the sum of individual outputs. Teams combine perspectives, expose assumptions, carry tacit context, notice weak signals, and decide what is worth doing. Those functions can be slow. They are also where much of the organization’s judgment lives.

    Recent research makes the tradeoff clear.

    In one experiment, access to AI ideas helped individuals write stories that evaluators found more creative and enjoyable. Across the group, however, the stories became more similar to one another. Everyone could improve locally while the collective pool of ideas narrowed.

    Another set of experiments found that human-AI collaboration improved immediate task performance but was followed by lower intrinsic motivation and more boredom when people returned to solo work. That does not prove AI will make work demotivating; the studies used bounded online tasks. It does warn us that the experience of performing better and the experience of developing capability are not the same thing.

    The collaboration evidence is not uniformly negative. A field experiment with 791 Procter & Gamble professionals found that individuals working with AI matched the performance of two-person teams without it. AI also helped technical and commercial professionals produce more balanced solutions, crossing some of the functional boundaries that usually shaped their ideas.

    That is a meaningful benefit. It also raises a management question: If AI can reproduce some benefits of a team for one task, what human capabilities still need a team to develop over time?

    The answer will differ by work. The point is to ask before collaboration disappears by accident.

    “AI native and people first” has to be testable

    Neel recommends that leaders describe the goal as “AI native and people first.” It is better language than “AI first” because it refuses to rank the tool above the people, customers, and purpose of the organization.

    But a phrase this appealing can become decoration. To mean anything, it has to change decisions.

    I would look for six signs.

    1. The organization measures the outcome of the system

    Do not lead with prompts written, licenses activated, hours “saved,” drafts produced, or agents built. Measure cycle time from need to usable outcome, downstream rework, error rates, customer response, decision quality, revenue, cost, or whatever the system exists to accomplish.

    The closer the metric is to clicking the AI button, the easier it is to mistake motion for value.

    2. Expertise is inside the loop

    Neel noted that he can detect weak AI writing about his own research but may not recognize errors in an explanation of quantum physics. That should determine where AI is used and who reviews it.

    The person who can judge the output needs enough context, time, and authority to change the process, not merely approve what the system hands them.

    3. The process preserves divergence before it converges

    When everyone begins with the same model, the model can become a hidden agenda setter. It defines the first set of possibilities, and people refine from there.

    For work that depends on novelty, teams should sometimes think independently before using AI, deliberately seek disconfirming views, and compare genuinely different approaches. Faster convergence is valuable only after the search space is rich enough.

    4. Learning is funded as real work

    If people are expected to learn only after their normal work is finished, the organization is choosing nights and weekends as its AI strategy.

    Protected practice time, real tasks, feedback from experts, and permission to discard a weak use case are investments in capability. A one-hour prompt-training session followed by an adoption quota is not.

    5. Leaders redesign their own work

    Leaders cannot remain at the level of saying, “Build an agent for that.” They need direct experience with the tool’s strengths, failure modes, and demands on judgment.

    More importantly, they need to change the calendars, approvals, meeting habits, decision rights, and incentives that keep everyone else working in the old way. Technology added on top of an unchanged management system usually becomes one more layer of work.

    6. The organization protects contribution, not just employment

    Neel’s hopeful scenario is a “golden age of work” in which AI helps more people solve problems and improve the systems around them. That requires more than promising not to eliminate jobs.

    People need consequential choices to make, skills to build, colleagues to learn with, and evidence that their contribution changed the outcome. A job can survive while everything motivating about it is removed.

    The scarce resource is no longer production

    The cost of producing a plausible draft has collapsed. The cost of deciding what deserves attention has not.

    As generation becomes abundant, several things become scarcer: trustworthy judgment, distinctive ideas, shared context, willingness to revise, and the motivation to take ownership of an outcome. Those are not soft concerns around the edge of AI adoption. They are the constraints that determine whether faster generation becomes value or noise.

    Neel is right that AI creates an unusual opening. Organizations can use it to give more people access to context, coaching, and the ability to improve work that once felt fixed. They can also use it to increase output, centralize control, weaken apprenticeship, and fill every channel with artifacts no one needed.

    The technology does not choose between those futures. The management system does.


    Eric Pratum and Neel Doshi, guest on The Unfolding Thought Podcast.
    Eric Pratum with Neel Doshi for The Unfolding Thought Podcast.

    Listen and read further