The Reality Behind the Rhetoric: Making the Case for AI Augmentation Over Replacement
Introduction
From the outset, let me be clear: I love artificial intelligence and have witnessed its transformative potential. In my experience, AI can drastically streamline workflows, enhance decision-making, and unlock new possibilities when applied thoughtfully. But I’m also a realist, and lately the narrative around AI in software engineering has drifted into unrealistic hype. We are not seeing AI inevitably replace jobs by its own doing, rather, we are seeing people in positions of power choosing to replace employees with AI tools to cut costs. This is a critical distinction. It’s an intentional business decision rather than a technological inevitability. Frankly, I disagree with using AI primarily as a direct replacement for human workers. The far better approach is to leverage AI to augment human talent instead of eliminating it. In this article, I take a pragmatic yet optimistic look at AI’s true impact. We will examine the current productivity paradox (why claims like “AI wrote 30% of our code” only translated into ~10% faster development), the new bottlenecks and hidden costs that AI introduces, the economic hype cycle driven by elusive ROI, and the enduring areas where human judgment remains supreme. Along the way, we’ll see that these patterns and trade-offs around AI adoption are appearing not just in software engineering but also in healthcare, finance, manufacturing, and retail. Finally, we’ll discuss why embracing AI as a tool for human augmentation, implemented ethically and strategically, is the most sustainable path forward.
Part 1: The Productivity Paradox – When 30% of Code Delivers Only a 10% Gain
AI’s proponents often tout staggering statistics about code generation, painting a picture of a revolution in programming. Tech leaders love to announce that AI now produces a large share of their company’s code: for example, Google CEO Sundar Pichai noted that over 30% of new code at Google is now generated with AI assistance (up from 25% a few months prior), and Microsoft reports its GitHub Copilot tool helps write 30–40% of its code. Meta’s Mark Zuckerberg has floated an even bolder vision of AI handling half of all coding work in the near future, and Salesforce’s Marc Benioff claims AI tooling boosted their engineering output by about 30%, so much so that Salesforce paused hiring new developers for 2025. On the surface, these figures suggest unprecedented efficiency gains in software development.
However, such headline numbers are deeply misleading without context. Developer feedback indicates that the 30% of code AI generates is largely confined to the easiest, lowest-value portions of the codebase. One engineer quipped that while their AI assistant might write half the lines of code, those were the very lines that used to consume only about 20% of their time: the boilerplate, repetitive helper functions and scaffolding. In other words, having AI draft 30% of the code does not equate to a 30% reduction in workload, it automates the routine tasks that were never the primary bottleneck. The focus on “percent of code written by AI” is essentially a revival of the old lines-of-code (LOC) metric, which the software industry abandoned long ago because more code does not equal more value. A senior engineer might deliver a huge improvement by deleting hundreds of lines of unnecessary code, something a LOC-based productivity metric would perversely score as negative productivity. AI coding assistants, being advanced text generators, excel at spitting out lots of code, making LOC-centric measurements look impressive. But this creates a distorted perception of progress that serves marketing hype more than it reflects real value created.
Only ~10% Actual Productivity Gain
In contrast to the high volume of AI-generated code, the metric that truly matters, engineering velocity (how quickly features and fixes are delivered), tells a far more modest story. Even Pichai has been careful to note that Google’s rigorous measurements show only around a 10% improvement in overall development speed despite all the AI-generated code. He emphasized that this velocity metric is “the most important” because it captures real business impact: how fast products get to users. Herein lies the productivity paradox: AI might be contributing 30% of the code, yet the team is only ~10% faster at delivering software. This 20-point gap represents a friction tax, the hidden overhead and new work introduced by using these AI tools. Within this gap, the polished narrative of “AI is supercharging programming” begins to crack, revealing a much more complex reality.
The Hidden “Productivity Tax” of AI
Why doesn’t 30% more code output translate into anything near 30% faster delivery? The answer is that AI introduces new tasks and inefficiencies that eat up the time it saves. Developers are paying a productivity tax in several forms of extra work:
Prompt “babysitting” and editing: Engineers must spend time crafting, refining, and reworking prompts to get useful output from the AI. It’s rarely a one-and-done deal, more often it’s a frustrating iterative dialogue. Guiding the AI and fixing its responses has become a new kind of cognitive overhead that isn’t captured in simple productivity stats. Developers jokingly call this “prompt babysitting”, since it can feel like supervising a willful child.
Rigorous validation and debugging: AI-generated code cannot be trusted blindly. Every suggestion requires careful review and often correction. Developer forums are filled with stories of Copilot or ChatGPT producing code that looks plausible but contains subtle bugs, inefficiencies, or downright bizarre approaches. In many cases, engineers spend more time debugging the AI’s output than it would have taken to write the code from scratch, especially for anything beyond trivial examples. This verification work falls on the human and reduces the net speed gain.
Handling AI “hallucinations”: Current AI models sometimes produce outputs that are confidently wrong or entirely made-up, a phenomenon known as hallucination. For example, one team reported that their AI assistant, when asked to use their codebase, read a small portion of the project and then fabricated non-existent APIs and data structures for the rest. The result was code that didn’t run at all and weeks of lost effort before the ghost code was identified. Recovering from such misdirections is a real cost. It’s not a common occurrence, but when it happens the time sink is enormous.
These hidden costs mean that AI is not simply subtracting work from a developer’s plate, it’s also transforming the nature of the work. The role of the engineer shifts from being the primary author of code to being a critical editor and shepherd of AI-generated output. That demands new skills: effective prompt design, an ability to read AI suggestions with deep skepticism, and a strong architectural sense to spot where the AI’s code may not fit the bigger picture. In effect, the AI is doing the easy 30% of the work, and the human now has to do the harder 70%, plus oversee the AI. This helps explain why we see only a 10% overall productivity boost instead of 30%. In fact, these dynamics aren’t unique to software engineering: even across other industries, early studies indicate that despite widespread experimentation with generative AI, its contribution to total work output has been just a few percentage points on average. In other words, a lot of automated activity is translating into only a small net dent in overall productivity so far.
The key takeaway from this paradox is that AI hasn’t made the hard parts of engineering vanish, it has just shifted our effort toward new tasks like prompt management, thorough reviews, and high-level problem-solving. The next section looks at how these shifts manifest as new bottlenecks in the development process (and similarly in other fields), rather than the elimination of old ones.
Part 2: The New Bottlenecks – How AI Shifts (Rather Than Eliminates) Toil
AI was sold as a technology that would radically eliminate tedious work, freeing humans to focus on higher-level creativity. In reality, what we see is that AI relocates the bottlenecks of work rather than removing them altogether. In software development, the drudgery of writing boilerplate code might be reduced, but it has been replaced by new forms of drudgery: managing the firehose of AI output, performing longer code reviews, and fixing an influx of hidden issues (like security vulnerabilities) introduced by AI. The toil hasn’t gone away, it has shifted to different parts of the workflow. This pattern is mirrored in other industries as well. The use of AI often uncovers new challenges even as it solves old ones. For instance, overly aggressive automation in manufacturing famously backfired for Tesla’s Model 3 assembly line, where Elon Musk admitted that excessive reliance on robots slowed down production and that “humans are underrated” when it comes to solving problems. In that case, automation created a new bottleneck that had to be resolved by reintroducing human flexibility. We see analogous new bottlenecks in software engineering:
Longer, Heavier Code Reviews
One immediate consequence of AI-generated code is that code review, the process of human reviewers checking and approving changes, becomes more burdensome. AI assistants tend to produce code that is verbose and not always consistent with a project’s style or architecture. This means reviewers must wade through larger diffs (change sets), untangle more convoluted logic, and ensure the AI’s additions conform to standards. Reviews, which are critical for quality control, end up taking longer and demanding more concentration.
Moreover, human reviewers have become the last line of defense against a new class of AI-induced errors. An AI, lacking true understanding, might introduce a subtle security flaw or a performance issue that compiles fine but is architecturally wrong. Reviewers now have to scrutinize code for these non-obvious problems. This cognitive load is especially high when less experienced developers rely on AI and accept its suggestions uncritically. The onus then falls on senior engineers in review to catch issues that the juniors (and the AI) missed. In effect, some of the thinking that would have happened during coding now happens during review.
This phenomenon isn’t unique to coding. Whenever AI is used to generate work products, human oversight often balloons. In the financial sector, for example, if an AI system auto-generates investment recommendations or reports, compliance officers and analysts must carefully vet those outputs for errors or regulatory issues, a parallel to the extended code reviews in engineering. In healthcare, if an AI drafts patient notes or treatment suggestions, doctors must double-check every detail, sometimes spending as much time reviewing as they would writing from scratch. The common thread is that human experts become editors and guarantors of quality, picking through the verbose output of an AI to find and fix problems. The time and effort for this reviewing/editing step can significantly offset the time initially saved by automation.
AI as a Security Risk and Source of “Technical Debt”
Beyond slowing down reviews, AI-generated code is emerging as a significant source of security vulnerabilities and maintenance headaches. The reason lies in how these models are trained: by studying masses of existing code from the internet, a body of examples that includes countless bad habits and insecure patterns. Thus, an AI coding assistant might confidently produce code that uses a known-bad practice (like constructing an SQL query via string concatenation, which opens the door to SQL injection attacks) because it saw many examples of it in training. In one study, nearly 32% of code snippets suggested by GitHub Copilot contained potential security flaws, and another analysis found about 62% of AI-generated solutions had at least one significant defect or weakness.
The AI has no true understanding of security or architecture, it just knows what code looks correct based on patterns. As a result, it can create functionally working code that is architecturally catastrophic for security or scalability. Engineers are now discovering hidden bugs and vulnerabilities introduced by AI assistance, a new form of “technical debt” we might call AI-induced debt. Just as rushing a feature can incur technical debt that must be paid down later through debugging and refactoring, blindly using AI outputs can incur security debt that may only become apparent when there’s a breach or major failure. This debt accumulates silently. For example, if an AI-generated piece of code inadvertently hard-codes a secret API key or neglects an important access control check, it might pass initial tests and go into production, only to create a security hole that isn’t discovered until much later (perhaps after an incident). Traditional testing tools may not easily catch these issues because the code is syntactically correct, the flaw is in the design or context. The result is a codebase that looks more complete (thanks to AI generating lots of it) but has latent flaws that will require significant human effort to find and fix down the line.
Managing this risk has become a new burden on software teams. Security reviewers must now assume that any AI-written code could be harboring a vulnerability, prompting more exhaustive auditing. It’s analogous to how other industries must handle AI outcomes carefully. For instance, a bank using an AI to approve loans might later discover the model unintentionally learned a bias or regulatory no-no, forcing costly remediation. Or consider retail: some large retailers deployed AI-driven self-checkout systems and later found that while these automated checkouts reduced cashier labor, they led to higher shoplifting and error rates, effectively a new “security” problem in a different form. In fact, several major chains have recently dialed back their self-checkout deployments after discovering theft jumped as much as 65% with self-checkout vs. human cashiers. Those companies ended up hiring more security staff and implementing additional checks, negating much of the original efficiency gain. This retail example is a vivid parallel to what we see in software: AI can automate a task, but it creates new work to control the side effects (be it code vulnerabilities or store losses).
In summary, AI has shifted engineering toil into areas like code review overload and security cleanup, rather than making engineering toil disappear. The net result is that the expected gains are diluted by new costs, which is why, as discussed, a 30% boost in code output only yielded ~10% faster delivery in practice. These hidden costs and new bottlenecks help explain a broader trend: many organizations aren’t yet seeing the blockbuster returns on AI that the hype would suggest. This realization leads us to examine the economic side of the AI hype, why companies continue to push the AI narrative and how the ROI (Return on Investment) imperative is driving certain decisions.
Part 3: The Hype Cycle and the ROI Imperative
It’s time to address the elephant in the room: the glaring disconnect between the grand promises of AI and its practical impact on the ground. This disconnect isn’t just an honest mistake or innocent optimism, it’s largely the product of a powerful hype cycle fueled by economic pressure. Companies have poured enormous investments into AI (for example, Meta boosted its 2025 capital spending to an eye-watering $72 billion, much of it for AI initiatives), and now they face intense scrutiny to justify those costs. When the real returns prove smaller than expected, there’s a temptation, sometimes a compulsion, to inflate claims about what AI is achieving. In other words, the marketing hype is running far ahead of the reality because it has to, financially.
The Economics of Exaggeration: Elusive ROI
Despite executives extolling AI as a game-changer, many organizations are struggling to see substantial financial returns from their AI projects. A recent survey by Boston Consulting Group found the median ROI from generative AI initiatives is only about 10%, far below the 20%+ that many businesses hoped for. Tellingly, roughly one-third of business leaders (many of them CFOs) reported limited or no measurable gains from their AI investments. This is a sobering reality: even though nearly everyone is experimenting with AI in some form, the payoff in bottom-line terms has often been marginal.
The situation is so common that Gartner has predicted a significant AI backlash in the coming years. According to Gartner’s analysis, by the end of 2025 at least 30% of all generative AI projects will be scrapped at the proof-of-concept or pilot stage, abandoned because they fail to demonstrate real value or suffer from issues like poor data quality and uncontrolled costs. In plainer terms, many companies will conclude that their AI efforts just aren’t worth it and will pull the plug. This kind of projection from Gartner underscores how widespread the gap is between AI’s promised benefits and what many projects are actually delivering.
Faced with this ROI shortfall, companies have a strong incentive to exaggerate the impact of AI. If you’ve invested tens of billions (as some tech giants have), you need to show progress, if not in profits, then at least in impressive-sounding metrics (hence the fixation on things like “percent of code written by AI”). The hype also helps keep stock prices buoyant and budgets flowing. It creates a “fake it till you make it” dynamic: tout the potential and interim achievements loudly in hopes that eventually the reality will catch up. In the meantime, any lackluster results are glossed over or attributed to temporary issues. One worrying outcome of this pressure to show ROI is that some companies resort to cost-cutting via AI as a shortcut to boost metrics. This brings us to a critical ethical point often lost in the discussion of AI and jobs.
The Human Choice Behind “AI Replacing Jobs”
Despite the way it’s often portrayed, AI is not an autonomous agent deciding to eliminate human jobs. People are. Specifically, executives and managers are making deliberate choices about how to deploy AI. When you hear that “AI is replacing workers,” what’s really happening is that leadership has chosen to use AI in a way that allows them to lay off staff or freeze hiring, usually to reduce costs. The technology might enable that choice, but it doesn’t mandate it.
This distinction matters because it reframes layoffs and role eliminations as business strategy decisions rather than an inevitability of technological progress. For example, when Salesforce’s CEO publicly touted productivity gains from AI and concurrently announced a hiring freeze for engineers, that was a business decision cloaked in tech narrative. It’s not that AI forced Salesforce to stop hiring, but leadership opted to pause hiring because they believed they could maintain output with AI-assisted teams. Similarly, if a bank implements an AI customer service chatbot and then lays off a portion of its call center staff, it’s the bank’s decision to use AI in a replacement mode rather than in an assistive mode. We see this across industries: some retail chains introduced automated self-checkout kiosks and then reduced cashier shifts, not because the old checkout process stopped working, but because management decided to prioritize short-term labor savings (even at the cost of other issues like theft, as mentioned). In manufacturing, a factory might invest in robots and subsequently downsize its workforce, again, that’s a cost-driven choice of how to apply automation.
Why emphasize this? Because labeling these moves as “AI took our jobs” implies a kind of fate or inevitability, when in fact alternate choices exist. Companies could choose to augment their workers with AI rather than replace them. They could aim for each employee to become, say, 30% more productive with AI and keep those employees, leveraging the gains to increase output or quality. That is a very different scenario than cutting 30% of the workforce and trying to have AI fill in entirely. The augmentation approach tends to preserve domain knowledge, morale, and long-term adaptability, whereas the replacement approach often sacrifices those for short-term efficiency.
I fundamentally disagree with the strategy of using AI primarily as a tool for outright replacement. Yes, if a job truly consists of nothing but repetitive, routine tasks, then that job is vulnerable to automation (and arguably has been for decades, even before modern AI). But the vast majority of roles, in software engineering and beyond, involve more than routine repetition. They require creativity, problem-solving, interpersonal skills, and adaptability. In software, a great developer is not valuable because they can type out boilerplate code quickly, they’re valuable because they solve novel problems, design good systems, and make judgment calls. In customer service, a representative’s value is in empathizing with customers and handling unexpected issues, not just reading a script. These are the aspects of work where human judgment remains crucial, and where AI, at least as it exists today, cannot substitute for human capability. So when companies choose a replacement path, they are often trading away long-term strengths for a short-term productivity bump. It’s an ethical and strategic choice. And it’s often justified by a narrative that “the AI revolution made us do it,” which conveniently removes agency from those decision-makers.
But let’s be clear: the future involving AI is ours to shape. We can choose to use AI to empower employees, making one developer as effective as two, for instance, rather than to simply cut the workforce in half. The push for ROI and the choice of replacement vs. augmentation also have to contend with a stubborn reality: there are certain things current AI just can’t do well. No matter the hype, there remain unbreakable barriers and hard problems that require human intellect. These limitations are a strong argument for why replacing people wholesale with AI is not just ethically fraught but also impractical. In the next section, we’ll explore these human-centric domains in software engineering that illustrate the ceiling of what AI can achieve on its own.
Part 4: The Unbreakable Barriers – Where Human Judgment Remains Supreme
For all the talk of AI’s exponential improvements, there are fundamental aspects of software engineering that have proven deeply resistant to automation by today’s AI. These are the “hard problems,” the complex, context-dependent, creative tasks where human judgment is not just slightly better than AI, but categorically different. They often involve dealing with ambiguity, devising novel solutions, or understanding nuance that isn’t present in any training data. Current AI, which excels at pattern recognition and imitation, struggles or utterly fails in these areas because success requires more than just learning from past examples. Let’s look at a few of these unbreakable barriers in software (and note that analogous challenges exist in other fields):
1. Architectural Design – Deciding What to Build and How
The most challenging and valuable work in software development isn’t typing out code, it’s figuring out what system to build in the first place, and how to structure that system. This is the realm of software architecture and design. Architects take vague, evolving requirements from the business and turn them into a concrete technical vision. They anticipate future needs, weigh trade-offs (like speed vs. security, or cost vs. scalability), and often invent new approaches when existing patterns won’t do. This process is as much art as science. It requires creativity, foresight, and sometimes leaps of intuition, essentially creating new patterns rather than just applying old ones.
AI, by contrast, is inherently backward-looking. A code-generating AI is trained on what architects and developers have done before. It can certainly help apply known design patterns or suggest code to implement a familiar architecture. But ask it to design something truly novel or solve a problem that hasn’t been solved before in the training data, and it’s stumped. The AI has no genuine understanding of the problem’s context or the abstract reasoning needed to, say, decide between a microservice architecture and a monolith for a particular new product, especially if that decision hinges on business strategy, team skillsets, regulatory considerations, etc. All it can do is regurgitate pieces of architectures it has seen. It cannot invent a fundamentally new architecture to meet a unique challenge.
In practice, humans still have to do the systems thinking and make the high-level design decisions. We see parallels in other industries. For instance, in healthcare, defining a treatment plan for a patient with a very rare combination of conditions is something that goes beyond existing “patterns.” A doctor might have to improvise and use first principles, whereas an AI would have no past cases to adequately draw on. In business, crafting a strategy to navigate an unprecedented market situation similarly can’t be looked up in historical data. These are situations where human creativity and judgment call the shots.
2. The Legacy System Quagmire
A very common, unglamorous task in enterprise software engineering is dealing with legacy systems: old, large, creaky systems (often decades old) that are deeply ingrained in a company’s operations. Modern AI tools have a very hard time here. Legacy codebases might be written in outdated languages (think COBOL from the 1970s), have minimal or no documentation, and contain myriad idiosyncratic fixes applied over many years. Understanding and safely modifying such a system is like archeology mixed with puzzle-solving. It requires a deep contextual understanding that is built up through experience and investigation, not something an AI model can instantly obtain by training on public code (because these legacy systems are often proprietary and unique).
An AI cannot simply “read” a 30-year-old COBOL program that’s been patched by hundreds of different engineers over the decades and understand the intent behind it. It will see the code, sure, but without documentation or a specification, the AI has no idea why things were done a certain way or what obscure business rule is embedded in that bizarre-looking piece of logic. Human engineers, by contrast, can slowly piece together that understanding by reading code, running experiments, and talking to people who maintain the system. Integrating modern AI with such legacy systems remains a very human-intensive challenge, it’s not simply a matter of generating more code. In fact, throwing AI-generated code at a legacy system without deep care can make things worse, because you might introduce changes that violate assumptions hidden in the old code. This is one reason many AI pilot projects in enterprises stall out. The tool might work fine on a greenfield project, but as soon as it encounters the ugly reality of the legacy stack, progress grinds to a halt. (It’s telling that in healthcare, where legacy electronic record systems and processes abound, only 30% of AI pilot projects ever make it to full production deployment, largely due to integration challenges and lack of contextual data readiness.) Human judgment, patience, and domain expertise remain the keys to navigating legacy systems in software, just as seasoned professionals are needed to navigate legacy processes in other industries (for example, a seasoned operations manager figuring out how to retrofit a 40-year-old manufacturing line for a new product. An AI wouldn’t have the necessary holistic understanding).
3. The “Physics” of Distributed Systems – Debugging the Undebuggable
Modern software systems are often distributed, running in the cloud across many machines, handling tons of concurrent events. These systems can exhibit complex, emergent problems like race conditions (where timing of events causes bugs) or intermittent failures that are notoriously hard to reproduce. Debugging such issues can feel less like traditional programming and more like detective work or even an experimental science. You formulate hypotheses, add extra logging, try to trap the bug in action, often working with incomplete data. It’s a task defined by dealing with uncertainty and incomplete information.
Current AI, which relies on spotting patterns in data, is ill-suited for these kinds of problems because by their nature, these bugs don’t follow a clear pattern. If they did, we likely would have fixed them already. For example, a race condition might only occur once in a million runs when just the “right” timing alignment happens between two services. There may be no training data that captures this, because it’s so rare and context-specific. Humans can use intuition and system knowledge to zero in on likely causes (for instance, suspecting a particular module when certain symptoms appear). AI doesn’t truly understand causality or have intuition, it just knows correlations from past data. In a large distributed system, many issues violate the assumptions that AI-based tools make. The systems are non-deterministic (the same input can lead to different outcomes depending on timing), and they often have hidden state or partial observability (we can’t perfectly log every detail without altering the system’s behavior, a Heisenberg-esque “observer effect”). So debugging becomes as much an art as a science, one that humans, with creativity and perseverance, still perform.
We can find echoes of this in other fields too. Think of diagnosing a sporadic issue in a complex mechanical system, say, a certain model of aircraft experiences an unusual vibration only under very specific conditions. Engineers (human ones) might spend months testing and hypothesizing to find the root cause. An AI that only knows normal operating data would be at a loss, because the problem manifests in the edge cases. Or consider chasing down the cause of a rare side effect in a new drug. Scientists rely on deep domain expertise and intuition since there may be no prior data connecting the dots. These scenarios, like debugging distributed software, underscore how human insight remains crucial when confronting new, intricate problems that aren’t amenable to brute-force pattern matching.
The examples above illustrate that human judgment and creativity continue to reign supreme in critical areas of software engineering. Architectural vision, contextual understanding of legacy quirks, and creative problem-solving in debugging are all beyond the reach of today’s AI. This is not to belittle AI’s capabilities, but to put them in perspective: AI is a powerful tool within a defined scope, but it hits a ceiling when tasks demand fundamentally human qualities. Recognizing these limits helps us avoid the trap of overhyping AI’s abilities. It also reinforces the argument that we should be using AI to support humans in these complex tasks, not to replace them. In the final section, we’ll outline what a pragmatic, human-centric approach to AI looks like, one that maximizes benefits while respecting the limitations and the ethical considerations we’ve discussed.
Part 5: A Realist’s Manifesto – AI as Augmentation, Not Annihilation
The gap between the breathless hype and the nuanced reality of AI’s impact calls for a more sober and balanced perspective. Instead of buying into what some have termed “VC theater” (dramatic proclamations that AI will imminently disrupt and replace everyone), we should ground our strategy in the evidence at hand. And the evidence suggests that AI’s most powerful role is as an augmenting tool alongside humans, not as an independent agent rendering humans obsolete. In software engineering (as in many fields), the future is looking far more like collaboration between humans and AI than a wholesale handoff of the reins to algorithms.
The Asymptotic Productivity Curve
One of the central insights from observing AI’s deployment in coding is that the productivity improvements seem to follow an asymptotic curve: a law of diminishing returns. The first gains are significant: automating the tedious 20% of coding (like writing boilerplate, basic unit tests, etc.) can indeed yield a noticeable boost in speed and efficiency. That’s the steep part of the curve. But as you try to automate progressively more of the work (into the more complex 80% of coding that remains), each additional percentage point of AI contribution yields a smaller and smaller benefit. The curve flattens out. Why? Because, as we’ve detailed, the harder the task, the more overhead and assistance the AI itself requires, and the more you run into things AI can’t easily do. Unless there’s a fundamental breakthrough that allows AI to truly comprehend context, handle design decisions, and reliably self-correct (capabilities far beyond current models), we shouldn’t expect the curve to suddenly turn upward again. It’s asymptotic, approaching a limit.
For example, getting from 0% to 30% AI-generated code gave some efficiency improvement (perhaps that ~10% velocity gain at Google). But trying to go from 30% to 60% AI-generated code is not going to double that gain. In fact, without major changes, it might barely budge the needle beyond the initial bump, because the remaining work is the kind AI struggles with. Each increment of automation comes with more support costs, and the net gains shrink. We see analogous “flattening” in other domains as well. A hospital might automate scheduling and some documentation with AI (getting a modest productivity bump), but automating the complex diagnostic work might prove elusive and yield little additional improvement. A factory can automate certain assembly steps, but beyond a point, additional automation runs into sharply rising complexity and diminishing returns, as Tesla discovered when overly ambitious automation led to a production logjam that humans then had to fix. The takeaway is not that AI has no benefit, it’s that AI is best at the assistive, well-bounded tasks, and when we push it beyond that, we encounter steeply rising difficulties. Knowing this, how should we proceed?
AI as a Tool for Augmentation, Not Replacement
The most productive and sustainable way to use AI is to treat it as a tool that enhances human capabilities. In practical terms, this means redesigning workflows so that AI handles what it’s good at (speedy number-crunching, generating drafts, suggesting solutions based on known patterns) while humans handle what they’re good at (judgment, critical evaluation, invention, relationship-building). It’s a partnership. In software engineering, that might mean an AI suggests code or test cases and a developer reviews and integrates them. The developer now works faster and can focus more energy on the tricky parts of the problem. In customer service, an AI chatbot might handle the simple FAQs and route more complex issues to human agents, who then have more time to give those customers personal attention. In medicine, AI might sift through medical images and flag areas of concern for a radiologist, who then makes the final call and spends more time on difficult cases. In all these examples, AI is an assistant, not a replacement, and the human is still very much in the loop, empowered by the AI to be more effective.
Crucially, many industry leaders who are actually implementing AI at scale have converged on this augmentation philosophy. Sundar Pichai at Google, for instance, often describes AI as a “companion” or an “assistant” to engineers, not a replacement. Google still plans to hire more engineers, even as their AI writes 30% of the code, because their view is that augmented engineers can tackle more ambitious projects and a larger scope of work. They see AI freeing developers from grunt work so those developers can concentrate on creative design, big-picture thinking, and polishing the product, the things humans excel at. This is exactly the augmentation mindset. It’s about liberating humans from the mundane so they can spend more time on the meaningful and challenging parts of their jobs.
In my own experience working with teams that adopted AI tools, I’ve observed a similar pattern. When AI is implemented thoughtfully (with training for staff on how to use it, with guardrails to prevent obvious errors, and with metrics that focus on real outcomes rather than vanity stats), teams often become more productive and more satisfied. Developers report that they enjoy their work more when they don’t have to write boilerplate code all day. They get to focus on interesting problems and let the AI handle the dullness. Projects that were daunting or would have required more personnel become feasible with a smaller, AI-augmented team. But these successes came not from trying to eliminate the humans, but from empowering them. The AI is treated as an “amplifier” of human effort, not a wholesale substitute for it. This approach tends to yield steady gains without the downside of lost expertise and low morale that the replacement approach can bring.
Beyond the Hype and Fear: A Pragmatic Path
To move forward, the industry (and really, any industry adopting AI) needs to cut through the extreme narratives, both the utopian hype of “AI will do it all” and the doom-and-gloom of “AI will take all our jobs.” We need a pragmatic, honest mindset about what AI can and cannot do. Grandstanding claims like “all software engineers will be obsolete in five years” make for attention-grabbing headlines (or VC pitches), but they are not grounded in reality. This kind of talk, what I referred to as “VC theater,” is designed to create a sense of urgency and inevitability, perhaps to drive investments or adoption out of fear of missing out. It might spur some short-term action, but it’s the wrong foundation for long-term strategy because it’s divorced from the on-the-ground truth we’ve been discussing.
Instead, companies should focus on strategically integrating AI in ways that truly augment their workforce. This means identifying the right use cases where AI can add value, investing in the training and tools to help employees leverage AI effectively, and setting realistic expectations. It also means building a culture of human–AI collaboration: encouraging teams to treat the AI as a colleague (one that is fast but not infallible) and encouraging individuals to develop skills that complement AI (for example, prompt engineering, data-driven decision making, and cross-disciplinary thinking). Organizations that get this right will likely see better outcomes than those who either blindly chase full automation or shun AI entirely.
It’s also important to practice intellectual honesty about AI’s limitations. Businesses should be transparent about where AI is used and where it struggles. If an AI system has a known bias or failure mode, acknowledge it and manage it, don’t sweep it under the rug for the sake of appearances. By being honest, you build trust with your employees, customers, and stakeholders. Trust that is crucial for successfully deploying AI. For instance, if a bank knows its AI loan approval system tends to disadvantage a certain group unless carefully monitored, admitting that and keeping humans in the loop to mitigate it is far better than pretending the AI is flawlessly objective. In software teams, if everyone knows the AI coding assistant is great for certain tasks but prone to certain errors, they can use it effectively and remain vigilant, rather than being misled by inflated expectations. In short, moving beyond the hype means treating AI as what it truly is: a powerful tool that still has constraints, and not a mystical solution to every problem. Companies that adopt that mindset will likely navigate the AI era far better than those who ride the rollercoaster of hype and disappointment.
Conclusion: A Call for Ethical, Human-Centric AI Implementation
Having examined AI’s impacts and limitations, I want to re-emphasize that this is not a call to resist AI or slow its adoption. On the contrary, I am genuinely excited about AI’s potential and have seen firsthand the good it can do. The call, rather, is for a more thoughtful and ethical implementation of AI. We, as industry leaders, engineers, policymakers, and workers, have choices in how we integrate this technology into our organizations and society. The choices we make will reflect our values.
The distinction between using AI to augment humans versus replace them is perhaps the most pivotal choice. Augmentation respects the fact that people bring creativity, empathy, and contextual judgment that AI lacks, and it strives to elevate human work to a higher level by offloading the drudgery to machines. Replacement, in contrast, treats humans as costs to be minimized, assuming (often incorrectly) that the machine can do an equivalent job. One path leads to workers empowered by technology and potentially huge leaps in innovation (because humans + AI can tackle things neither could alone). The other path can lead to devaluing human skills, workforces in upheaval, and systems that might actually perform worse because they’ve lost the human touch in areas that needed it.
As we’ve seen, hype sells. It grabs headlines and can even inflate stock valuations in the short term. But when it comes to actually delivering value, reality asserts itself. In software engineering, the ultimate test is production code and real users, this domain quickly punishes wishful thinking. If an AI-generated code is buggy or insecure, the software will crash or be breached. There’s no hiding from that outcome. Likewise, in other fields, an AI decision will eventually meet the real world: the medical AI’s suggestion will face real patients and either help or harm, the finance AI’s trades will either make money or cause losses. The unforgiving nature of real-world outcomes means that building on shaky assumptions or ethical shortcuts is bound to backfire.
The evidence we have gathered paints a clear picture: the most valuable, complex, and mission-critical work still requires a human mind at the helm. Coding the tough bits of a system, designing an architecture, integrating with messy real-world systems, debugging the “unsolvable” issues, these remain human-intensive tasks. In other industries, the equivalent might be strategizing a business pivot, performing a tricky surgical procedure, or resolving a serious customer complaint. AI can assist with all of these, but it isn’t ready to autonomously handle them. And it may never be, if those tasks fundamentally require qualities of consciousness or creativity that aren’t easily replicated.
Therefore, the future of software engineering (and many knowledge industries) lies not in eliminating human expertise, but in elevating it. Imagine what we can achieve when every skilled professional is equipped with AI tools that make them superhuman in productivity at routine tasks, while they focus on the harder problems. That is a future where AI’s benefits are fully realized without discarding what humans do best. It’s a future where technology serves humanity, rather than displacing it.
To get there, we must be intentional and ethical in how we adopt AI. This means actively deciding, “We will use AI to help our people, not replace them.” It means measuring success not just in cost savings, but in quality, innovation, and human well-being. And it means being honest when the technology falls short and needs a human in the loop. By doing so, we can avoid the hidden costs and social pitfalls that come with the hype-driven approach. AI’s story in software engineering is still being written, and we have a say in how it unfolds. Let’s choose the path of augmentation, collaboration, and ethical implementation. By pairing the speed and scale of AI with the ingenuity and judgment of humans, we can build amazing things and do so in a way that benefits organizations and their people alike. Let’s build that future thoughtfully, ethically, and honestly.
