The Part of Software Engineering AI Cannot Replace

Ever since the AI boom started, I am constantly hearing about all the different ways AI is going to replace software engineers. From the programming to the planning part, it seems everyone thinks we are in the middle of another web revolution that will make entire roles obsolete and I do not think it is the right mindset. Will AI replace software engineers? No. But it will replace the part of the job that was never really engineering to begin with.

I will preface this blog post with my take: AI is not going to replace software engineers. But software engineers need to use a new tool and adapt their processes and most importantly: their thinking, to include it and use it. Once you take the load of writing code, what is left is all engineering: judgement, system thinking and consequence awareness. The answer to "What cannot AI replace?" is more specific than people think.

What AI Actually Replaced

Back in 2022 when I joined Okta, my role at the time was mainly focused on creating code samples and writing tutorials on the Auth0 blog. Building a code sample, even though it was a simple app, took me time. I still had to do the scaffolding, writing the API endpoints and put it all together, and let's not talk about creating a web application with a UI (taking into consideration that I do not like writing frontend code).

Today, I can create a code sample in minutes with my good friend Claude Code and nobody can argue with that. AI "took the weight" of writing code (in quotes because some of us actually enjoy this process but that is a discussion for another blog post). What AI did not replace was the creative process prior to writing code like deciding what to build. Understanding why something fails, evaluating trade-offs or even knowing when code is unsafe.

We have a new "bottleneck" now and it is our judgement as software engineers. These decisions are now more visible than ever and certainly not easier. In fact, one would argue that the situation is even more complicated now because sometimes code generated by LLMs can be a bit tricky and almost feel like a black box if you do not go through it thoroughly enough.

The Gap Between Generating and Understanding

It might sound like the title is a new gap, but is it really? I think this has always existed, but back when you had to write the code yourself you had to understand what was going on. Today? Probably not in terms of the code but definitely in terms of the product you are building. So the programming part might have gotten faster, but the understanding part did not.

Let's take junior engineers as an example, the reality is they can now generate a whole functioning app before they can understand the complexities of the system it lives in. Does this mean they are more productive? In the short term, I would say yes, but not in the long run.

This will eventually bring a comprehension gap because you cannot ship code you do not understand, but most importantly, you cannot engineer a system you do not understand. There is a slight difference, and I do not think this means junior engineers are doomed. It means the learning path changed. Instead of learning by writing code from scratch, they learn by reading, questioning and breaking generated code. The opportunity is there, but it requires willingness to understand what is happening under the hood and intentional curiosity.

In any engineering branch there is an engineering design process, which can be split into four different stages (although many authors have different takes on this, I found these to be the best for what I am trying to convey):

• Define the problem
• Conceptual design
• Detailed Design
• Learn and generalize

AI participates in all four stages (research, planning, building, analysis), but it does not own any of them. Let's look at what the engineer is still doing at each one.

Define the problem. We do research on the topic and most likely write down a series of requirements for a product. At this point there is still a lot of human thinking happening, you are the one deciding what to build because you are the one who understands the need that you want to solve.

Conceptual design. How is my product solving this problem? You can brainstorm with an agent, but you are still the one who understands how the solution (or solutions) are going to help with the problem you defined earlier.

Detailed design. Here is most likely where a lot of the burden of this process is taken from you and put into an agent or LLM, at least in this context. You can build a product, review it and iterate with your LLM until you have a solution you feel confident about.

Learn and generalize. This is where a lot of program analysis and maintenance comes into play, requiring both your understanding of the problem and the solution, as well as support from your favorite AI.

All this to say, nowadays the software engineer reads, evaluates and decides more than they type. This speed makes their judgement matter more.

Where AI Confidence Breaks Down: Security

This applies to any stage of the software engineering process: architecture decisions, performance trade-offs, data modeling, accessibility. But security is where the consequences are sharpest, so let's use it as an example.

When we think about security, the engineer's judgement plays a huge role. AI-generated code passes linting, passes tests, but could still have subtle vulnerabilities. Things like input validation gaps, race conditions, are things that we need to be aware of and understand to then make sure they are covered in the code because security bugs are invisible at the behavioral level. The code works, it just is not safe. Vibe coding (shipping AI-generated code without reviewing it) a security-critical path is more complex as AI does not have the same threat awareness as us, usually the attack surface is not in the happy path.

The Engineer as Auditor

So going back to the initial discussion: what is the new role of the engineer? What I am seeing is a shift from implementing to auditing. The reality is auditing is harder than implementing if you never learned to implement, and the cost of getting it wrong is not small.

In the context of security, you need to know what a vulnerability looks like before you can spot one in generated code. OWASP's Top 10 for Agentic Applications was written for engineers building agents, not necessarily for engineers using them. But if you are using a coding agent daily, understanding how it can fail helps you understand what it might get wrong in your code:

• Unexpected Code Execution: An agent can generate and run code you did not review, install packages you did not ask for, or execute shell commands you did not need.
• Tool Misuse: The agent operates within the privileges you gave it, but that does not mean it uses them wisely. It might pass untrusted output to a shell, or use write access where read-only would suffice. It will not question its own scope the way you would.
• Improper Output Handling: AI output that looks like valid code might still contain unsanitized inputs, hardcoded secrets, or logic that works in the happy path but breaks under adversarial conditions.

As engineers we are the last line before running any code in production. So now, as auditors we need to ask ourselves concrete questions such as:

• Does this tool need write access?
• Is this input validated before hitting a shell?
• Why is this package being pulled in?
• Would this still be safe if the prompt was manipulated? (when building AI applications)

Engineers who will thrive know which AI outputs to trust and which to interrogate, and when in doubt, apply the zero trust policy: never trust, always verify.

Where This Leaves the Engineer

The engineer's role is shifting from code writer to code auditor. Today the engineer's job will focus on decisions that carry bigger consequences, choosing a stack used to be a big discussion because it determined how long things would take to build. Now that an agent can scaffold anything, that pressure is gone, but the decision still matters for a different reason: you need to understand the stack well enough to audit what the agent writes. This means that even though the coding part got faster, the engineering did not get easier. The value is in the code you do not ship because you understood why it was wrong, and if you only ever learned to write code, you are competing with a tool.

Some will argue AI judgement is improving and will eventually close this gap too. Maybe. But right now, the model does not know your system, your users, or your threat model. You do. If you learned to engineer systems, you are the person who decides what the tool builds.