TL;DR: Going from idea to a Pull Request is a well understood problem today, and there are plenty of tools and avenues to do that. What really will make the difference in software development going forward is defining what to build, and then once built, verify and validate that it can provide the level of quality and service that our customers expect.

I witness to this day, many projects, products, and startups that are focusing on systems that generate code for you starting from requirements. It is an amazing endeavor, but it is so 2025. This is a solved problem today, or at least self-solving: we know how to do it.

If you build software that goes from idea to pull request, at this point you are not solving anything new. While there may still be a market, the intellectual space is commoditized.

Quick State of the Art for AI-based Software Development

For the purpose of this post, I will oversimplify a spectrum to its two extremes.

Today we are observing two big trends of how to do AI based development: vibe coding, and a more structured approach that I will call human-supervised agentic development (with some experiments of full agentic development that I will not discuss).

What is Vibe Coding?

The former, vibe coding, is about throwing vibes at AI and seeing what happens. You think about what your product should do, write it, and keep iterating on it until you are happy. Vibe coding is not engineering, but it is amazing for prototypes, POCs and MVPs, and frankly lots of fun. But, clearly, it is not the way to go for mature products.

What is Human-Supervised Agentic Development?

In human-supervised agentic development, we use a more structured approach:

• We plan.
• We define more clearly what to build (possibly with spec-driven development).
• We even give reference tests to compare the results with.

This gives us a much more predictable result, and it is a lot closer to proper engineering. Once the agent is finished, it can create a super-detailed and neat pull request for you, or a drier one, depending on your liking.

This is all magic, but let's say you have millions or billions of users, hundreds of functionalities and services. Let's say you have significant money riding on your product, would you let the agent merge and release autonomously?

No? Why? What is the problem?

The Problem with High Speed

So we are building this FTL (Faster Than Light) engine, and yet, putting it on our spaceship and letting it take us to the other side of the universe is scary. Why?

Well, first of all, we do not know if our ship can hold together at that speed, and we need to be able to test that. We need to validate that we are not breaking the ship by going faster than it can bear. All the systems need to stay functional.

Also, if you are going to activate your FTL engine, you better point your ship in the right direction, otherwise you are going to very quickly crash onto something, and very likely hurt yourself.

To translate this into agentic development terms, the first problem we have is to decide what to build and the second is validation of the code produced.

Focusing on Customer Value aka Choosing What to Build.

Let’s say we have the ability to build whatever we want, at infinite speed, and without any resource boundaries: what are we going to build?

Building just everything you have in mind might not be a good idea for a real product: nobody wants to buy The Homer car.

You could throw a thousand features out there, A/B test them, and then retire the ones which do not get traction. This would create a huge amount of churn in customer experience, and a lot more product surface to support for your teams.

How Do You Choose What to Build?

Well, honestly, a lot of the methods for prioritization stay the same. While we are no longer resource bound, we still need to think about what is best for our customers and for our business.

So here are four ideas:

1. Does it add value for our customers? This never changes, our customers are the most important focus point we should have. We should only add features which have a high ratio of customer value vs noise and complexity.
2. Is the objective clear? What problem are we solving? We need to be able to clearly articulate what problem this feature is solving within the context of our product.
3. Is this feature a good product fit : does this feature fit into the narrative of our product? Is it a natural extension of what we have already built or does it feel weird like a chatbot on a news site? While we think that building a product that does everything may be a winning strategy, actually we will be causing noise both to our customers' mental map and our go to market strategy.
4. Is this consistent with our financial strategy? We are adding product surface, but is this surface that customers will pay for, or that will bring in more revenue somehow? And if so, will it do it in a way that is compatible with our margins?

The Validation Challenge

Now, let's imagine we have decided what to build, and we are going to have an agent code it in zero time and create a pull request for us.

How do we know that we can merge that pull request and ship it to production safely?

What is the Meaning of Safely?

Well, first of all the meaning of “safely” varies from product to product, and from component to component. Ultimately, I would say that “safely” means maintaining integrity with your design principles.

A few external dimensions to consider that we may want to keep integrity on:

• Functional: Does everything still work well?
• Performance: Did we introduce any performance regression?
• Security: Was any new vulnerability introduced?
• Resilience: Are we still as resilient to failure as we were before?
• Compliance: Do we still meet all of our regulatory/policy requirements (for example, privacy, regulatory controls, and audit requirements)?
• Internal Quality: Is this good code (maintainable, readable, etc.)?
  We do not want to forget about all the internal qualities of code such as complexity, maintainability, etc.

So how are we going to attack the problem of validating the outputs of an automated software building machine?

The Tools We Have

Thankfully foundational engineering work still comes in handy with clear and deterministic methods. Plus, we have some new tricks up our sleeves thanks to the strength of LLMs.

The purpose of this post is not a dissertation on validation, so I am just going to leave a table here of tools and which dimensions they cover, then I will leave it to the reader to find more info on what the methods are.

A Risk-Based Approach to Software Validation

All the tooling we have can help us build a huge amount of confidence in generated code. Can it lead us to do automatic merges? And is it not expensive to have that full pipeline for validation?

It depends on which part of our software we are talking about.

In some areas, making mistakes may have big implications, for example, in highly secure parts of your code. Or parts of your code that may irreparably damage your data; on your documentation site, probably less so. What about SDKs, UIs, and Quickstarts? And what about demo applications?

Depending on where the value of our software comes from, we may have different levels of risk for different parts of our code, and you may want to apply different levels of validation.

Deep levels of validation are expensive, we should use them where there is the highest risk, and maybe still wait for a human to merge. Yet, where there is lower risk, we can maybe ease off, and go as far as doing automated merges for agents.

Time to shift the focus

So what is the point of this whole post? It is time to shift the focus!

The middle section of agentic based development is commoditized. How to automatically go from a well defined idea to a PR is well understood. We should let teams keep executing and solving it by inertia (if solving it internally), or buy one of the many available products out there.

What really will make the difference for the times to come is getting super crisp in defining what to deliver, and achieve safe high speeds thanks to proper validation frameworks. The effort needs to go into defining what to build, into saying no to what should not be built, and into making sure our validation and verification pipeline can guarantee the quality our product and customers expect. These are the new hard problems to solve.