Blog
Who checks the AI builder? Independent QA for AI-built software
· 5 min read
Your supplier shows you a new application. It was built in a week, it looks tidy, and the builder says: "Tested, everything works." Two weeks after go-live it turns out that an invoice amount lands on the wrong day around midnight, and that a colleague can see more than intended.
That is not necessarily sloppiness. It is what happens when the same party builds and assesses. With software made using AI this is even more pronounced, because the builder works incredibly fast and approves its own work with just as much conviction.
The builder that marks its own homework
An AI tool that writes code is also very good at writing a reassuring conclusion. "The tests pass" and "the feature works" come easily out of such a tool, even when nothing has actually been tried. We have seen this often enough in our own build work. Our rule of thumb: never ask for a confirmation, ask for a measured outcome. Not "does it work?", but "show me what you see on this screen when you log in as a planner".
Research shows that this is more than a gut feeling. Veracode has AI models carry out coding tasks and checks whether the result is secure. Their 2026 GenAI Code Security Report states that the average security pass rate across models is 56%, barely changed from 55% in the first report. It also states that around 44% of the coding tasks in the tests produced a risky vulnerability. So the models are getting better at code that runs, but not automatically better at code that is secure.
DORA's research into AI in software teams points the same way. They report that 30% of developers have little or no trust in AI-generated code. The same page states that higher AI adoption goes hand in hand with more throughput and more instability in software delivery. Building faster therefore does not automatically mean fewer errors. The time you gain while writing can disappear again during checking. If nobody checks, it disappears at the first outage.
What independent means here
Independent QA is not a grand term reserved for large organisations. It is one simple requirement: the person checking has no stake in the answer "it is fine" and did not make the work themselves. That can be a colleague who did not do the build. It can be a second AI model that reads along critically with a different brief. It can be the client, provided they have a concrete checklist. Our lesson from our own practice: let someone else reject the result before it is called "done". A second pair of eyes catches what the builder no longer sees.
Automated checks certainly belong in this, but they do not replace it. OutSystems, for example, has an AI Mentor System that, according to the product page, automatically runs extensive code reviews every twelve hours on maintainability, security and performance. That is useful: it helps to find unused code and other problematic patterns. But such a review looks at the quality of the build. Whether your invoice rule is correct, whether your planner sees the right things and whether your customer receives a correct certificate is a different question. For that you need someone who knows the process and holds it up against the business rules.
Where it goes wrong in practice
The same kinds of errors keep coming back in our build practice. They are not spectacular errors, but exactly the kind that a builder checking its own work skips over:
- Time zones. In some systems "today" is calculated in a different time zone than the Netherlands. Around midnight you are then a day off.
- Order. A list that should show the cheapest first is right in the mockup and wrong in the app, because the sorting is not recorded anywhere in the data.
- Roles. A screen without a clear role becomes a forgotten loose end. Setting up permissions afterwards is exactly the kind of work that gets dropped.
- Misclicks. A delete button without a confirmation step works until someone presses it during a demo.
- Staleness. Demo data tied to "today" is empty a week later, right in the middle of a customer meeting.
- Piled-up warnings. Build and validation warnings that keep accumulating make a new, real error invisible.
A test report saying "all tests green" tells you nothing about any of these. Green tests prove that the function does what the test asks, not that the person working with it gets what they need. That is why we test the path a person takes: log in with a real role, carry out the task, look at what is on the screen.
On a platform like the one for MEA Beveiliging, where, according to our projects page, dealers manage their own installations, SEPA flows run automatically and certificates are generated instantly, an error in the logic is not a cosmetic problem. The more an application does on its own, the more an independent check pays off. That applies to apps built with AI assistance as well as to apps in which AI agents take part.
What does this mean for your business?
You do not need to set up a testing department. You do need to agree who rejects the work before it goes live.
- Separate building from assessing. Ask your supplier who checks the delivery if it is not the builder. If the answer is "the builder", ask for a second pair of eyes: a colleague, another model or yourself.
- Write the checklist in advance. The goal in one sentence, what is never shared with production, and whether testing uses synthetic or real data. Add the roles that need to be able to log in and what each of them may and may not see. That prevents arguments afterwards.
- Ask for measured outcomes. Do not accept "it works", but a report: what was done, with which role, what was the result. A screenshot or a concrete example run is better than a confirmation.
- Test the real path, including around midnight. Walk through the process the way your employee does, including the edge cases: date boundaries, an employee without permissions, a misclick.
Would you like to know how we approach this, or would you like an independent look at a delivery? Also read our post on having a mockup assessed first and what keeps you in control when building with AI assistance in OutSystems. Or get in touch with us for a no-obligation conversation.