We Let AI Write Code. Here's What Checks It.

An agent can write a working feature in an afternoon. It cannot tell you whether that feature is safe to ship. Those are different problems, and the second one does not get easier just because the first one got faster.

Most teams adopting AI-assisted development solve the first problem and quietly skip the second. The code compiles, the tests pass, the demo works. Review becomes “I read the diff and it looked fine” — which, on a 600-line agent-generated change, means nobody reviewed it.

At quickdev we run two automated review gates on every feature slice. Here is how they work and, more usefully, what we got wrong building them.

Asking an AI “Is This Okay?” Is Not a Review

The obvious move is to paste the diff back into the model and ask whether it looks secure. This produces confident, fluent, largely worthless output. The model will find something, because you asked it to find something. It has no consistent bar for what counts as a finding, no way to distinguish a real problem from a pattern that resembles one, and no memory of what it flagged last time.

A review is only useful if it is repeatable. Same code, same standard, same bar — every time. That means the checklist has to live outside the conversation, in a versioned file, benchmarked against a published standard rather than the model’s recollection of one.

So ours does. Two review gates, each with its own checklists: one for security, one for performance. Both run per feature slice and again before merge. Neither is optional, and neither depends on anyone remembering to ask.

Which Framework Applies, and When

The security gate does not apply one checklist. It applies a baseline to everything, then loads additional checklists based on what it detects in the code.

Always applied — OWASP Top 10:2025. Every finding maps back to a current category number and name. The 2025 revision matters here because two categories are new, and both land squarely on how software gets built now:

• A03 — Software Supply Chain Failures. New in 2025, and it has the highest incidence rate on the list. This is the category that catches an agent adding a dependency nobody vetted. The checklist requires verifying that every added package exists, is intended, is maintained, and has acceptable provenance — plus reviewing lockfile changes and install scripts. When a model hallucinates a plausible-sounding package name, this is what catches it.
• A10 — Mishandling of Exceptional Conditions. Also new. It covers what your code does when something fails: timeouts, partial failures, dependency outages, malformed input. The rule is fail closed for authentication, authorization, validation, and signature checks. A catch block that logs a failed permission lookup and continues with a permissive default is a critical finding under A10, and it is the kind of thing that reads as defensive, careful code right up until you trace what happens when the lookup fails.

Note also that SSRF no longer has its own slot in 2025 — it folded into A01, alongside the object- and function-level authorization failures that used to sit only in the API list.

Conditionally applied, based on detected surface:

If the code contains	The gate also applies
HTTP/RPC endpoints, GraphQL, gRPC, webhooks, WebSocket commands	OWASP API Security Top 10 (2023)
Browser JavaScript, SPA code, service workers	Client-side checklist
LLM calls, prompts, RAG, embeddings	OWASP GenAI/LLM Top 10
Tool-using agents, MCP/A2A, persistent memory, autonomous goals	OWASP Top 10 for Agentic Applications (2026)
Code identified as agent-generated	AI-assisted development overlay
.NET, Angular, or Node.js	The matching stack checklist

Surfaces are detected independently of language. An API written in Python still gets the API checklist even though we have no Python stack file — the surface is what determines the standard, not the syntax.

What That Looks Like on Real Code

Frameworks are only useful if they turn into specific findings. Some of the checks that earn their place most often:

API1 — Broken Object Level Authorization. An endpoint accepts GET /orders/{id}, looks the order up by id, and returns it. Authentication passed, so it feels safe. But nothing scoped the query to the caller’s tenant, so any authenticated user can read any order by guessing an identifier. The checklist requires authorizing every operation against its specific object, including nested and bulk objects, and testing guessed, swapped, and cross-tenant identifiers. This is consistently the single most valuable check we run.

API3 — Broken Object Property Level Authorization. A request body binds straight onto an entity, so a caller can set isAdmin or tenantId by adding a field the client was never meant to send. The fix is an explicit input DTO allowlist rather than model binding onto the domain object — and the mirror image on the way out, projecting responses explicitly instead of serialising whole entities and hoping nothing sensitive is on them.

A05 — Injection, extended to model output. The classic case is string-concatenated SQL. The version that catches teams out in 2026 is treating LLM output as trusted: a model returns a value, and that value reaches a query, a shell command, a template, or a URL without validation. The GenAI checklist is explicit that model output must be validated and contextually encoded before it hits any execution context — the model is an untrusted input source, not an authority.

Excessive agency and tool authorization. When an agent can call tools, the check is whether each tool call is authorized for the actual acting user, with least-privilege short-lived credentials, and whether irreversible actions require approval. The rule we lean on hardest: enforcement has to live outside the model. A system prompt instructing an agent not to delete production data is not a control. It is a request.

The AI-assisted overlay. When the code is agent-written, the gate looks for a specific failure pattern: placeholders and mocked authorization left in place, permissive defaults, sample credentials, disabled TLS verification, broad CORS — and comments that claim a control the code does not actually enforce. It also treats generated tests with suspicion, because tests that mirror the implementation confirm the code does what it does, not what it should. Adversarial tests for authorization and tenant isolation have to be written deliberately.

One rule in that overlay is worth stating plainly: AI authorship alone does not raise severity. Code is judged on evidence. The overlay exists because agent-written code fails in characteristic ways, not because it is presumed guilty.

Severity and Confidence Are Two Different Questions

This is the part that matters most, and it took us longest to get right.

Early versions produced findings that read as certain. “SQL injection in UserService.cs:42.” Sometimes correct. Sometimes the input was already validated upstream, in a file outside the reviewed scope. The finding was not wrong exactly — it was unqualified, and an unqualified finding is one a developer learns to ignore.

Every finding now carries two independent ratings:

Severity answers how bad is this if it is real — Critical, High, or Low.

Confidence answers how sure am I that it is real — Confirmed, Likely, or Needs verification.

These do not collapse into one number. A Critical / Needs-verification finding is not a weak finding. It is a potentially serious problem that nobody has checked yet, and treating it as low priority because the reviewer hedged is precisely the wrong response.

Static review genuinely cannot establish runtime behaviour. It cannot tell you the real cost of a query without knowing the row count, or whether an endpoint is exploitable without knowing what sits in front of it. A reviewer that pretends otherwise is not being helpful, it is being confident. Those are not the same thing.

”Needs Verification” Cannot Be Deferred

The natural failure mode of any severity system is that everything below Critical becomes “later.”

We wrote one rule into the definition of done to stop it: a finding marked Needs verification cannot be deferred. It has to be resolved — run the named check, then re-rate it as Confirmed or dismiss it.

The reasoning is simple. Deferring a low-severity finding is a decision: you know what it is and you have judged it minor. Deferring a Needs-verification finding is not a decision, it is an omission. You do not know what it is yet. Its real severity is still unknown, so you cannot possibly have judged it acceptable.

Both reviewers are required to name the specific check — enable EF Core query logging and count statements per request, take a heap snapshot across navigations, profile the change detection cycle. “Verify this” without naming the measurement just moves the ambiguity somewhere else.

The Checklists Needed Their Own Review

Here is the uncomfortable part, and the reason this post exists.

We recently audited our own checklists. They contained four factually wrong technical claims. One described a Fastify plugin behaviour that does not exist. One conflated a Debug build with an environment variable that has no effect on JIT compilation. One told the reviewer to flag a missing track in Angular’s @for — which cannot happen, because omitting it is a compile error. One prescribed an API name that had since stabilised under a different name.

None of these would have surfaced in a single review. They would have surfaced as a wrong finding in every review, indefinitely, with the full authority of a checklist behind them.

This is the actual risk in AI-assisted development, and it is not the one people worry about. The danger is not that the model writes a bad line of code — you will catch that. It is that a wrong assumption gets encoded into the thing that does your checking, and then quietly scales. Errors in tooling compound in a way errors in code do not.

So the checklists are versioned, reviewed by an engineer who knows the stack, and re-audited when the underlying framework moves. The reviewer is a tool. Tools need maintenance.

What Responsible AI Use Actually Means Here

“We use AI responsibly” is not a claim about tooling. Everyone has access to the same models.

It is a claim about process: what has to be true before code ships, whether that bar is enforced or merely encouraged, and whether you have any mechanism for noticing when your own standards have drifted out of date.

For us that means every slice passes two gates before it is done. Findings carry an honest confidence rating rather than false certainty. Unverified findings get verified rather than deferred. And the checklists themselves get audited, because a review process nobody reviews is just a source of confident, well-formatted mistakes.

The gates are one half of the process. The other half happens before any code exists — the documents, decisions, and open questions that give an agent something real to build against. That is how we take a project from PRD to first slice.

The speed is real. The discipline is what makes it usable.

Building with AI-assisted development and want a second opinion on your review process? Talk to the quickdev team.