Nothing checked that fix#

Your agent hands you a fix with 92% confidence.

Ask yourself what produced the 92. Not what it means. What produced it.

A model wrote that number about its own output. Nothing ran. Nothing parsed. No test executed. It is a language model's estimate of how a language model feels about a language model's suggestion. And it renders in the same font, the same shade of green, as a number that came out of a compiler.

That is the part I could not stop thinking about while building a debugging tool. Not that AI fixes are wrong. Most are fine. The problem is that the good ones and the bad ones arrive looking identical, so you have to read every one carefully, which is most of the time you were trying to save.

Stack Overflow's 2025 survey found 84% of developers using AI tools, 45% saying debugging AI-written code takes longer than expected, and 66% naming the top frustration as answers that are "almost right, but not quite".

Almost right is the expensive failure. Obviously wrong costs five seconds. Almost right costs twenty minutes, three files away, after you have already built on top of it.

A field instead of a feeling#

So DebugAI returns a verified field on every proposed fix, and it has three states, not two.

{
  "rank": 1,
  "title": "Add the missing import",
  "confidence": 88,
  "verified": true,
  "verification_reason": "ast.parse succeeded"
}

• true means a mechanical check ran and passed.
• false means a check ran and failed. The fix still comes back, with its confidence capped, because a failed check is information you want.
• null means nothing checked it. The confidence is the model's own estimate and should be read as exactly that.

The rule I hold is that null never renders as false, and never renders as nothing at all. "Not checked" and "checked and fine" are different claims. Collapsing them is how a tool starts lying without anyone deciding to lie.

Here is the part where I lose some of you.

We check two kinds of bug. Out of all of them.#

Version one of this covers exactly two classes.

Parse. SyntaxError and IndentationError. Python goes through ast.parse in-process. JavaScript goes to node --check in a temp directory under a subprocess with a 2 second CPU limit and a 2 second wall clock. This proves the fix's syntax is valid. It proves nothing about whether the fix is correct.

Import. ImportError and ModuleNotFoundError. This one resolves the named import against the files DebugAI already retrieved for the request.

And now the sentence that belongs in every tool like this and appears in almost none of them. The import check does not prove the package is installed. The engine has no access to your real dependency graph. A verified: true on an import check means "this import matches the context we retrieved", not "this import will work when you run it".

That limitation is written in the source file, above the function, so nobody maintaining it can pretend otherwise later.

Everything outside those two classes returns null. Your TypeError on line 42 gets a diagnosis, ranked fixes, exact edits, and a null verification, because nothing mechanical checked it. Type-level checking is planned. It is not built, so it does not claim to be.

Total budget for all of this is 5 seconds across every candidate fix in one request. A check that does not finish returns null, not false. Failing to finish is not a verdict.

Why ship something this narrow#

Because the alternative was shipping something broad that guesses, and a guess wearing a checkmark is worse than no checkmark.

A narrow honest signal is usable. You learn quickly that syntax and import fixes come back green, and that a hard concurrency bug comes back grey, and you calibrate. A broad dishonest signal teaches you nothing except to ignore the badge, which is where most confidence scores have already landed.

There is a second reason, less noble. Narrow is testable. Two classes with deterministic checkers can be verified in CI. "Our AI validates your fix" cannot.

The sandbox, and two things that did not work#

Running node --check on model-written code means running a subprocess on input you did not write. That gets bounded.

What bounds it today: RLIMIT_CPU at 2 seconds, plus a wall-clock timeout on the subprocess call, plus a temp directory it cannot see out of.

What we tried and reverted, on 2026-07-09, empirically rather than by assumption:

RLIMIT_AS (address space) made node --check hang or abort with a uv_thread_create assertion failure on a trivial file, at every value from 128MB to 2048MB. V8 reserves a large virtual address space at startup regardless of actual heap use, so a hard AS cap fights Node's own initialization rather than the candidate code.

RLIMIT_NPROC is a per-real-UID limit on Linux, not a per-subprocess one. Setting it there capped process and thread creation for the entire user account the engine runs as. At 16 it broke Node's worker thread startup outright, with the same abort. At any value it is the wrong instrument: it throttles the whole process, not the sandboxed child.

There is no syscall-level network block, which is a real gap and is written down as one. It is acceptable only because node --check parses without executing the candidate's code, so nothing on this path can make an outbound call. The note in the source says plainly that if a future check needs to actually execute candidate code, resource limits alone are not sufficient and it needs a network namespace or a dedicated low-privilege UID first.

I am telling you the limits of my own sandbox because you are about to send it your stack traces, and you should know what it does before you do.

Using it#

DebugAI runs as an MCP server, so any MCP client can call it.

npx -y @debugai/mcp setup

Browser sign-in, no key to copy. It writes config for every MCP client it finds (Claude Code, Claude Desktop, Cursor, Windsurf, Zed, Gemini CLI, Cline), backs up each file first, then checks the wiring actually works.

Look before it writes:

npx -y @debugai/mcp install --dry-run

Remove every trace:

npx -y @debugai/mcp uninstall

There is also a VS Code extension, which registers the same server, which is why the CLI skips VS Code by default rather than giving you every tool twice.

Free tier is 10 debugs a day, no card. Pro is $12/mo. A founding rate of $9/mo stays $9 for life and is claimable until September 1, 2026.

What I actually want you to take from this#

Not the product. The field.

If you build anything that hands a developer a suggestion, the useful question is not "how confident is the model". It is "what, if anything, checked this", and the honest answer is frequently "nothing". Say that. A grey "not checked" next to a fix is worth more than a green 92% that means nobody looked.

I would rather show you a null than dress a guess as a result.