What Tree-sitter taught me about reading code I didn't write

AIZEN, my capstone, has one job: look at a codebase it has never seen and say where it is likely to break under attack. The first version did what most tools do — matched patterns against source text. It found plenty. Almost none of it was real.

Text search doesn't know what a function is

A regex for a dangerous call cannot tell you whether the argument is user-controlled, whether the call sits behind an auth guard, or whether the whole file is dead code nobody imports. It sees characters. Reviewers do not read characters; they read structure, and then they read intent.

Tree-sitter gave me the structure part cheaply. It parses incrementally, it is per-language, and it hands back a concrete syntax tree I can walk instead of a pile of match offsets. Suddenly the question stopped being “does this string appear” and became “does a tainted value reach this parameter”.

The query that changed the tool

(call_expression
  function: (identifier) @fn
  arguments: (arguments (identifier) @arg)
  (#match? @fn "^(exec|spawn)$"))

Not clever. But paired with a taint pass over the tree, the false-positive rate fell far enough that the findings became worth reading — which is the only metric that matters for a security tool nobody is paid to use.

A finding a developer ignores is worse than no finding at all. It teaches them to ignore the next one.

Then: prove it

Structure narrows the field; it does not confirm anything. So AIZEN takes each surviving candidate into a sandboxed Docker container and attempts the exploit against a disposable copy of the app. If nothing happens, the finding is downgraded, not shipped. Confirmed ones arrive with the transcript attached.

That verification stage is slow, awkward and by far the most useful thing in the project. It is also why the LangGraph orchestration exists — something has to decide when a candidate is worth a container.

What stuck

• I read diffs structurally now: what calls this, what reaches it, what guards it.

• I distrust any tool that reports without demonstrating.

• Parsers are boring infrastructure and boring infrastructure is where the leverage is.

None of this is novel research. It is just the difference between a tool that produces output and a tool someone will actually act on.