Atlas: keeping AI code under control

In my post about systems thinking, I wrote about a problem with AI coding: producing code faster doesn’t make that code easier to maintain. It can actually make the problem worse. Every new function, dependency and abstraction becomes something that the next agent, or the next human, has to understand.

And the agent will happily keep adding things. One more parameter, one more branch, one more helper that knows a little bit too much about the rest of the system. Each change looks reasonable on its own. After a while, making a small change requires reading half the project.

That’s what I mean by entropy in a codebase. Not a formal measurement, but the gradual loss of structure that makes the next change harder than the previous one.

I’ve been working on two capabilities in Atlas to help with this: check, to put limits on how the code grows, and mutation manifests, to question whether the tests would notice if the code were wrong. They address different problems, and I think the combination is useful when most of the typing is done by AI.

Instructions are not enough

I can put “write small functions” or “avoid coupling” in an AGENTS.md file. And I do. But those instructions leave a lot of room for interpretation. What does small mean? When does a function become too complex? Does the agent remember the instruction after spending twenty minutes fixing something else?

I don’t want to keep repeating the same feedback in every review. If a rule can be measured, I would rather give the agent a tool that measures it.

Atlas parses the source with Tree-sitter and builds a graph of files, functions, types and their relationships. From that structure, it calculates measurements such as line counts, declared parameters, complexity, nesting and dependency impact. atlas report shows those measurements. atlas check compares them with configured limits and fails when they are exceeded.

atlas report -f .
atlas check -f .

By default, check allows up to 600 lines per file, 100 lines per function or method, three parameters, weighted complexity of eight, and decision depth of three. It allows no nested loops and no runtime circular import groups. These are strict defaults, especially the nested-loop rule, and they are configurable in the project’s root atlas.json. Setting a limit to null disables it.

The numbers are not a definition of good software. A function with 99 lines isn’t automatically better than one with 101. What I want is a point where the agent has to stop and reconsider the design instead of continuing to add code in the same place.

Less code to hold in your head

Small files matter for humans, but they also matter for models. An agent often reads a file to understand one behavior. If that behavior lives alongside a thousand lines of unrelated code, all of that competes for context. A focused file gives the agent less irrelevant material to process and gives me something I can review without losing track of what changed.

Nevertheless, splitting a large file into ten files is not necessarily an improvement. If all ten depend on each other, I still need to understand all of them to change one thing. Now I also need to jump between files.

This is why size limits alone aren’t enough. The dependency graph matters too. Atlas reports fan-in, fan-out and transitive change impact: how many internal peers depend on something, how many it depends on, and how widely a change could spread through dependents. Optional maxChangeImpact and maxInstability checks can put limits on some of these measurements. Both are disabled by default. These are static relationships, not proof of what will break at runtime.

The parameter limit is another useful source of friction. If a function needs seven separate inputs, maybe it is doing too much. Maybe it receives details it shouldn’t need to know. But wrapping those inputs in a generic options object just to pass the check doesn’t solve anything. The interface still needs to express a clear responsibility.

The same applies to complexity and nesting. Moving every branch into a helper might make the numbers smaller while making the behavior harder to follow. The point is to simplify the work, not hide it from the measurement.

So check doesn’t enforce an architecture or prove that interfaces are well designed. It enforces constraints that push me and the agent to ask those questions earlier. Fewer circular dependencies, smaller responsibilities and clearer interfaces are the direction I want. Reviewing whether the refactor actually moves in that direction is still part of the job.

I also run it against the whole project, not just the changed files. A new import can create a cycle involving code that wasn’t edited. Looking only at the diff misses that.

Making the feedback repeatable

One thing I like about this setup is that the agent and CI can use the same command. atlas check exits with zero when checks pass and one when it finds violations. Findings identify the rule, the measured value, the limit and the source location. The agent can read them, fix the relevant problem and run the check again.

Analysis happens locally, without a database by default. I don’t need to start the dashboard or maintain an index just to check current source. Incomplete analysis also produces findings; a missing measurement isn’t quietly treated as zero.

There are limits, of course. Atlas uses static analysis, not a compiler’s understanding of every possible runtime relationship. An unresolved import cannot establish a cycle. A passing check means the supported analysis found no configured violations, not that the architecture is perfect.

Still, this gives the agent a concrete feedback loop. The instruction stops being only “please keep this maintainable.” There is now something it can run before handing the work back to me.

But do the tests actually check anything?

A well-organized implementation can still be wrong. And a green test suite can still be missing the assertion that matters.

Imagine this eligibility rule:

export function isEligible(age: number): boolean {
  return age < 18;
}

Tests for ages 12 and 20 exercise both outcomes. But change < to <= and those tests still pass. The behavior at 18 is wrong, and neither test notices. Even branch coverage can look fine.

Mutation testing asks that question directly: if I deliberately introduce a small bug, do the tests catch it?

In this example, the mutation changes < to <=. A test asserting that isEligible(18) is false should fail. If the tests still pass, I have a concrete reason to inspect the boundary assertions, instead of asking the agent to vaguely “improve test coverage.”

Not every mutation reveals a missing test. Some edits don’t change observable behavior in the relevant context; others are invalid. But a plausible bug that survives is worth investigating.

What Atlas does, and what the agent does

There is an important distinction here: Atlas does not run mutation tests itself.

atlas mutations produces deterministic JSON manifests. Each manifest describes one exact operator replacement against the original source. The catalog includes changes such as + to -, < to <=, equality to inequality, and logical AND to OR, with explicit language mappings. It is not arbitrary AI-generated code rewriting.

Rules are opt-in. For the boundary example, the project configuration could include:

{
  "mutations": {
    "rules": ["comparison.less-to-less-equal"]
  }
}

Then:

atlas mutations --rules
atlas mutations -f .

The first command lists the rules. The second produces exact edit proposals without changing the source. Atlas checks that each proposal matches the current file, but it doesn’t run tests.

For that, I use the mutation-testing skill in Matato Pi. The agent picks a few edits relevant to the change and works in an isolated copy. First it confirms the tests pass. Then it applies one edit, runs the same tests, records what happened, and restores the original code before trying the next one.

My working files stay untouched. The copy isn’t a security sandbox, so the test commands still need to be safe to run. The result is a report with evidence, not just the agent saying the tests are good.

Why use AI here? I want the agent to read the change, choose plausible bugs and use the project’s existing test setup, instead of building another runner into Atlas. For an age-threshold change, it should test the boundary, not unrelated arithmetic nearby. Atlas generates the edits deterministically, but the agent’s selection and interpretation aren’t deterministic. It can miss things or misread failures, so I want evidence: an assertion failing because of the edit, not just a failed build or a timeout. If the tests still pass, that’s a possible gap to investigate. And a few selected experiments don’t prove the whole test suite is good.

What I get from both

I see check and mutation testing as two different kinds of feedback. Check puts repeatable constraints on the structure we keep adding to. Mutation testing questions whether the tests can distinguish the intended behavior from a plausible mistake.

Neither replaces requirements, code review or integration tests. Small functions can implement the wrong requirement. Tests can reliably reject mutations while asserting the wrong business rule. I still need to understand what the system is supposed to do.

But this is much more useful to me than an agent saying “I added tests and everything passes.” I can see which structural limits were checked, which deliberate bugs were tried, what failed, and what wasn’t tested.

I still have to judge the result. Now I have specific evidence to review.