Reducing AI Slop

A growing codebase can make even a small change difficult, despite the tests, code reviews, and QA you put in place.

You start building with AI, and features come together quickly. You add unit tests, review the code, and run QA before release. You have a development loop and a pipeline to check the work before it ships.

As you add features, the codebase grows. Changes that used to be quick take longer. Your agent writes the patch, gets feedback in review, and makes another pass. You review it again. The tests pass, but getting the change accepted takes more work than you expected.

On a project I was building from scratch with AI, I was having the same problems. I had good coverage and more bugs than I thought there should be. Code reviews were taking longer too. A single change sometimes went through around 20 rounds of review before it was accepted. I felt I was doing everything right. That wasn’t normal.

I was also reading accounts from other developers describing similar experiences: fast progress at first, then agents having a harder time working in the codebase they had helped build. People were describing it as code rot. I wanted to understand what was happening in my own repo and where a refactor would make a difference.

Follow the changes

My first step was to look through the code for obvious issues. That’s hard to do across a growing repository without first knowing where to look. I could ask an agent to inspect everything, but I wanted a reason to choose one part of the code over another.

The Git history gave me a starting point. I looked at the files with the most changes. If fixes, patches, and new features kept bringing me back to the same files, there was a reason to inspect them more closely.

Using the Git history, I asked my agent to repeat my search across the codebase and rank the hotspots by number of changes. That gave us a list we could work through instead of starting another open-ended cleanup.

A frequently changed file might simply contain the part of the product under active development. The change count gave me a place to look. I still needed to decide what to look for and what an improvement would mean.

What to focus on first?

To give the review a consistent basis, I compared metrics and tools. I wanted to identify code that was difficult to change, choose a bounded refactor, and check the result.

I chose Change Risk Anti-Patterns, or CRAP, introduced by Alberto Savoia and Bob Evans in 2007. It combines cyclomatic complexity with automated test coverage to flag functions worth examining.

Cyclomatic complexity counts independent paths through a function. A syntax-based count starts at one and adds decisions for branches, loops, catches, non-default switch cases, ternaries, and supported short-circuit expressions. More decisions mean more paths to consider when you change the function.

Coverage supplies the second input: how much of the code the tests execute. In the formula below, coverage is a fraction between zero and one.

CRAP = complexity + complexity² × (1 − coverage)³

For a function with complexity 10, the formula gives these scores:

The original warning threshold was above 30. Full coverage removes the coverage penalty; the underlying complexity remains.

That gave me two concrete places to investigate: decisions the tests never reach, and functions with enough branching to make a small edit difficult to follow. Deep nesting adds another burden. To understand a statement inside several conditions, you have to keep the enclosing conditions in mind too. Nesting needs its own check because the CRAP formula doesn’t measure it separately.

I also wanted checks for patterns a low CRAP score could miss. A short function can do unnecessary work or hide a type mismatch. For these checks, I used Oxlint, a linter that flags code matching configured rules. The anti-slop rules from dmmulroy’s project included checks for patterns like these:

• Copying a growing accumulator. In items.reduce((acc, item) => [...acc, item], []), each iteration copies everything collected so far. Across n items, the copies add up to quadratic work. The lint policy checks accumulating spreads and other accumulator copies, including Object.assign({}, acc, ...).
• Chained type assertions. value as unknown as User asks TypeScript to accept a type without establishing that the value has that shape. The assertion-chain rule flags the shortcut so the review can preserve the original type or validate external input.
• Replacing entire modules in tests. The module-mocking rule rejects calls such as vi.mock and jest.mock. I chose to replace dependencies through real interfaces so the boundary being tested stays visible in the code.

My plan was to combine CRAP with those anti-pattern rules in a tool my coding agent could use. The score would help it choose which functions to inspect, and the failed rules would give it specific patterns to fix. I could then compare the results after a cleanup.

Building the audit tool

In Cleaning Up the CRAP, I proposed giving a second agent the cleanup job after implementation and verification.

To make that review repeatable, I built CRAP Score MCP. It’s a local Model Context Protocol server that coding agents can query from their development tools. I included uncommitted work in its starting snapshot so the comparison reflects the code the agent actually received.

To measure each function, I needed to identify its boundaries and the decisions inside it. I used Tree-sitter here. It’s a parser built around a grammar, a set of rules for recognizing the structure of code. It uses those rules to turn a source file into a syntax tree. The tree represents the structure of the code: functions contain statements, conditions contain branches, and those branches can contain further conditions.

I then built the scanner to walk that tree and locate functions, methods, and closures. For each one, it records the name and line range, counts decisions for complexity, and tracks the deepest nesting. A function declared inside another function gets its own measurements. Its decisions belong to that inner function’s count. The language pack defines which syntax nodes represent functions, decisions, and nesting for that language.

I had my local agent (Codex) run the project checker using my configured linting rules and test command. The tests produced a coverage report, and the recorder saved the measured lines with a fingerprint of the source and test inputs. When a test failed, I had Codex inspect the failure, fix the code, and rerun the checker. I used the passing run’s coverage results for the audit.

Those function boundaries let me match the coverage lines to each function. That gives each function a count of measured lines, the lines the tests executed, and the lines they missed. Combined with its complexity, those counts produce the CRAP score.

In the report, I connected lint findings to the functions they apply to. When you inspect a function, you see the anti-pattern rules it fails alongside its complexity and coverage, giving the agent specific patterns to fix.

The first version took a long time and bogged down my computer. I wanted to use it throughout development, so I needed to reuse work between reviews.

The service could return the latest audit results when the input fingerprint, recorded test run, and Git commit still matched. I also cached the parsed functions for each version of a file. During a refresh, the scanner reuses the parsed structure of files already in the cache and parses the remaining files. The scan yields between files so the service can handle other work during a background refresh.

While Codex worked on one change, I kept the starting snapshot fixed. The review compared the latest results with that baseline and picked out functions whose code or measured score had changed. That let me inspect what the cleanup improved and which findings remained. After those changes, I no longer noticed the audit slowing down my normal code-change cycle.

Read the audit

The audit can order findings by CRAP score, file changes over the past 90 days, or project-defined critical paths. This request follows the same approach I took with the Git history:

scan_repository_hotspots({
  project: "<checkout-id>",
  priority: "churn",
  limit: 10
})

The change count belongs to the file, so it prioritizes findings in frequently edited files. For a candidate function, analyze_function_health returns its location, complexity, nesting, coverage, and uncovered lines.

The function report also shows the code-pattern rules it fails. In the documentation’s illustrative example, Oxlint flags typescript(no-floating-promises) at notes.ts:4, inside submitNote. The function calls saveNote(text) and returns before the save finishes. The finding identifies the function and source line, explains the risk of an unhandled rejection, and suggests a fix and a test.

If the caller needs the save result, the function can return the promise. Have the test make saveNote reject and check that the caller receives the error. Rerunning the language check then shows whether the rule finding has cleared. The report gives the agent a specific statement to change and a behavior to verify, even when the function has no coverage score.

To show a complete review, I included a small HTTP-status example. describeStatus translates ten status codes into labels through ten if statements. Its only test checks that 200 returns "ok". That call returns at the first branch and covers 5 of the function’s 43 measured lines.

The server counts a complexity of 11 from the parsed code. Combined with that coverage, the function scores 94.51, above the default review threshold of 30. This implementation uses measured line coverage; the original CRAP proposal specified basis-path coverage. Line coverage tells us which lines ran, and the assertions tell us what the test checked.

The uncovered lines tell me what to inspect first. Two adjacent branches return "unauthorized" for 401 and "forbidden" for 403. Swapping those strings would leave the complexity, coverage, and passing test exactly where they were.

The next test checks all ten labels and the fallback for an unrecognized code. Every measured line runs. CRAP drops to 11, the complexity of the original function, before a single branch has been removed. The server clears the CRAP finding and keeps the complexity finding open.

In the proof script, I deliberately gave 401 the label for 403. This time an assertion fails. The edit also makes the old coverage report stale. Ask for the score now and the server returns null. I made the recorder require a passing run and a fresh report for the current source and tests.

With those cases covered, the ten decisions are easy to inspect side by side. Each compares the same input with a constant and returns a label. A map holds those pairs directly, leaving a lookup and a fallback.

export function describeStatus(code: number): string {
  return descriptions.get(code) ?? "unknown";
}

Adding another status code now takes one entry in the map and a corresponding test case. The same assertions pass against the map. The complete example records three stages:

A call to review_changes with the original snapshot shows both findings resolved.

The report gives you the score change. The deliberately wrong label gives you evidence that the tests catch that mistake. With Node.js 24.12 or newer, you can inspect both implementations and run the proof yourself with npm ci and npm run prove from the repository root.

Build the review into the agent

I built two agent skills to give the agent a consistent way to use the MCP server. crap-development-review handles a feature, fix, or refactor already in progress. It registers the checkout, reads recent check results, and captures the starting snapshot before editing. The agent inspects the affected functions and code-pattern findings, makes a bounded change, and reruns the checks after each attempt.

For the maintenance backlog, I built crap-backlog-triage. It reads the repository’s findings and recent check results, inspects a candidate, and carries its snapshot into development review. That gives a weekly scan a path from a ranked backlog to one change the agent can verify.

At the end of the cycle, I had the development-review skill call review_changes with the original snapshot and scorecard: true. The agent returns the PNG scorecard and a link to the detailed HTML report in the chat.

QA replays check a fixed scenario against expected results. Benchmarks compare how long the same workload takes before and after a change.

The card below is from a sample HTTP-status cleanup included in the MCP’s own repository.

I wanted the review to finish where I was already working with the agent. The chat now carries the result of the change, and the saved report lets me open the findings and inspect the code behind it.

Dogfooding

To test it out, I ran the MCP’s own code through its audit tool. The audit flagged compareChecks, which compares check runs after a patch. The cleanup separated workload matching from report formatting. A regression test checked that a changed benchmark workload was reported as having no matching baseline.

The saved report contains these comparable function scores:

For compareChecks, complexity also fell from 16 to 7. The report marked the overall session Unrated because one file was unassessed.

Both revisions matched a replay of 48 TypeScript functions. Repeated benchmark runs left the timing difference unresolved. I recorded the results and the next step: match workload versions and reproduce the timing change before adjusting a budget. The record_project_learning tool keeps that evidence and instruction in .crap-team.json for the next agent.

Try it on your codebase

TypeScript, TSX, and Swift support ships with the MCP. Python, Rust, and Go are available through language packs you can enable in project settings. You can add language packs for other Tree-sitter grammars and extend the checks with your own linting rules.

The macOS installer connects detected Codex, Claude Code, and OpenCode clients to one local service and installs the review skills. Numeric scores require your project’s coverage command to be configured and its passing suite recorded.

During a feature or fix, ask your agent to use crap-development-review on the current change. In Codex, the request can be as short as “Use $crap-development-review to review my patch.”

For an existing backlog, use crap-backlog-triage and ask it to prioritize churn. A weekly review can use the integration view to inspect merged code.

I started looking for files that kept changing. Now I can see exactly where to look, and my coding agent has a new goal: measurable improvements with scores to beat. My code is better for it.

Fixing weak tests and type assertions that hide a mismatch is part of finishing the change. I expect the agent to show what improved and verify that the behavior I asked for still works.

I’m done accepting AI slop as the cost of building with AI and can now get back to building new features instead of fixing bugs.

The CRAP Score MCP repository includes the code, setup instructions, and review skills.