claude-review-all

/review-all

Project-agnostic code review for Claude Code. One slash command runs deterministic gates, ten parallel review agents, and an adversarial verification pass. Every finding cites file:line and is independently re-checked before the report — false positives stay out.

Demo

One verdict line up top, automated gate results, then findings by severity — Critical/Important carry full anatomy (impact · suggested fix · evidence), Debt/Suggested collapse to one line each. Every finding cites file:line and is independently re-verified before it reaches the report.

Severity tiers

• 🔴 CRITICAL — Breaks functionality, exposes data, crashes systems, violates requirements
• 🟠 IMPORTANT — Missing error handling, unhandled edge cases, potential bugs
• 🟡 DEBT — Code duplication, convention violations, refactoring needed within 6 months
• 🔵 SUGGESTED — Measurable improvements only. If you can't measure the improvement, don't suggest it.
• ⚪ QUESTION — Items requiring human judgment about requirements or intent

Install

Plugin (recommended)

Inside Claude Code, add the marketplace and install the plugin:

/plugin marketplace add ncoevoet/claude-review-all
/plugin install review-all@ncoevoet-review-all

/review-all is available right away. Update later with /plugin update review-all@ncoevoet-review-all, remove with /plugin uninstall review-all@ncoevoet-review-all. CLI equivalents work too: claude plugin marketplace add ncoevoet/claude-review-all then claude plugin install review-all@ncoevoet-review-all. The plugin bundles the skill's scripts/ and resolves them relative to the skill, so it works wherever Claude Code installs it.

Manual (make install)

For hacking on the skill itself, copy it straight into ~/.claude/skills/:

git clone https://github.com/ncoevoet/claude-review-all.git
cd claude-review-all
make install   # copies skills/review-all/ → ~/.claude/skills/review-all/

make uninstall removes it. make review-self installs then reminds you to run /review-all in this repo. The skill works in Claude Code only — it depends on filesystem access and bash.

Use

Inside Claude Code, run /review-all with any of these targets:

Examples:

/review-all
/review-all --staged
/review-all PR #123
/review-all last 3 commits
/review-all vs main
/review-all src/auth/login.ts src/auth/session.ts
/review-all PR #42 --exclude apps/legacy
/review-all gate --severity critical

How it works — exact steps

Phase 0 — Project discovery

The rules cache is keyed on a manifest of per-file content hashes over every repo CLAUDE.md (+ root CLAUDE.local.md), carries a schema version, and expires after 7 days — a branch switch, a CLAUDE.md edit, or a legacy cache file all force a fresh extraction. The report's gate table shows Profile cache: HIT / MISS(reason) so cache behavior is always visible (and eval-gradeable).

Phase 1 — Deterministic gates (in parallel)

Gate-confirmed findings are tagged VERIFIED and skip the verification phase. They are real, by definition.

When a gate fails, its output is also handed to every agent and to the verifier as a <gate_results> block (failures only, truncated) — a compiler or test runner is ground truth no amount of reading matches, which is why hybrid static-analysis + LLM review outperforms either alone. Agents treat it as a lead to the underlying defect, never as a finding to restate: the failure is already on the report.

Phase 1.5 — Runtime probe (optional, self-skipping)

If a UI file changed AND a dev-server port is open AND curl exists:

1. Health-check each port (3× with 2s backoff to absorb dev-server warm-up).
2. If Playwright/Puppeteer is installed and a baseline screenshot exists, headless-screenshot the changed routes and pixel-diff against baseline.

Catches dead routes and visual regressions that static review cannot.

Phase 2 — Parallel agents

Ten specialized agents review the (filtered) diff slice in parallel, each on its own concern:

standards · bugs+security · DRY · consistency · simplification · security-deep-dive · performance · test-quality · API-contract · a11y/i18n

Agents share _shared.md (severity tiers, 3-question gate, quotas, auto-drop rules, codegraph-tool resolution).

Each agent receives the same diff in a different file order (scripts/agent-order.py, a reproducible sha256-derived permutation per agent). Attention isn't uniform across a long prompt, so identical ordering would give all ten agents the same weak middle; permuting decorrelates that at zero token cost. Hunks within a file are never reordered, and chunk membership is computed on the canonical order first, so it stays identical across agents.

Big diffs are auto-chunked (chunkMaxFiles=40, chunkMaxBytes=200000) and re-merged by root_cause_key.

Phase 2.5 — Dedupe → adversarial verify

1. Dedupe via scripts/dedupe.py: groups by root_cause_key, annotates confirmed_by and corroborating_agents (how many personas independently reached this root cause — the verifier uses it to prioritize re-read effort and break borderline ties, never as a substitute for verification, and the report surfaces it as "Flagged independently by N agents"), applies global caps (SUGGESTED ≤ 10, QUESTION ≤ 8).
2. Verify in parallel — one verifier per source agent, spawned at verifierModel tier (default Haiku — cheap, fast, JSON-bound). Verifier stance is hostile to the finding, not the code: assume every finding is wrong until disproven. Its primary gate is a citation check — a behavior claim must be provable from a quoted source line, not inferred from naming; ungrounded claims are dropped (or kept only as a ⚪ question). Top severity (🔴/🟠) must be earned by that proof.
3. Score: ≥75 → main report, 50–74 → appendix, <50 → silently dropped.
4. Claim class — every finding is classified static / runtime / data / rendering, and must hold the proof its class demands. Reading a template proves what the template says; it proves nothing about what the server returned or what the user saw. A runtime/data/rendering claim backed only by a source read gets the unverified verdict — orthogonal to the score, so even a well-argued one lands there. It is neither asserted as fact nor dropped: it surfaces in the report's 🔬 section naming the exact observation that would settle it, and it never blocks gate mode.
5. State sweep via scripts/state-sweep.py: applies fixed/stale/snoozed/wontfix transitions to .claude/review-all/state.json.

Phase 2.75 — Completion gate

Every spawned agent and every verifier must have returned with valid JSON, or be explicitly retried once, or be surfaced as ⚠️ PARTIAL REVIEW in the report. No silent drops.

Phase 3 — Unified report

Opens with a one-line Verdict (N must-fix before merge, or ✅ none) for instant triage, then: Intent · Summary · Gate Results · 🔴 Critical · 🟠 Important · 🟡 Debt · 🔵 Suggested · ⚪ Questions · 🔬 Unverified · Dependency Changes · Appendix · Scope footer (files reviewed / skipped). The Gate Results table carries a mandatory Provenance column — the exact command, its exit code, and when it ran — because a gate with no provenance may not be rendered PASS; that is what separates a build that happened from one that was assumed. 🔴/🟠 get full anatomy (failure-mode title, [severity · confidence] tag, one-sentence impact, suggested fix, ≤8-line evidence); 🟡/🔵/⚪ collapse to one line each. The Summary also reports a Merge-readiness % (a transparent resolved/total must-fix ratio that climbs as fixes apply) and change-type buckets (files Added/Modified/Deleted/Renamed). The last line is a machine-readable <!-- review-all-severity: {…} --> comment for CI parsing; the Phase 4 Export findings action additionally emits review-<ts>.json + review-<ts>.sarif for CI gates.

Heartbeat lines print at each phase boundary so the user sees forward motion on long runs.

Phase 4 — Post-report menu

Presenting the menu is a mandatory closing step — a finished report is the start of Phase 4, not the end of the turn (skipped only when every section says "None found." with no appendix). Ordering is a hard rule: the full report renders as text first, then the menu immediately after it with zero tool calls in between — so the menu can never appear before (or without) the report — and the menu's question line repeats the verdict summary in case the report has scrolled off-screen. The primary menu (AskUserQuestion, single-select, ≤4 options) offers four modes:

• Fix by scope… — apply by severity scope (critical / +important / +debt) or a Custom expression mixing severity letters and finding IDs/ranges (e.g. I D #11, 1-7, 11).
• Triage one-by-one — walk each must-fix finding with a per-finding micro-menu (Fix · Ask · Create ticket · Snooze · Wontfix · Skip).
• More actions… (multi-select) — Save full report · Ask a follow-up question · Generate tests · Create a ticket/issue · Export findings (JSON + SARIF) · Deep-dive · Generate fix patches · Draft commit/PR · Post to GitHub PR · Snooze · Wontfix · Schedule re-review · Re-run on fixed code.
• Skip / done.

The two fix modes appear only when fixable findings exist; otherwise the menu leads with More actions… so the non-fix choices stay reachable (this is the fix that restored discoverability). After a clean apply-fixes (all post-fix gates pass), an auto-delta scoped review runs against the just-edited files and appends a ## Post-fix delta section.

Gate mode — headless verdict (CI / autonomous loops)

/review-all gate (or any target with --ci) runs Phases 0–2.75 unchanged, then replaces the Phase 3 report and Phase 4 menu with a machine-readable verdict — no prose, no AskUserQuestion. It writes .claude/review-all/gate-verdict.json, prints the same JSON, and exits 0 (pass) / 1 (blocked) / 2 (malformed):

{ "pass": false, "severityFloor": "critical", "partial": false, "blockingCount": 1,
  "blocking": [ {"id": "F3", "severity": "CRITICAL", "file": "src/x.ts", "line": 42, "title": "unguarded null deref"} ] }

A finding blocks only when its severity meets the floor (gateSeverityFloor, default critical → 🔴 only; --severity important → 🔴+🟠). Only main-report findings (verdict keep, score ≥ 75) gate — the appendix never blocks, and neither does an unverified finding at any score, since nobody observed the behaviour it claims. Partial review coverage fails closed. This is what lets a CI step or an autonomous loop (e.g. the goal-loop plugin's oracle) consume review-all as a hard gate. See skills/review-all/references/phase-gate.md.

How it's tested & improved

Every change to this skill is eval-driven — the same develop-tests loop Anthropic recommends for agent harnesses:

• 92 labeled scenarios (skills/review-all/evals/*.json) across Java, TypeScript, Python, SQL, Go, and Rust. Most are recall cases (a planted real bug the review must catch: races, leaks, injections, N+1s, broken contracts…); a growing set are precision counter-cases — correct code that looks suspicious (an intentional except Exception boundary, a consistent lock discipline, a neutralized CSV export, a TODO comment) that must NOT become a finding. Two cases exercise gate mode end-to-end; two guard the profile cache (a poisoned legacy cache must MISS, a valid warm cache must HIT and still apply its rules); two cover the repo-convention files (a REVIEW.md that must raise one finding's severity and suppress another in the same review, and a CLAUDE.md claim the diff falsifies alongside a neighbouring claim that stays true); and one carries a runnable toolchain whose test gate genuinely fails, so the failing-gate path is exercised end-to-end rather than assumed. The two convention cases were rewritten after their first A/B revealed the originals passed on the pre-feature baseline too — evals/README.md records that finding and the rule it produced: a case proves nothing until the baseline arm has been shown to fail it.
• Headless LLM-graded runner (scripts/run-evals-headless.sh): each fixture is materialized into a throwaway git repo, /review-all runs there via claude -p, and a second LLM call grades the report against the case's rubric. Single runs flicker (LLM output is non-deterministic), so trustworthy baselines use REVIEW_ALL_EVAL_RUNS=3+ and compare pass-rates.
• A/B before shipping: persona or verifier edits are measured against the relevant eval subset with and without the change — a change that doesn't move recall without hurting precision is reverted. scripts/eval-scorecard.py turns the runner's per-case RESULT/SCORE lines into a suite-level recall % / precision % / F1 / SNR scorecard, so an A/B diffs an aggregate precision number, not just per-case PASS rates (the SNR is an honest suite-derived proxy, not a CR-Bench per-comment metric).
• No-API CI gates on every push (tests/run.sh): anonymization check (no real project names in fixtures), eval-schema validation, shellcheck on all scripts, Python unit tests, and a static doc-invariant gate per instruction-only feature — the Phase 4 menu, verifier votes, the dismissed digest, claim classes, REVIEW.md, diff ordering, and <gate_results> (tests/check-*.sh). Instructions can't be exercised headlessly, so their load-bearing sentences are grepped from the published docs instead, and each gate is proven non-vacuous by deleting the sentence on a scratch copy and confirming it fails. tests/check-config-sync.sh goes further and set-diffs SKILL.md's config schema against the documented key table, so that pair cannot drift again.
• Every real-world escape becomes a case: a missed bug or a false positive observed in actual use is converted into a new eval before the fix lands, so it can never regress silently.

See skills/review-all/evals/README.md for the schema, the full scenario list, and the iteration loop.

Pros / Cons

Review instructions (REVIEW.md)

Drop a REVIEW.md at the repository root to change what this review flags, at what severity, and where. It is read fresh on every run (never cached) and injected verbatim into all ten agents and the verifier as the highest-priority instruction block — where it conflicts with a persona or with the shared rules, REVIEW.md wins. No configuration key is involved: the file is the interface.

# Review instructions

## What CRITICAL means here

Reserve 🔴 for findings that break behavior, leak data, or block a rollback:
incorrect logic, unscoped queries, PII in logs, non-backward-compatible
migrations. Style and naming are 🟡 at most.

## Raise the bar per path

- In `scripts/`, only report if near-certain and severe.
- In `src/payments/`, treat any missing error handling as 🔴 CRITICAL.

## Do not report

- Anything CI already enforces: lint, formatting, type errors
- Generated files under `src/gen/` and any `*.lock` file

## Always check

- New API routes have an integration test
- Database queries are scoped to the caller's tenant

Notes:

• Verbatim means verbatim — the file is never summarized or truncated, and @-imports are not expanded (unlike CLAUDE.md). Write the rules you want enforced directly in the file. Keep it focused: a long REVIEW.md dilutes the rules that matter, and it is injected into eleven-plus prompts. Soft guideline ≤150 lines; over 10 KB the review warns.
• It steers what is reviewed, never how it is proven. Severity recalibration, skip rules, and per-path bars are honored. The 3-question gate, claim classes, file:line evidence, and Phase 2.5 verification are not overridable — a severity REVIEW.md promotes still has to be earned by proof at that tier.
• In gate mode it moves findings across the blocking floor in both directions — a promotion to 🔴 becomes CI-blocking under the default floor, and a demotion can un-block a real defect. Same trust model as CLAUDE.md: text committed to the repo is authoritative. When REVIEW.md is itself modified in the reviewed diff, the gate summary says so.

Optional configuration

Drop a .claude/review-all.json into any project to tune behavior. All keys optional; documented defaults apply when absent. See skills/review-all/references/config-keys.md for the full table with per-key rationales.

Common keys:

{
  "devServerPorts": [4200, 5173, 3000],
  "verifierModel": "haiku",
  "verifierVotes": 1,
  "extraAgents": [],
  "skipAgents": []
}

verifierVotes defaults to 1 (single hostile pass). Set it to an odd N>1 (e.g. 3) to majority-vote the 🔴/🟠 findings across N independent verifier passes — a finding reaches the main report only if ⌈N/2⌉ verifiers keep it. Voting is scoped to top severity (🟡/🔵/⚪ stay single-pass) and adds verifier cost only when 🔴/🟠 survivors exist; see references/config-keys.md.

Finding-count caps

Two layers trim a report. Both are config-driven — set the relevant keys to 0 for a complete verified list. 🔴 CRITICAL / 🟠 IMPORTANT are never capped at any layer.

To get every verified finding, zero out both layers for the tier — a per-agent quota drops findings before dedupe, so a global cap alone cannot recover them:

{
  "quotaDebt": 0,
  "quotaSuggested": 0,
  "quotaQuestion": 0,
  "suggestedGlobalCap": 0,
  "questionGlobalCap": 0
}

/review-all init walks an interactive wizard that writes a populated config.

Optional: CodeGraph

If a CodeGraph MCP server is wired into Claude Code and the project has a .codegraph/ directory, /review-all uses its tools for cross-file analysis (callers, callees, impact). Tool names are resolved at runtime, so any MCP namespace works. Without CodeGraph, the relevant agents fall back to grep / git grep.

Requirements

• Claude Code CLI
• git, bash, python3 (defaults on macOS/Linux)
• gh — for PR #N review mode, and optionally for the Phase 4 Post to GitHub PR (gh pr comment) and Create a ticket/issue (gh issue create) actions; both are write-scoped and confirmation-gated, and Create-ticket falls back to writing an issue-markdown file when gh/GitHub isn't present

Layout

claude-review-all/
├── skills/review-all/
│   ├── SKILL.md              # orchestrator entry point
│   ├── agents/               # 10 persona files + _shared.md + verifier.md
│   ├── references/           # per-phase rules, config schema, state-file lifecycle
│   ├── evals/                # labeled scenarios + success criteria + grader rubrics
│   └── scripts/              # discover (one-call Phase 0), preflight, detect-toolchain,
│                             # dev-server-probe, test-pattern-probe, dedupe, state-sweep,
│                             # gate-verdict, export-findings, validate-evals,
│                             # materialize-fixture, run-evals, run-evals-headless,
│                             # eval-scorecard (recall/precision/SNR aggregate),
│                             # agent-order (per-agent diff permutation)
├── tests/                    # unit tests + check-anonymization.sh (gitignored blocklist)
│                             # + one check-*.sh doc gate per instruction-only feature
└── .github/workflows/ci.yml  # shellcheck + test suite (incl. anonymization + eval-schema gates)

All plain Markdown / shell / Python — read, fork, extend.

Development

bash tests/run.sh            # anonymization gate + eval-schema validation + shellcheck-clean shell scripts + Python unit tests (no API key)

tests/run.sh runs four no-API gates: an anonymization gate (tests/check-anonymization.sh — fails if a real employer/product/ticket name leaks into published artifacts; the real blocklist is gitignored, a placeholder *.example.txt ships), eval-schema validation (scripts/validate-evals.py — every evals/*.json must have a non-empty grader.rubric, a materializable fixture, and an id matching its filename), the shell-script tests, and the Python unit tests. CI (.github/workflows/ci.yml) runs shellcheck + this suite on every push / PR. The eval suite under skills/review-all/evals/ is materialized into throwaway git repos and LLM-graded headlessly by scripts/run-evals-headless.sh (needs the claude CLI); see skills/review-all/evals/README.md.

License

MIT — see LICENSE.