vibecheck

vibecheck

39 rules for catching AI-generated code smells. ESLint for AI slop.

npx @yuvrajangadsingh/vibecheck .

  src/api/routes.ts
    12:5    error  no-hardcoded-secrets   Hardcoded secret detected. Use environment variables instead.
    45:3    error  no-empty-catch         Empty catch block swallows errors silently.
    89:1    warn   no-console-pollution   console.log left in production code.

  src/utils/db.ts
    34:5    warn   no-sql-concat          SQL query built with string concatenation.

  4 problems (2 errors, 2 warnings)
  2 files with issues out of 47 scanned (0.8s)

Why

AI coding tools generate code that works but cuts corners. Empty catch blocks, hardcoded secrets, as any everywhere, comments that restate the obvious. CodeRabbit found AI-generated PRs have 1.7x more issues than human PRs. Veracode reports 45% of AI code samples contain security vulnerabilities.

vibecheck catches these patterns with zero config, zero API keys, zero cost. Runs locally, stays fast.

Install

# Run directly (no install)
npx @yuvrajangadsingh/vibecheck .

# Install globally
npm install -g @yuvrajangadsingh/vibecheck

# Standalone binary (no Node required)
curl -fsSL https://github.com/yuvrajangadsingh/vibecheck/releases/latest/download/vibecheck-darwin-arm64 -o vibecheck
chmod +x vibecheck
./vibecheck .

Standalone binaries (built with Bun) available for macOS (arm64, x64) and Linux (x64, arm64) on the releases page. No Node.js or npm required. Python developers can use these to scan their codebases without installing Node.

What it catches

Security

Error Handling

Code Quality

AI-Specific Tells

Framework

Options

vibecheck [path] [options]
vibecheck rules [--json]

Options:
  -c, --config <file>     Path to config file
  -d, --diff              Only scan lines changed in git diff (unstaged)
  --staged                Only scan lines changed in git diff --cached (staged)
  --diff-stdin            Only scan lines changed in a unified diff read from stdin
  --fix                   Auto-remove fixable findings (AI attribution comments)
  --format <format>       Output format: pretty, compact, json, quiet, gh, sarif (default: pretty)
  --json                  Output as JSON (alias for --format json)
  --ignore <patterns...>  Additional ignore patterns
  --severity <level>      Minimum severity to report: error, warn, info (default: warn)
  --fail-on <level>       Exit 1 at this severity: error, warn, info, never (default: error)
  --max-warnings <n>      Exit 1 when more than n warnings are reported
  --statistics            Append per-rule finding counts (pretty and json output)
  --update-baseline       Record current findings in .vibecheck-baseline.json, then exit 0
  --show-suppressed       List findings suppressed by inline directives (pretty and json)
  --rule <spec>           Set one rule's severity for this run: 'rule-id: error|warn|info|off' (repeatable)
  --no-defaults           Start with every rule off for this run; enable a subset with --rule
  -q, --quiet             Only show summary (alias for --format quiet)
  --mcp                   Start MCP server (stdio transport)
  -v, --version           Show version
  -h, --help              Show help

--format compact prints one path:line:col severity rule message line per finding (cmd-clickable in most editors). --format gh emits GitHub Actions workflow commands so findings show up as PR annotations. --format sarif emits SARIF 2.1.0 for the GitHub Security tab.

vibecheck rules lists every rule with its severity, category, languages, and whether --fix can fix it. vibecheck rules --json emits the same list as JSON.

One-off rule selection

--rule changes a single rule for one run without touching .vibecheckrc — same 'rule-id: level' shape the config file uses, repeatable, off disables:

vibecheck --rule 'no-deep-nesting: off' .                        # everything except one rule
vibecheck --no-defaults --rule 'no-obvious-comments: warn' .     # exactly one rule

--no-defaults starts the run with every rule off (the config file's rules section is ignored too), so only rules enabled by --rule execute — the eslint-style opt-in model when you want it. Flag order doesn't matter; an unknown rule id or level is a usage error (exit 2) with a hint. Rules enabled at info still respect the --severity floor — add --severity info to see them.

Slop score

vibecheck --score .

  slop score  68/100  (B)
  16.5 weighted findings per 1k lines, over 360.2k lines

    code-quality       12.1/KLOC   1042 findings
    security            3.2/KLOC     47 findings
    error-handling      1.2/KLOC     31 findings

One number for how much AI residue a codebase carries, with the breakdown that explains it. It never prints bare: a score with no categories is a vanity metric, and the first question anyone asks is which part is bad.

Per-KLOC, severity-weighted, capped per rule. An error counts 10, a warning 3, an info 1. No single rule may contribute more than 20 points per KLOC — in a 31-repo scan no-deep-nesting alone produced 46% of all findings, so without a cap the score would mostly measure nesting depth while claiming to measure slop. When the cap bites, the output says so and names the rule.

Grades are relative to real code, not to school.

A 90=A scale would be wrong here, because the curve puts the median repo at exactly 50. The first version used one, and it graded five of the six calibration repos an F and gave the cleanest of them a D. The bands are pinned to the corpus quartiles instead, so C means typical.

50 means typical. The curve is 100 * exp(-density * ln2 / D50), where D50 is the median density of a calibration corpus. Above 50 is cleaner than the codebases measured, below is worse. Current calibration: D50 = 30, median of 6 repos (adk-python, codex, App, brandmd, gemini-cli, gitea) on ruleset 1.15.0, measured 2026-08-04. The full manifest ships in src/calibration.json, and you can re-derive it yourself:

npx tsx scripts/calibrate.ts <dir-of-repos> --write

That reproducibility is the point. A hand-picked constant would make every published score unfalsifiable.

The score describes the codebase, not your view of it. It counts every finding regardless of --severity, and counts baselined findings too, so changing what you display never moves the number. Baselining is a decision to defer slop, not to remove it — before this was enforced, writing a baseline sent the score straight to 100 with no code changed, and --min-score could be passed by running --update-baseline. Use --fail-on to gate on new findings; that is what the baseline is for.

What diff mode can and cannot see

Diff mode reports a finding when the line it is anchored to changed, or when your change introduced it. The file is scanned as it was before the change too, so an edit that creates a problem is reported even when the line it is reported on did not move:

  } catch (err) {          // <- reported here
    console.error(err);    // <- the only line you changed
  }

That also covers a function crossing the length threshold, whose declaration never changed, and a build-time guard removed from above an untouched console.log. It deliberately does not resurface a finding that was already there: editing one line inside a function that was already too long stays quiet, because that debt is not yours.

--staged reads the index, not the working tree, so a file staged and then edited is checked as it will be committed. --diff-stdin verifies the piped diff describes the files being scanned, using the blob hashes in its index headers, and refuses to report against a checkout the diff does not match.

A file the scanner cannot read is never reported as clean. Unreadable files, files over the 1MB cap, and staged blobs that are binary, symlinks or submodules exit 2 with the path named. Files you mean to skip belong in ignore, which stays silent.

Known limitations, all needing non-default git configuration:

• A .gitattributes clean filter shifts --diff line numbers, because git describes filtered content while the working tree holds the original. --staged is unaffected.
• A diff from another repository whose index hashes are shorter than 7 characters cannot be verified, and is scanned unverified.
• Explicitly naming a staged symlink scans what it points at. A directory scan refuses it.
• A stream that modifies then deletes the same file is not refused the way other multi-patch streams are.
• A deletion-only change is invisible: only added lines enter the diff map, so deleting a line that creates a finding reports nothing.
• A staged move below git's rename-similarity threshold arrives as an add plus a delete, so the destination reads as a new file.
• When a file already contains an identical finding and gains another, the copies cannot be told apart, so that group falls back to changed-line matching and the new one may go unreported. A baseline can hide it for the same reason.

Not available in diff mode. --score, --min-score and --badge refuse to run alongside --diff, --staged or --diff-stdin. The score is findings per KLOC of a whole codebase; in diff mode only changed lines are scanned and the 1 KLOC floor then divides those findings by 1000, which reads as a codebase score while being nothing of the sort. It also flatters: a commit adding an eval() scored 69 (B) in diff mode against 46 (C) for the same repo scanned whole. Gate pull requests with --fail-on and --max-warnings, which are designed for changed lines, and gate the codebase with --min-score.

Gate a build on it

vibecheck --min-score 60 .

Exits 1 below the threshold. Sits alongside --fail-on and --max-warnings rather than replacing them: those catch specific findings, this catches drift in the whole codebase.

Badge

vibecheck --badge slop.svg .

Writes a shields-style SVG you can commit and reference from your README:

![slop score](./slop.svg)

vibecheck's own, regenerated on every release:

The SVG is self-contained. It renders no external images and calls no external service, so adding the badge does not report your CI runs to anyone, and it cannot break because someone else's server is down. Combines with --score and --min-score; all three read the same computation, so the badge can never disagree with the gate.

Machine-readable form:

vibecheck --score --format json .

adds a score object to the JSON. Without --score the JSON is unchanged, so existing consumers keep parsing what they always did.

Exit codes

--fail-on and --max-warnings count reported findings only: anything hidden by --severity never fails the run. With a baseline present, both gates count new findings only. --fail-on never always exits 0 on a successful scan.

Auto-fix

--fix removes findings that are safe to fix mechanically. Currently that's no-ai-attribution: full-line comments like // Generated by ChatGPT or // Co-Authored-By: Claude get deleted, code lines are never touched. A line is only removed when its content still matches what the scan saw, and results are re-scanned after fixing so reported line numbers stay accurate.

# Strip AI attribution comments across the repo
vibecheck --fix .

Diff mode

Scan only the lines you changed, not the entire codebase. Useful for pre-commit hooks and incremental CI.

# Scan unstaged changes
vibecheck --diff .

# Scan staged changes (pre-commit hook)
vibecheck --staged .

# Scan a diff from anywhere (PRs, other branches)
gh pr diff 42 | vibecheck --diff-stdin .

# CI-friendly output
vibecheck --staged --json .

Paths inside a piped diff are treated as repo-root-relative (what git diff and gh pr diff emit) and resolved against the scan path, so scanning a subdirectory works. Outside a git repository they are taken relative to the scan path.

Pre-commit hook

Add to .git/hooks/pre-commit or use with husky:

#!/bin/sh
npx @yuvrajangadsingh/vibecheck --staged .

Config

Create .vibecheckrc in your project root:

{
  "rules": {
    "no-console-pollution": "off",
    "no-obvious-comments": "warn",
    "no-ts-any": "error"
  },
  "ignore": [
    "node_modules",
    "dist",
    "*.test.ts"
  ]
}

All rules are on by default at their recommended severity. Set any rule to "off" to disable it.

Inline suppressions

Silence a finding where it happens instead of turning the rule off for the whole project:

// vibecheck-disable-next-line
const html = element.innerHTML;

// vibecheck-disable-next-line no-eval, no-ts-any
const out = eval(expr) as any;

const legacy = eval(expr); // vibecheck-disable-line

// vibecheck-disable no-console-pollution

# vibecheck-disable-next-line no-py-print
print("startup banner")

A bare directive suppresses every rule on its target; add comma-separated rule ids to scope it. vibecheck-disable (optionally with rule ids) covers the whole file. Append -- reason to document why: // vibecheck-disable-next-line no-eval -- sandboxed input. The old undocumented vibecheck-ignore marker still works as an alias for vibecheck-disable-line.

Directives are parsed with a string-aware lexer, so directive text inside string literals, template literals, or regex literals never suppresses anything, and a directive must be the first token of its comment. Suppressions run inside the scanner, so the CLI, the GitHub Action, the MCP server, and the VS Code extension all honor them.

In JSX files (.tsx/.jsx) use the expression-comment form, same as eslint: {/* vibecheck-disable-next-line no-eval */}. Bare // and /* */ directives are ignored there, because rendered element text could otherwise forge them. A file-level // vibecheck-disable still works before the first < in the file.

Suppressed findings stay visible: the summary reports how many were suppressed, and --show-suppressed lists them (JSON output gets suppressedCount, plus the full list with the flag). Suppression abuse can't hide silently.

Baseline

Adopting a linter on an existing codebase usually means a wall of findings. A baseline records them so only new ones fail:

vibecheck --update-baseline .   # record current findings (commit the file)
vibecheck .                     # 76 findings (76 baselined, 0 new) -> exit 0

--update-baseline writes .vibecheck-baseline.json in the working directory (found automatically on later scans, like .vibecheckrc). Exit codes, --fail-on, and --max-warnings then count new findings only, and the summary shows N findings (M baselined, K new).

Fingerprints are sha1(rule|path|trimmed snippet) with no line number, so baselined findings survive code moving around a file. Editing a flagged line changes its fingerprint and it comes back as new; re-run --update-baseline after intentional changes. All severities are recorded, so the baseline is independent of --severity. Run scans from the same directory you baselined from (fingerprints use scan-root-relative paths).

VS Code Extension

Get inline diagnostics (squigglies) right in your editor:

ext install yuvrajangadsingh.vibecheck-linter

Available on the VS Code Marketplace. Works in VS Code, Cursor, and Windsurf.

Features: inline error/warning/info markers, Problems panel integration, status bar count, runs on save.

GitHub Action

Add vibecheck to your CI with inline PR annotations:

- uses: yuvrajangadsingh/[email protected]
  with:
    severity: warn       # minimum severity to report (default: warn)
    fail-on: error       # fail the check at this severity (default: error)
    ignore: "tests/**"   # comma-separated ignore patterns
    sarif: false         # upload to the Security tab (see below)

The action automatically scans only files changed in the PR. On push events, it scans the full repo.

Available on the GitHub Marketplace. See it in action on the demo repo.

SARIF / GitHub code scanning

Set sarif: true and findings land in the repo's Security tab next to CodeQL, with stable fingerprints so alerts track across commits:

permissions:
  security-events: write
steps:
  - uses: actions/checkout@v4
  - uses: yuvrajangadsingh/[email protected]
    with:
      sarif: true

The action writes vibecheck.sarif and uploads it via github/codeql-action/upload-sarif (uploaded even when fail-on fails the check, so the Security tab stays current).

CI without the Action

--format gh emits GitHub Actions workflow commands, so findings show up as inline PR annotations from a plain run step:

- run: npx @yuvrajangadsingh/vibecheck . --format gh --fail-on warn

Or upload SARIF to code scanning from a plain workflow:

- run: npx @yuvrajangadsingh/vibecheck . --format sarif --fail-on never > vibecheck.sarif
- uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: vibecheck.sarif

Or keep machine-readable output for other pipelines:

npx @yuvrajangadsingh/vibecheck . --json > vibecheck.json

Pre-commit Hook

Add vibecheck to your pre-commit config:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/yuvrajangadsingh/vibecheck
    rev: v1.2.0
    hooks:
      - id: vibecheck

MCP Server

vibecheck ships with a built-in MCP server so AI coding agents can scan code for slop automatically.

# Start the MCP server
vibecheck --mcp

# Or use the dedicated binary
vibecheck-mcp

Add to your Claude Code, Cursor, or Windsurf config:

{
  "mcpServers": {
    "vibecheck": {
      "command": "npx",
      "args": ["-y", "@yuvrajangadsingh/vibecheck", "--mcp"]
    }
  }
}

3 tools available: scan-files (scan files/directories), scan-diff (scan a git diff), get-rules (list all rules).

How is this different from ESLint?

ESLint catches syntax and style issues. vibecheck catches patterns that are specifically common in AI-generated code.

Real-world results

Scanned 31 repos built with Cursor, Bolt, Lovable, Replit, and v0:

• 14,695 issues across 3,597 files
• 46% of all issues were deep nesting
• Cursor repos averaged 6.36 issues/file (highest)
• Each tool leaves a different fingerprint

Full writeup with per-tool breakdowns

How it works

vibecheck uses regex pattern matching to scan your JS/TS/Python files. No AST parsing, no external APIs, no AI. Each rule has a detection pattern and an anti-pattern to reduce false positives.

It skips node_modules, dist, build, __pycache__, .venv, lockfiles, and minified code by default.

If vibecheck saved you a review cycle, consider starring the repo.

Built by Yuvraj Angad Singh. Also: brandmd (extract any site's design system for AI coding agents) and vemb (embeddings from the command line). Follow on GitHub for new dev-tool experiments.

License

FSL-1.1-Apache-2.0