FREE1 free audit every month. No card, every finding included.

ASTER / THE BENCHMARK LAB

No score
without the evidence.

Aster hasn’t published a benchmark yet. When it does, you’ll see the dataset, the bug counts and the test date beside it. Until then, here’s how to read any score, including ours.

  • Visible sources
  • No unpublished scores
Aster score
not published yet
NOT PUBLISHEDA score appears here only with its dataset, bug counts and test date.

HOW TO READ THIS

A percentage
is a fraction.

Every rate you see is bugs found divided by bugs evaluated. Look at the second number: tools in this study weren’t all tested on the same count.

  1. FoundKnown bugs the tool detected.
  2. EvaluatedBugs the tool finished. Failed and skipped cases drop out.
  3. RateFound ÷ evaluated. Compare rates only with both numbers in view.

57 found ÷ 118 evaluated = 48.3%

Each dot is one evaluated bug. The grid holds 118, the largest evaluated count in Macroscope’s study.

THE NUMBERS

Rate, count, source.
Side by side.

Bug detection rates published in Macroscope’s 2025 study, with bug counts and source for each tool. No Aster score is published.
ToolDetection rateBugs found / evaluatedSource
Macroscope48.3%57 / 118Macroscope · 2025 study
CodeRabbit45.8%54 / 118Macroscope · 2025 study
Cursor Bugbot42.4%50 / 118Macroscope · 2025 study
Greptile23.6%17 / 72Macroscope · 2025 study
Graphite Diamond18.3%21 / 115Macroscope · 2025 study

STUDY DETAILS

One published study.
One we haven’t published yet.

PUBLISHED

Macroscope’s 2025 study

Publisher
Macroscope
Evaluation
September 2025
Tested
August 25–September 14, 2025
Focus
Known runtime bugs across eight languages
Bugs evaluated
72 to 118, depending on the tool
Read the study (opens in a new tab)
NOT PUBLISHED

Aster’s own benchmark

Score
Not published
Focus
Security issues common in AI-built apps
Dataset
Published with the score
Bug counts
Published with the score
Test date
Published with the score

No Aster number appears on this site until all of these exist.

WHAT MAKES A SCORE USEFUL

A clear test.
A result you can inspect.

Before you trust any score, including ours, check these four things.

  1. 01

    The dataset

    Ask which repos, commits and languages were tested. Then check the tool versions, settings and test date.

  2. 02

    The evidence

    Each finding should match a known bug. A detection rate alone won’t tell you how many suggestions were wrong or noisy.

  3. 03

    The denominator

    Failed and skipped cases matter. Look for how many bugs each tool was actually tested on, next to the percentage.

  4. 04

    The source

    Can you read the study yourself? Macroscope’s is linked here. Aster’s will be published the same way, with the evidence, or not at all.

Runtime bugs and security coverage measure different things.

The cited study tests known runtime bugs across eight languages. Aster checks something else: leaked keys, Supabase and Firebase access rules, package versions with known holes, JavaScript/TypeScript syntax and some risky code patterns. It doesn’t yet check webhooks, admin access or rate limits. A high bug-detection score, from any tool, is not a security guarantee. See what Aster checks.

START WITH YOUR OWN CODE

The benchmark that matters
is your own repo.

Judge Aster on your own code. You get each finding critical first, with the file, the line and a plain-English why. 1 free audit every month.

Aster

CHECKING

YOUR GUIDE TO ASTER

Hi, I’m Aster.
Ask me what Aster checks.

I can’t see your code from here. Please don’t paste secrets or private code.

Prepared answers are always available.