How will Clervo measure drainer detection?

Against a labeled corpus of serialized Solana transactions — known drainers and known-benign transactions — executed in a real simulator, with negative controls that must flip the verdict when the detection path is disabled, and every result linked to a provenance-tracked evidence artifact.

Benchmark

How Clervo measures drainer detection.

Q: Does Clervo publish detection accuracy numbers?

Not yet. This page describes the benchmark methodology only. No precision, recall, or latency figure is published until a real run produces a linked evidence artifact. A number measured on a planted calibration fixture would be labeled as calibration, not as production accuracy.

Clervo does not publish a detection score until a real run produces a linked evidence artifact. This page is the methodology — the corpus, the metrics, and the controls — so the eventual numbers are auditable, not asserted. No precision, recall, or latency figure appears here yet.

What corpus does the benchmark use?

A labeled set of serialized Solana transactions: known drainer transactions (replayed from documented incidents and adversarial fixtures) and known-benign transactions (ordinary transfers, swaps, and approvals). Each transaction is stored as a base64 payload with a ground- truth label and a source note, so any result traces back to its input.

What does the benchmark measure?

The rate of correct BLOCK or REVIEW on the malicious set and correct ALLOW on the benign set, reported as separate false-ALLOW and false-BLOCK rates rather than a single blended score. Latency is measured for the full hot path — decode, load pinned state, simulate, decode effects, compare to intent. Ambiguous cases resolve to REVIEW by design and are reported as such, not counted as failures.

What are the negative controls?

Every claimed detection is checked with a control that disables the detection path: when the intent comparison or the simulation is broken on purpose, the verdict must flip away from the correct answer. A benchmark that passes with the detector switched off is measuring nothing, and is rejected.

How is a result proven?

Each run writes a provenance-linked evidence record — run id, target, source commit or tree hash, parent digest, stage version, and the raw simulator dump digest. A published number is derived from that artifact, never from a summary typed by hand. A result measured on a planted calibration fixture is labeled calibration and never presented as production precision or recall.

When will the numbers be here?

When the engine can execute the full corpus end to end and write the evidence artifacts. Until then, publishing a number would violate Clervo's first rule: verify before you claim. Follow the waitlist for the release.

Join the waitlist Back to overview