Email Verification Accuracy: What 99% Claims Actually Mean and How to Test Any Provider

Every verification service advertises an accuracy number, and almost nobody buying verification knows what it measures or how to check it. Accuracy is not one number: it is precision, recall, and unknown rate pulling against each other, and a headline percentage can be gamed by dumping hard cases into the unknown bucket or testing against easy lists. This guide gives you the math in sender terms, a ground-truth test method built from your own send history, and a working script that scores any provider.

Three Ways Accuracy Claims Get Gamed

Accuracy marketing is not usually lying; it is selective measurement. Three techniques produce a big headline number without a correspondingly good tool:

Never compare two providers by their homepage numbers. Compare them by running the same messy list through both and scoring the disagreements.

Precision, Recall, and Unknown Rate in Sender Terms

The three metrics map directly onto sender pain. Precision failures bounce: the tool said passed, you sent, the mail came back 550 and your reputation absorbed the hit. Recall failures cost revenue silently: the tool said failed on a live mailbox and a real contact left your list forever. Unknown inflation costs decisions: the tool handed the hard third of your list back to you unclassified.

For most senders, precision on the passed bucket is the metric that protects the domain, and a low unknown rate is the metric that makes the tool useful. A provider with 97 percent precision and a 5 percent unknown rate beats one with 99 percent claimed accuracy and a 25 percent unknown rate on any real-world list, because the second tool made you do a quarter of the work yourself.

Building a Ground-Truth Test List

You already own the best test data available: your send history. It reflects your actual audience mix (your domains, your geography, your B2B-to-consumer ratio), which no vendor benchmark does.

The Test Script

This script reads your ground-truth CSV (columns: email, truth where truth is good or bad), verifies each address through the real-time email verification API, and prints precision, recall, and unknown rate. Point it at any provider by swapping the endpoint and field mapping.

import csv, os, time
import requests

API_KEY = os.environ["BEC_API_KEY"]
ENDPOINT = "https://api.bulkemailchecker.com/real-time/"

tp = fp = fn = tn = unknown = total = 0

with open("ground_truth.csv", newline="") as fh:
    for row in csv.DictReader(fh):
        total += 1
        try:
            r = requests.get(ENDPOINT, params={
                "key": API_KEY, "email": row["email"]
            }, timeout=30)
            status = r.json().get("status", "unknown")
        except Exception:
            status = "unknown"

        if status == "unknown":
            unknown += 1
        elif status == "passed" and row["truth"] == "good":
            tp += 1
        elif status == "passed":
            fp += 1   # said valid, truth was bad: future bounce
        elif row["truth"] == "good":
            fn += 1   # said invalid, truth was good: lost contact
        else:
            tn += 1
        time.sleep(0.2)  # be polite to the API

classified = total - unknown
print(f"Precision: {tp / max(tp + fp, 1):.3f}")
print(f"Recall:    {tp / max(tp + fn, 1):.3f}")
print(f"Unknown:   {unknown / max(total, 1):.3f}")
print(f"Classified accuracy: {(tp + tn) / max(classified, 1):.3f}")

The response fields the script relies on (status, event, and the boolean flags) are documented in the email verification API documentation. Testing a provider that only offers file upload works the same way: run the CSV through their bulk email verification equivalent, download the results, and score the columns with the same math.

Reading the Results (What Good Looks Like)

One refinement worth running: break the scores out by domain type. Compute the same three metrics separately for consumer domains, corporate domains, and known catch-all domains. Providers converge on consumer mail and diverge hard on the corporate and catch-all segments, which is where your test tells you something the homepage never will. A quick way to identify which of your test domains are catch-all before scoring is a pass through the domain verification tool.

Why Results Drift and What It Means

Verification answers a question about a moment in time. Mailboxes fill, employees resign, domains lapse, and disposable providers mint new domains daily, so published retest methodology showing 3-8 percent classification drift over 30 days is not a flaw in any tool; it is the ground moving. Two practical consequences follow.

First, verification age belongs next to every address in your database (batch dates in the email verification dashboard make this easy to backfill), because a passed from January is a weaker signal in June. Second, re-verification cadence is an accuracy decision: quarterly re-verification caps accumulated drift at roughly the same few percent the best tools achieve on day one, while an annual cadence lets drift dwarf provider differences entirely. Choosing a great verifier and then running it once a year is buying a precision instrument and never calibrating it.

The Honest Truth About Unknowns

Some unknowns are legitimate. A catch-all domain accepts every RCPT TO probe, so no SMTP-based tool on earth can confirm the specific mailbox; an engine that keeps deferring past a greylisting retry budget has genuinely run out of road. The difference between an honest unknown and an unknown dump is the rate and the transparency: a strong engine returns unknown on the genuinely undecidable few percent and tells you why (event codes like is_catchall, visible on any address through the instant email verification check), while a weak one returns unknown on everything that required effort.

Frequently Asked Questions

The Bottom Line

Accuracy claims are the least useful information on any verification homepage, not because they are false but because they are unfalsifiable as presented. The falsifiable version takes an afternoon: your own send history as ground truth, one script, three numbers. Precision protects your domain, recall protects your revenue, and the unknown rate tells you how much of the job the tool actually did.

Run the test before committing to any provider, re-run the delayed-bounce audit 30 days after your first cleanup, and let your own numbers, on your own list, make the decision.