Duplicate Invoices and Receipts: How They Get In, and How to Catch Them Automatically

Paying the same bill twice is one of the easiest mistakes to make and one of the most annoying to undo. Usually nobody notices until the vendor statement doesn't match, or the books show more spend than the bank account does.

Automating data entry makes duplicates both more likely and easier to stop. More likely, because documents now flow in from every inbox and folder without a person looking at each one. Easier to stop, because once every document is structured data, you can compare it to everything you've already recorded before saving it. Catching duplicates is one part of accounts payable automation.

How the same document gets in twice

Most duplicates come from a short list of situations:

• The vendor emails the invoice, then emails a payment reminder with the same PDF attached.
• The invoice arrives by email and on paper, and both get captured.
• Two people on the team forward the same email to the invoices inbox.
• A receipt is photographed at the till, and the store's email receipt is forwarded later.
• A file gets uploaded again after a failed or slow upload.
• A vendor sends a corrected invoice with the same number, and the original was already entered.

Each of these produces a slightly different kind of duplicate. A forwarded email is often the identical file. A paper scan and the emailed PDF are the same invoice in two different files. A till photo and an email receipt are the same purchase with no shared number at all. One check can't catch all three, so you need three.

Check 1: the identical file

Hash the file bytes before you do anything else. If you've seen that hash before, it's the same file, and you can skip it without sending it for extraction at all. That also saves a document from your monthly quota.

import hashlib

def file_hash(path: str) -> str:
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(65536), b""):
            h.update(chunk)
    return h.hexdigest()

This catches forwarded emails, double uploads and re-sent attachments. It won't catch a re-scan or a re-saved PDF, because changing a single byte changes the hash.

Check 2: same vendor, same document number

For invoices, the vendor plus the invoice number should be unique. The catch is that both are written inconsistently. "ACME Corp.", "Acme Corporation" and "ACME CORP" are the same vendor. "INV-00412", "inv 412" and "#00412" may be the same number.

Normalize both before comparing:

import re

SUFFIXES = r"\b(inc|incorporated|llc|ltd|limited|corp|corporation|co|gmbh|sarl|sa|bv|plc)\b"

def norm_vendor(name: str | None) -> str:
    name = (name or "").lower()
    name = re.sub(r"[^\w\s]", " ", name)        # drop punctuation
    name = re.sub(SUFFIXES, " ", name)           # drop company suffixes
    return re.sub(r"\s+", " ", name).strip()

def norm_number(number: str | None) -> str | None:
    number = re.sub(r"[^A-Za-z0-9]", "", (number or "")).upper()
    number = re.sub(r"^(INVOICE|INV|NO|NR)", "", number)
    return number.lstrip("0") or None  # "000" or "" means no usable number

Stripping the prefix and leading zeros is a judgment call. If one of your vendors really issues INV-12 and CN-12 as different documents, keep the prefix for that vendor.

Then look it up:

def find_number_match(db, vendor, number):
    key = norm_number(number)
    if not key:
        return None
    return db.execute(
        "SELECT id FROM documents WHERE vendor_key = ? AND number_key = ?",
        (norm_vendor(vendor), key),
    ).fetchone()

A match here is a duplicate with high confidence. The one legitimate exception is a corrected invoice reissued under the same number, which you want to replace the original, not sit beside it. Send these matches to review rather than discarding them.

Check 3: same vendor, same amount, close dates

Most receipts have no reliable number, and some invoices don't either. For those, compare vendor, total and currency, within a few days:

from datetime import date, timedelta

def find_amount_match(db, vendor, total, currency, doc_date, window_days=3):
    if not (total and currency and doc_date):
        return None
    d = date.fromisoformat(doc_date)
    return db.execute(
        """SELECT id FROM documents
           WHERE vendor_key = ? AND total = ? AND currency = ?
             AND doc_date BETWEEN ? AND ?""",
        (norm_vendor(vendor), total, currency,
         (d - timedelta(days=window_days)).isoformat(),
         (d + timedelta(days=window_days)).isoformat()),
    ).fetchall()

This catches the till photo plus email receipt, and the paper invoice plus PDF where the number was misread on one of them.

The window is what keeps it from flagging real repeat charges. A software subscription bills the same amount every month, about 30 days apart, so a three-day window ignores it. A window of a few days still covers the gap between a purchase date and the date an email receipt was sent.

Some real purchases will match anyway. A case that comes up on the QuickBooks Community forum is two fuel receipts for the same amount, two days apart, both genuine. With a three-day window this check flags them, which is why it should send matches to review and never delete them.

Putting it together

If clients send a quarter's documents at once, upload them as one batch in the DocuParse API dashboard. Each file is checked on its own, so one unreadable photo doesn't hold up the rest. Then run the checks below on the results before anything reaches the books.

Run the checks in order, simplest first, and only send a document for extraction if it passes the first one:

def ingest(db, path, extract):
    h = file_hash(path)
    if db.execute("SELECT 1 FROM documents WHERE file_hash = ?", (h,)).fetchone():
        return "skipped: identical file"

    data = extract(path)  # e.g. the parse_document() helper from our Python guide
    vendor, number = data.get("merchant"), data.get("document_number")

    status = "ok"
    if find_number_match(db, vendor, number):
        status = "review: same vendor and number"
    elif find_amount_match(db, vendor, data.get("total"), data.get("currency"), data.get("date")):
        status = "review: same vendor, amount and date"

    db.execute(
        """INSERT INTO documents (file_hash, vendor_key, number_key, total, currency, doc_date, status)
           VALUES (?, ?, ?, ?, ?, ?, ?)""",
        (h, norm_vendor(vendor), norm_number(number), data.get("total"),
         data.get("currency"), data.get("date"), status),
    )
    db.commit()
    return status

The extract function can be anything that returns the fields. With DocuParse API, document_number holds the invoice number or receipt number, whichever the document has, and total comes back as a two-decimal string like "412.50", so amounts compare exactly without float rounding. The full submit-then-fetch helper is in the Python invoice parsing guide.

Store every document, including the flagged ones, with its status. A reviewer then sees the new document next to the one it matched and picks: duplicate, replacement or genuine.

If you work in a spreadsheet

You can do a lighter version without code. In Google Sheets, with vendor in column C, number in column J and total in column H, add a helper column:

=IF(COUNTIFS(C:C, C2, J:J, J2) > 1, "Same number", IF(COUNTIFS(C:C, C2, H:H, H2) > 1, "Same amount", ""))

It doesn't normalize vendor names or check dates, so it will flag more than it should. It still turns a hidden problem into a visible column you can sort. The spend dashboard tutorial uses the same column layout.

The last line of defense

No check catches everything. A vendor that sends the same charge on two invoices with different numbers needs a human or a bank reconciliation to spot. Reconcile against the bank and card statements every month. If the books show two payments where the bank shows one, you'll find it there.

This post is part of the receipt and invoice data entry guide. To test these checks on real documents, Sign up free for 20 documents a month with full API access. The invoice parser API page lists the fields it returns, including document_number.