n8n's Compare Datasets node: spotting gaps, duplicates and mismatches between two sources

"The CRM says 1,842 active customers, billing counts 1,796." Every organization that keeps the same data alive in two systems knows that moment — and the usual manual answer, the Excel export with VLOOKUP, doesn't scale and doesn't rerun every night. n8n's Compare Datasets node industrializes exactly that reconciliation: two inputs, a match field, and four outputs that tell the whole story — same, different, present only on one side, present only on the other. The topic is less anecdotal than it looks: the reference survey by Chu et al. presented at SIGMOD 2016 ("Data Cleaning: Overview and Emerging Challenges", see on Google Scholar) reminds us that detecting and repairing inconsistencies across sources remains one of the major costs of information systems — and that automating the reconciliation is its first lever.

Anatomy of the node

Compare Datasets takes two inputs (Input A and Input B) and is configured around one central parameter, Fields to Match: the field or fields identifying the same entity on both sides — an email, a SKU, an order ID. The node produces four outputs:

• In A only: the entity exists in A but not in B (say, a CRM customer missing from billing).
• In B only: the reverse.
• Same: present on both sides, all compared fields identical.
• Different: present on both sides, but at least one field diverges.

Each output wires into its own downstream handling: that's what makes the node so expressive — the workflow is the reconciliation policy.

Configuration: the three settings that matter

The match fields. Pick a stable, normalized identifier. A raw email comparison will fail on Jane.Doe@example.com vs jane.doe@example.com: normalize upstream (lowercase, trim) with a Set node or an expression. Several match fields combine (SKU + warehouse for multi-location stock).

The fields to skip. Two systems always carry technical fields that diverge without business meaning (updated_at, internal IDs). Exclude them from the comparison, or the "Different" output becomes permanent noise drowning the real divergences.

Difference resolution. When an item diverges, the node can prefer version A, version B, or include both versions in the output item. For serious reconciliation, keep both versions: the rest of the workflow logs the discrepancy, applies the source of truth field by field, and a trace remains of what was overwritten.

Three typical reconciliations

CRM versus billing. Matching by email or company ID between HubSpot/Pipedrive and the invoicing tool: "In A only" goes to record creation, "Different" feeds a review of diverging contact details. The same pattern holds for Salesforce.

ERP stock versus Shopify. The nightly full count described in our guide to Shopify inventory sync: both systems' levels are exported, matched by SKU, and each output has its consequence — quantity gaps corrected per the source of truth, missing SKUs escalated to the catalog team.

Today's export versus yesterday's. Compare Datasets also works as a change detector when a source has neither webhooks nor an events API: keep the previous export (in a Data Table or a file), compare it with the new one, and "In B only" + "Different" become your synthetic event stream — new items and modifications, ready to trigger what follows.

Downstream: what to do with each output

The value of the reconciliation is decided after the node. The robust scheme:

1. Same: nothing, or a simple counter for the report.
2. Different: log the discrepancy (field, value A, value B, timestamp) then apply the truth rule — possibly a direct SQL write via the Postgres node.
3. In A only / In B only: create or archive depending on the sync direction, or queue ambiguous cases for human review with Slack approval.
4. Report: an Aggregate counts each category and posts a daily digest ("1,793 same, 12 gaps fixed, 3 in review") to Slack — and a loud alert beyond a threshold, because 400 discrepancies on a Tuesday morning aren't a fluctuation, they're an incident.

That discipline — observe, trace, correct, alert — is the core of the auditable pipelines we package in the Compliance & Audit Pack (€149), where every corrected value keeps the memory of its original.

Compare Datasets or Merge?

Keep the usage distinction in mind: Merge assembles (join, enrichment, concatenation), Compare Datasets observes (same, different, missing). If your need contains the words "gap", "reconciliation", "what's missing" or "what changed", it's Compare Datasets; if it contains "enrich" or "combine", it's Merge — and to remove duplicates within a single source, it's Remove Duplicates. Three nodes, three questions, and workflows that stay readable.