← Insights

API Integration & Workflow Automation

Integration monitoring: catch the records that never arrive

Integration monitoring that catches missing records: why Zapier, Make and API syncs fail silently, vendor retry limits, and a checklist with thresholds.

Published
Author
Cass & York Team
Series
Cass & York
Reading time
10 min read
0

The Zap history is green. Every scenario in Make shows a check mark. Then finance asks why three closed deals never became invoices in QuickBooks, and nobody can say when the sync stopped or how many records it dropped.

Integration monitoring is the practice of checking that data moving between your systems arrived, complete and on time. Error alerts cover half of it. The other half is counting: every day, compare how many records the source created with how many landed in the destination, per object, and alert on any gap.

A run log only records runs that happened. When a token expires, a webhook subscription gets removed or a trigger stops firing, there is no run to log, so the dashboard stays green while data goes missing. This guide walks through why integrations break, what the vendors' own documentation says about each failure, and a monitoring checklist with thresholds you can set up this week.

Why integrations break without throwing an error

Silent failures usually come from one of five causes. Each is documented by the vendor, and each can leave your automation tool showing a clean history.

Expired OAuth tokens

Every connection between Zapier, Make or a custom script and your CRM or accounting system runs on an OAuth token. Those tokens expire on schedules most teams never write down.

The person who connected the account often leaves the company or changes their password. The sync then fails at the first step, every time, until someone reconnects it.

API rate limits

Every API caps how fast you can call it. When a backfill, a month end batch or a busy sales day pushes past the cap, the API returns HTTP 429 and records wait or drop, depending on how the integration handles retries.

The shared daily quota is the trap. A new marketing tool connected to HubSpot can burn the quota your CRM sync depends on, and the sync fails the next afternoon with nothing changed on its side.

Webhook delivery gaps

Webhooks push an event to your endpoint when something happens: an order is paid, a deal closes. If your endpoint is down or slow, the sender retries for a while and then gives up.

Schema changes

Someone renames a custom field in the CRM, adds a required field in the accounting system, or changes a dropdown value. The integration still runs. It now writes blanks, maps the wrong value or skips records that fail validation. Members of the Zapier community list renamed fields and stopped triggers as common causes of silent failures, right alongside expired auth.

API versions shift too. Stripe builds each event in the API version set on your account when the event occurs, so an account upgrade can change the payload shape your handler expects.

Duplicate retries

Retries fix delivery gaps and create a new problem: the same event arriving twice. Stripe warns that endpoints might receive the same event more than once and tells you to log processed event IDs. Shopify tells you to ignore duplicate deliveries using the X-Shopify-Webhook-Id header.

A slow handler is a common path to a duplicate invoice. Your script takes six seconds to create the invoice in QuickBooks, Shopify marks the delivery as failed at five, and the retry creates a second invoice. Both runs show as successful.

Failure taxonomy at a glance

Failure mode

What the run log shows

What actually happened

The check that catches it

Expired OAuth token

Errors at step one, or no runs at all

Every record since expiry is missing

Token expiry tracking plus a daily count

Rate limit (429)

Some errors, some successes

Records dropped during the busy window

Count of 429s and quota used per day

Webhook subscription removed

Nothing

Sender stopped sending

Daily check that the subscription exists, plus a count

Schema change

Green

Blank or wrong fields, skipped records

Null rate on mapped fields, rejected record count

Duplicate retry

Green, twice

Two invoices or two contacts

Duplicate key count in the destination

Why a green run log can hide missing data

Automation tools report on runs. A run needs a trigger, and most silent failures remove the trigger. A heartbeat on the last step of a Zap will never fire if the first step never fires, a point made in the Zapier community thread on silent failures.

The tools also switch themselves off. Zapier turns a Zap off when 95% of its runs in the last 7 days have errored, and it stays off until someone turns it back on. In Make, a scenario that starts with an instant trigger is deactivated after the first error. After that, the history shows a quiet week, which looks the same as a slow week.

So integration monitoring needs a second signal that lives outside the automation tool: a count of what the source system says happened.

How long a missed event survives

Retry windows decide how long you have to notice an outage before events are gone for good. Look at the gap between Shopify and Stripe below and compare it with how long your team takes to notice a broken sync on a weekend.

Data report

Shopify stops retrying after 4 hours; Stripe keeps going for 72

Automatic retry window for a failed webhook delivery. GitHub does not redeliver failed webhooks automatically. A Friday 6 pm to Monday 6 am weekend outage runs 60 hours.

Hours of automatic retries

Bar chart. Automatic retry window for a failed webhook delivery. GitHub does not redeliver failed webhooks automatically. A Friday 6 pm to Monday 6 am weekend outage runs 60 hours. Use the data point selector below to inspect exact values, or expand the data table.0 h18 h36 h54 h72 hGitHub — Retry window: 0 hShopify — Retry window: 4 hHubSpot — Retry window: 24 hStripe (live mode) — Retry window: 72 hGitHubGitHubShopifyShopifyHubSpotHubSpotStripe (live mode)Stripe (l…

Scroll across the graph to see all categories.

Retry window
0 h
View data table
Shopify stops retrying after 4 hours; Stripe keeps going for 72
CategoryRetry window
GitHub0 h
Shopify4 h
HubSpot24 h
Stripe (live mode)72 h

Stripe figure is live mode. GitHub shows 0 because it has no automatic retries.

Source: Shopify, HubSpot, Stripe and GitHub developer docs, accessed November 3, 2026

A Friday night outage on a Shopify webhook endpoint outlasts the four hour retry window long before Monday. Stripe gives you three days of automatic retries, and its List Events API returns events from the last 30 days with a delivery_success filter so you can replay what failed. GitHub, for comparison, does not redeliver failed webhooks automatically at all. Whatever the window, a daily reconciliation that pulls from the source API recovers what retries miss.

The two layers of integration monitoring

Good integration monitoring watches two layers. The technical layer tells you a run failed. The outcome layer tells you a record is missing, which is the thing the business cares about.

Layer

Signal

Example

Catches

Technical

Errored runs

Zap history, Make incomplete executions, Lambda errors

Failures inside a run

Technical

HTTP 429 and 401 counts

Script logs, API usage dashboards

Rate limits, dead tokens

Technical

Heartbeat or dead man's switch

A ping on each successful run to Healthchecks.io or Cronitor

A job that stopped running

Outcome

Reconciliation count

Paid Shopify orders yesterday vs QuickBooks invoices created from them

Missing records from any cause

Outcome

Freshness

Time since the last record landed in the destination

Stalled triggers

Outcome

Field quality

Share of synced records with a blank mapped field

Schema drift

The outcome layer catches all five failure modes, because it measures the result: the records that arrived. It works the same whether the integration runs on Zapier, Make or your own code.

How to set up reconciliation counts

A reconciliation count asks one question per object per day: did every record the source created show up in the destination? Here is how we build one.

  1. List the flows that touch money or customers. Orders to invoices, closed deals to projects, completed jobs to billing. Start with three to five.
  2. Pick the shared key. The destination record needs the source ID stored on it, such as the Shopify order ID in a QuickBooks invoice memo or a custom field. Without a shared key you can count, but you cannot say which records are missing.
  3. Fix the window. Count records created yesterday, midnight to midnight in one time zone. Run the check early enough in the morning that someone can fix gaps before the day's work depends on them.
  4. Compare counts, then IDs. If the counts match, log it and move on. If they differ, list the source IDs absent from the destination and the destination IDs that appear twice.
  5. Store every result. A table with date, flow, source count, destination count, missing and duplicate gives you a history. The history is how you set fair thresholds and prove the sync is healthy to finance.
  6. Alert the process owner. Send the gap list to the person who owns the outcome, such as the controller for invoices, with a link to the source records.

On a high-volume data pipeline I ran for a state agency before founding Cass & York, encrypted files moved securely into AWS for batch processing. For batch pipelines like that one, we design the check around the file itself: did it arrive by its deadline, did it decrypt, and does the number of rows loaded match what the file contains? A job that finishes on time with fewer rows than it received is a failure, and the count is how you see it.

Integration monitoring checklist with thresholds

These are the starting thresholds we design for. Treat them as first settings and tune them after four weeks of stored results.

Check

Starting threshold

Why

Reconciliation count, financial records (invoices, payments)

Alert on any gap of 1 or more

One missing invoice is lost revenue

Reconciliation count, CRM records (contacts, deals)

Alert above 1% or 5 records, whichever is larger

Small timing differences are normal

Duplicate keys in destination

Alert on any duplicate in financial objects

Catches retry duplicates

Freshness

Alert when the last record is older than 2x the normal gap during business hours

Catches stalled triggers within hours

Volume drop

Alert when a day's count falls below 50% of the same weekday average over the prior 4 weeks

Catches a trigger that stopped without errors

OAuth token age

Alert 30 days before a known expiry; check QuickBooks connections against the 100 day refresh window

Gives time to reconnect

HTTP 401s

Alert on the first one per connection

A dead token fails every record after it

HTTP 429s

Alert when 429s exceed 1% of calls in an hour, or daily quota use passes 70%

Leaves room before the hard cap

Error share of requests

Keep below 5% of daily requests

HubSpot sets 5% as the bar for marketplace apps

Webhook subscriptions

Daily check that each expected subscription exists

Shopify removes subscriptions after persistent failures

Automation tool state

Daily check that every Zap and scenario is on

Zapier and Make switch themselves off

Blank mapped fields

Alert when the null rate on a required field rises 5 points above its 4 week average

Catches renamed or removed fields

Alert path test

Break a test flow on purpose once a month

Proves the alerts still reach a person

The volume and freshness checks need a baseline that respects weekends, holidays and business hours. Comparing Tuesday with the average of the last four Tuesdays avoids most false alarms.

What production integrations taught us

During our Shopify engineering engagement we worked on event integrations with Salesforce and on production maintenance. Event integrations are where retries, duplicates and order matter most, because Stripe and Shopify both document that events can arrive late, twice or out of order. Our rule since then: every handler stores the event ID, returns a 2xx fast, does the work from a queue, and a scheduled job pulls from the source API to fill anything the webhooks missed.

Before Cass & York, I also built communications software for the elevator industry on Node.js, TypeScript and AWS Lambda. In a system like that, the failure that matters is silence. We carry that into every integration: each one gets a check that expects a signal on a schedule and raises an alarm when the signal stops.

Who should own integration alerts

An alert with no owner gets muted within a month. Give every integration one named owner and route the outcome alerts to the person who feels the gap: finance for invoices, sales ops for CRM records, operations for field jobs.

Write a short runbook for each flow. It should say how to reconnect the account, how to replay missed records from the source, and how to remove duplicates. When the alert fires on a Monday morning, whoever is on hand can fix it in minutes.

Know about a missing record before finance does

The goal of integration monitoring is simple: when a record fails to arrive, you find out the same morning, with the list of IDs to fix. Error alerts tell you a run broke. Reconciliation counts tell you what the business lost, whatever the cause.

If your CRM, accounting and field tools are connected by Zapier, Make or scripts nobody has looked at in a year, we can help you map the flows that matter and put daily reconciliation and alerts on them. See how we approach systems integration, or show us your workflow and we will tell you where we would add the first checks.

Related reading: integration platforms vs custom code, human in the loop automation for who reviews flagged records, and what the Jobber QuickBooks sync drops. See our systems integration services, or tell us which syncs you run and we will show you where a reconciliation count belongs.

Sources

Share this article

Show us the workflow your existing software can't handle.

You don't need a technical specification. Walk us through the process, the systems involved, and where your team is losing time.

Book a Workflow Call →
Systems involved