API Integration & Workflow Automation
Integration monitoring: catch the records that never arrive
Integration monitoring that catches missing records: why Zapier, Make and API syncs fail silently, vendor retry limits, and a checklist with thresholds.
- Published
- Author
- Cass & York Team
- Series
- Cass & York
- Reading time
- 10 min read
The Zap history is green. Every scenario in Make shows a check mark. Then finance asks why three closed deals never became invoices in QuickBooks, and nobody can say when the sync stopped or how many records it dropped.
Integration monitoring is the practice of checking that data moving between your systems arrived, complete and on time. Error alerts cover half of it. The other half is counting: every day, compare how many records the source created with how many landed in the destination, per object, and alert on any gap.
A run log only records runs that happened. When a token expires, a webhook subscription gets removed or a trigger stops firing, there is no run to log, so the dashboard stays green while data goes missing. This guide walks through why integrations break, what the vendors' own documentation says about each failure, and a monitoring checklist with thresholds you can set up this week.
Why integrations break without throwing an error
Silent failures usually come from one of five causes. Each is documented by the vendor, and each can leave your automation tool showing a clean history.
Expired OAuth tokens
Every connection between Zapier, Make or a custom script and your CRM or accounting system runs on an OAuth token. Those tokens expire on schedules most teams never write down.
- QuickBooks Online access tokens last one hour and refresh tokens last 100 days before the user has to reconnect, according to Intuit's OAuth 2.0 token guide.
- Intuit has also put a maximum validity of five years on refresh tokens. Tokens with accounting or payments scopes issued from October 2023 start expiring in October 2028, and QuickBooks warns customers in the product 30 days before a connection expires.
- Google issues refresh tokens that expire in 7 days for apps left in Testing status, and a token unused for six months stops working.
- HubSpot states that a 401 response is an unreliable signal that you need a new access token. Your script has to read
expires_inand refresh ahead of time.
The person who connected the account often leaves the company or changes their password. The sync then fails at the first step, every time, until someone reconnects it.
API rate limits
Every API caps how fast you can call it. When a backfill, a month end batch or a busy sales day pushes past the cap, the API returns HTTP 429 and records wait or drop, depending on how the integration handles retries.
- QuickBooks Online throttles at 500 requests per minute per company (realmId) and 10 concurrent requests per app and company.
- HubSpot private apps on Free and Starter get 100 requests per 10 seconds and 250,000 per day. The daily limit is shared by every app in the account and resets at midnight in the account's time zone.
- Salesforce Enterprise Edition allows 100,000 API requests per 24 hours plus 1,000 per Salesforce license, and caps concurrent long running requests at 25 in production.
- Zapier returns 429 errors when instant triggers exceed 20,000 requests every 5 minutes per user.
The shared daily quota is the trap. A new marketing tool connected to HubSpot can burn the quota your CRM sync depends on, and the sync fails the next afternoon with nothing changed on its side.
Webhook delivery gaps
Webhooks push an event to your endpoint when something happens: an order is paid, a deal closes. If your endpoint is down or slow, the sender retries for a while and then gives up.
- Shopify gives your endpoint five seconds to respond, retries up to eight times in a four hour period, and removes the subscription if failures persist. A removed subscription receives nothing until you recreate it.
- Shopify says plainly that webhook delivery isn't always guaranteed and tells developers to run reconciliation jobs that fetch data on a schedule.
- HubSpot retries failed webhook notifications up to 10 times over 24 hours, and counts a response slower than five seconds as a failure.
- Stripe retries for up to three days in live mode with exponential backoff, and makes no promise about event order.
Schema changes
Someone renames a custom field in the CRM, adds a required field in the accounting system, or changes a dropdown value. The integration still runs. It now writes blanks, maps the wrong value or skips records that fail validation. Members of the Zapier community list renamed fields and stopped triggers as common causes of silent failures, right alongside expired auth.
API versions shift too. Stripe builds each event in the API version set on your account when the event occurs, so an account upgrade can change the payload shape your handler expects.
Duplicate retries
Retries fix delivery gaps and create a new problem: the same event arriving twice. Stripe warns that endpoints might receive the same event more than once and tells you to log processed event IDs. Shopify tells you to ignore duplicate deliveries using the X-Shopify-Webhook-Id header.
A slow handler is a common path to a duplicate invoice. Your script takes six seconds to create the invoice in QuickBooks, Shopify marks the delivery as failed at five, and the retry creates a second invoice. Both runs show as successful.
Failure taxonomy at a glance
Failure mode | What the run log shows | What actually happened | The check that catches it |
|---|---|---|---|
Expired OAuth token | Errors at step one, or no runs at all | Every record since expiry is missing | Token expiry tracking plus a daily count |
Rate limit (429) | Some errors, some successes | Records dropped during the busy window | Count of 429s and quota used per day |
Webhook subscription removed | Nothing | Sender stopped sending | Daily check that the subscription exists, plus a count |
Schema change | Green | Blank or wrong fields, skipped records | Null rate on mapped fields, rejected record count |
Duplicate retry | Green, twice | Two invoices or two contacts | Duplicate key count in the destination |
Why a green run log can hide missing data
Automation tools report on runs. A run needs a trigger, and most silent failures remove the trigger. A heartbeat on the last step of a Zap will never fire if the first step never fires, a point made in the Zapier community thread on silent failures.
The tools also switch themselves off. Zapier turns a Zap off when 95% of its runs in the last 7 days have errored, and it stays off until someone turns it back on. In Make, a scenario that starts with an instant trigger is deactivated after the first error. After that, the history shows a quiet week, which looks the same as a slow week.
So integration monitoring needs a second signal that lives outside the automation tool: a count of what the source system says happened.
How long a missed event survives
Retry windows decide how long you have to notice an outage before events are gone for good. Look at the gap between Shopify and Stripe below and compare it with how long your team takes to notice a broken sync on a weekend.
Shopify stops retrying after 4 hours; Stripe keeps going for 72
Automatic retry window for a failed webhook delivery. GitHub does not redeliver failed webhooks automatically. A Friday 6 pm to Monday 6 am weekend outage runs 60 hours.
Hours of automatic retries
Scroll across the graph to see all categories.
- Retry window
- 0 h
View data table
| Category | Retry window |
|---|---|
| GitHub | 0 h |
| Shopify | 4 h |
| HubSpot | 24 h |
| Stripe (live mode) | 72 h |
Stripe figure is live mode. GitHub shows 0 because it has no automatic retries.
Source: Shopify, HubSpot, Stripe and GitHub developer docs, accessed November 3, 2026
A Friday night outage on a Shopify webhook endpoint outlasts the four hour retry window long before Monday. Stripe gives you three days of automatic retries, and its List Events API returns events from the last 30 days with a delivery_success filter so you can replay what failed. GitHub, for comparison, does not redeliver failed webhooks automatically at all. Whatever the window, a daily reconciliation that pulls from the source API recovers what retries miss.
The two layers of integration monitoring
Good integration monitoring watches two layers. The technical layer tells you a run failed. The outcome layer tells you a record is missing, which is the thing the business cares about.
Layer | Signal | Example | Catches |
|---|---|---|---|
Technical | Errored runs | Zap history, Make incomplete executions, Lambda errors | Failures inside a run |
Technical | HTTP 429 and 401 counts | Script logs, API usage dashboards | Rate limits, dead tokens |
Technical | Heartbeat or dead man's switch | A ping on each successful run to Healthchecks.io or Cronitor | A job that stopped running |
Outcome | Reconciliation count | Paid Shopify orders yesterday vs QuickBooks invoices created from them | Missing records from any cause |
Outcome | Freshness | Time since the last record landed in the destination | Stalled triggers |
Outcome | Field quality | Share of synced records with a blank mapped field | Schema drift |
The outcome layer catches all five failure modes, because it measures the result: the records that arrived. It works the same whether the integration runs on Zapier, Make or your own code.
How to set up reconciliation counts
A reconciliation count asks one question per object per day: did every record the source created show up in the destination? Here is how we build one.
- List the flows that touch money or customers. Orders to invoices, closed deals to projects, completed jobs to billing. Start with three to five.
- Pick the shared key. The destination record needs the source ID stored on it, such as the Shopify order ID in a QuickBooks invoice memo or a custom field. Without a shared key you can count, but you cannot say which records are missing.
- Fix the window. Count records created yesterday, midnight to midnight in one time zone. Run the check early enough in the morning that someone can fix gaps before the day's work depends on them.
- Compare counts, then IDs. If the counts match, log it and move on. If they differ, list the source IDs absent from the destination and the destination IDs that appear twice.
- Store every result. A table with date, flow, source count, destination count, missing and duplicate gives you a history. The history is how you set fair thresholds and prove the sync is healthy to finance.
- Alert the process owner. Send the gap list to the person who owns the outcome, such as the controller for invoices, with a link to the source records.
On a high-volume data pipeline I ran for a state agency before founding Cass & York, encrypted files moved securely into AWS for batch processing. For batch pipelines like that one, we design the check around the file itself: did it arrive by its deadline, did it decrypt, and does the number of rows loaded match what the file contains? A job that finishes on time with fewer rows than it received is a failure, and the count is how you see it.
Integration monitoring checklist with thresholds
These are the starting thresholds we design for. Treat them as first settings and tune them after four weeks of stored results.
Check | Starting threshold | Why |
|---|---|---|
Reconciliation count, financial records (invoices, payments) | Alert on any gap of 1 or more | One missing invoice is lost revenue |
Reconciliation count, CRM records (contacts, deals) | Alert above 1% or 5 records, whichever is larger | Small timing differences are normal |
Duplicate keys in destination | Alert on any duplicate in financial objects | Catches retry duplicates |
Freshness | Alert when the last record is older than 2x the normal gap during business hours | Catches stalled triggers within hours |
Volume drop | Alert when a day's count falls below 50% of the same weekday average over the prior 4 weeks | Catches a trigger that stopped without errors |
OAuth token age | Alert 30 days before a known expiry; check QuickBooks connections against the 100 day refresh window | Gives time to reconnect |
HTTP 401s | Alert on the first one per connection | A dead token fails every record after it |
HTTP 429s | Alert when 429s exceed 1% of calls in an hour, or daily quota use passes 70% | Leaves room before the hard cap |
Error share of requests | Keep below 5% of daily requests | HubSpot sets 5% as the bar for marketplace apps |
Webhook subscriptions | Daily check that each expected subscription exists | Shopify removes subscriptions after persistent failures |
Automation tool state | Daily check that every Zap and scenario is on | Zapier and Make switch themselves off |
Blank mapped fields | Alert when the null rate on a required field rises 5 points above its 4 week average | Catches renamed or removed fields |
Alert path test | Break a test flow on purpose once a month | Proves the alerts still reach a person |
The volume and freshness checks need a baseline that respects weekends, holidays and business hours. Comparing Tuesday with the average of the last four Tuesdays avoids most false alarms.
What production integrations taught us
During our Shopify engineering engagement we worked on event integrations with Salesforce and on production maintenance. Event integrations are where retries, duplicates and order matter most, because Stripe and Shopify both document that events can arrive late, twice or out of order. Our rule since then: every handler stores the event ID, returns a 2xx fast, does the work from a queue, and a scheduled job pulls from the source API to fill anything the webhooks missed.
Before Cass & York, I also built communications software for the elevator industry on Node.js, TypeScript and AWS Lambda. In a system like that, the failure that matters is silence. We carry that into every integration: each one gets a check that expects a signal on a schedule and raises an alarm when the signal stops.
Who should own integration alerts
An alert with no owner gets muted within a month. Give every integration one named owner and route the outcome alerts to the person who feels the gap: finance for invoices, sales ops for CRM records, operations for field jobs.
Write a short runbook for each flow. It should say how to reconnect the account, how to replay missed records from the source, and how to remove duplicates. When the alert fires on a Monday morning, whoever is on hand can fix it in minutes.
Know about a missing record before finance does
The goal of integration monitoring is simple: when a record fails to arrive, you find out the same morning, with the list of IDs to fix. Error alerts tell you a run broke. Reconciliation counts tell you what the business lost, whatever the cause.
If your CRM, accounting and field tools are connected by Zapier, Make or scripts nobody has looked at in a year, we can help you map the flows that matter and put daily reconciliation and alerts on them. See how we approach systems integration, or show us your workflow and we will tell you where we would add the first checks.
Related reading: integration platforms vs custom code, human in the loop automation for who reviews flagged records, and what the Jobber QuickBooks sync drops. See our systems integration services, or tell us which syncs you run and we will show you where a reconciliation count belongs.
Sources
- Troubleshooting webhooks, Shopify, accessed November 3, 2026
- Webhook best practices, Shopify, accessed November 3, 2026
- Receive Stripe events in your webhook endpoint, Stripe, accessed November 3, 2026
- Manage webhook endpoints, Stripe, accessed November 3, 2026
- Webhooks API guide, HubSpot, accessed November 3, 2026
- API usage guidelines and limits, HubSpot, accessed November 3, 2026
- API request limits and allocations, Salesforce, accessed November 3, 2026
- QuickBooks Online API throttling changes coming soon, Intuit Developer Blog, August 8, 2016
- Important changes to refresh token policy, Intuit Developer Blog, accessed November 3, 2026
- OAuth 2.0 tokens, Intuit Developer on GitHub, accessed November 3, 2026
- Using OAuth 2.0 to access Google APIs, Google for Developers, accessed November 3, 2026
- Error handling, Zapier Developer Platform, accessed November 3, 2026
- Why was my Zap turned off, Zapier Help Center, accessed November 3, 2026
- Best practice for monitoring client zaps for silent failures, Zapier Community, accessed November 3, 2026
- Scenario settings, Make Help Center, accessed November 3, 2026
- Handling failed webhook deliveries, GitHub Docs, accessed November 3, 2026
Share this article