Failure class

An import that silently loaded 700 of 1041 records

The symptom

A nightly import 'mostly works'. The record count differs between runs and is always short. There is no error and no failed run — nobody noticed for weeks.

What the log shows

GET /v2/contacts?page=7  ← 200  items: 100  next_cursor: "eyJ..."
GET /v2/contacts?page=8  ← 200  items: 0
loop END   total 700 / 1041

The question that separates the causes

Is the shortfall an exact multiple of your page size?

Yes — 700 with a page size of 100The loop terminated early. Either the cursor is not carried between iterations, or the exit condition tests items.length when the API signals completion only through a null cursor.

No — an arbitrary number like 683Records are being dropped downstream by a filter or a type error. Pagination is not your problem; look after the fetch.

Verdict

Exiting on an empty page rather than on an absent cursor. Many APIs return an empty page mid-sequence, and a loop that treats that as 'done' stops early without ever erroring.

The fix

Exit the loop when the cursor is null or absent — never on an empty page, and never on a max-iteration guard alone.
Thread the cursor explicitly through each iteration and log it, so a future short run is diagnosable from the log alone.
Assert the final count against the API's reported total, and fail loudly on mismatch.

Hardening

Check whether the source is sorted by a mutable field. Paginating over data ordered by updated_at while it is changing skips and repeats rows no matter how correct your cursor handling is.

This is the most expensive failure in this family because it does not look like a failure. No error, no alert, no red run — just quietly incomplete data, discovered weeks later by someone downstream.

Still not it?

Send the workflow export and the execution log of one failing run. Within 24 hours you get a written diagnosis: the exact line in your own log that caused it, the fix in your own UI, and a corrected workflow file you import yourself. No account access, no credentials, no call. If the artifacts do not contain the answer, the report says so and it is refunded.

Send an autopsy request →

Or write to hello@workflowautopsy.com.