Synthetic polling trace: five strategies over the same five records

The fixture

Five invented records carry an updated time, in minutes, and a visible time, the minute from which a list call can first see them. Record R3 is updated at minute 9 but is visible only from minute 12, which models a delay before a new record appears. The job runs at minutes 10, 20 and 30 and asks for records updated after its starting point. In the two crash cases the job is restarted at once, still in the minute-20 run, and the minute-30 run follows. Everything here is synthetic, and this simulation was run on 11 October 2026; it is not a trace of any real API.

If the matrix is wider than the box, scroll horizontally to read every column. Keyboard: focus the matrix and use Left/Right.

id  updated  visible-from
R1  3        3
R2  8        8
R3  9        12      (slow to appear)
R4  14       14
R5  18       18
Runs at minutes 10, 20, 30. Where used, the overlap is 5 minutes.

Five strategies, same records

Strategy A asks for everything since the last run and moves its checkpoint to the run time. It never sees R3, because at minute 10 R3 was not yet visible and at minute 20 it asks only for records updated after 10. Strategy B adds a five-minute overlap and catches R3, but now reads R2 and R5 twice and acts on them twice. Strategy C adds a record of handled keys, so the overlap causes no repeats. D and E repeat C and B with a crash after R3 at the minute-20 run, before the checkpoint is saved.

If the matrix is wider than the box, scroll horizontally to read every column. Keyboard: focus the matrix and use Left/Right.

A  since last run, no keys
   10: found R1,R2        processed R1,R2        checkpoint 10
   20: found R4,R5        processed R4,R5        checkpoint 20
   30: found -            processed -            checkpoint 30
   missed R3 | repeated none | 4 actions

B  overlap 5, no keys
   10: found R1,R2        processed R1,R2
   20: found R2,R3,R4,R5  processed R2,R3,R4,R5
   30: found R5           processed R5
   missed none | repeated R2 x2, R5 x2 | 7 actions

C  overlap 5, handled keys
   10: found R1,R2        processed R1,R2
   20: found R2,R3,R4,R5  processed R3,R4,R5
   30: found R5           processed -
   missed none | repeated none | 5 actions

D  as C, crash after R3 at minute 20
   missed none | repeated none | 5 actions
E  as B (no keys), same crash
   missed none | repeated R2 x3, R3 x2, R5 x2 | 9 actions

What each result teaches

A shows that a single timestamp can silently lose a record, with no error anywhere. B shows that fixing the loss with an overlap alone trades it for repeats. C shows the pairing that works: overlap to catch the late record, keys to stop the repeats. D shows that the pairing also survives a crash, because the checkpoint stayed behind and the keys prevented repeats on the rerun. E shows the cost of skipping the keys: nine actions for five records. None of this depends on the made-up numbers; it depends on the structure.

• Overlap protects against missing records.
• Handled keys protect against repeats.
• Advancing the checkpoint only after the work is recorded protects against loss on a crash.

What the example does not show

It uses a single process and an in-memory set of keys. A real job needs the keys and the checkpoint in durable storage, keys the API guarantees are unique, complete paging, an overlap longer than the real delay, and handling for changed records, not only new ones. It does not model rate limits, result caps or pages. The paid outcome builds a job with these pieces against a synthetic stand-in for one documented API, and accepts it on tests like these. It does not promise that the live API behaves as documented.