zero gravitySOLUTIONS
← All insightsPRODUCT DATA / DATA CORRECTION GUIDE

Late-arriving events in product metrics.

QUICK ANSWER

Discover new data by arrival time and attribute behavior by occurrence time. Recompute affected reporting periods from canonical events, distinguish duplicates from corrections, and publish an explicit policy for provisional and older metrics.

Inside this article 5 sections

Late-arriving product events should update the period in which the activity occurred while retaining the time the pipeline received them. Define how long reports remain provisional, which periods can be recomputed automatically, and how older corrections are reviewed.

A daily import that selects only yesterday's occurrence time will miss events delivered today about last week. A pipeline that groups by arrival time will capture them, but it will report the wrong day for customer behavior.

Keep two clocks and one event key

Store occurred_at for the business action and received_at for arrival at a defined ingestion boundary. Keep both in a consistent time representation, then derive the reporting day using the agreed timezone. A stable event ID lets retries be recognized without discarding legitimate repeated customer actions.

Occurrence time may come from an offline device, a server, or an external provider. Record its origin and validate implausible values. A device clock set far into the future can place activity outside the correction window, while replacing it silently with arrival time hides the data-quality problem.

Original SQL fixture · UTC reporting days
Event IDOccurred dayReceived atIngest sequence
e12026-10-302026-10-30 10:00:001
e32026-10-312026-10-31 11:00:002
e22026-10-302026-11-01 09:00:003
e22026-10-302026-11-02 09:00:004
After all four deliveries, October 30 has two distinct events and October 31 has one. The second delivery of e2 changes neither total.

Find new arrivals, then rebuild affected days

Use the ingestion boundary to discover newly arrived records. Identify the occurrence days affected by those records, then recompute each day's metric from the canonical event set. Reading only new arrivals and replacing the daily total with their count would erase earlier activity.

Maintain a durable checkpoint and use an overlap or another mechanism appropriate to the ingestion system's ordering guarantees. A timestamp alone may not uniquely order simultaneous arrivals. Advance the checkpoint only when the corresponding facts and aggregates are safely committed, so a retry cannot leave the pipeline claiming to have processed data it lost.

For immutable events, a deterministic deduplication rule can retain the first accepted delivery. The following query demonstrates the fixture's counting rule over a table named event_fixture. Production ingestion must also reject conflicting payloads that reuse an event ID; choosing the first row is not a complete correction protocol.

WITH ranked AS (
  SELECT event_id, occurred_day,
    ROW_NUMBER() OVER (
      PARTITION BY event_id
      ORDER BY received_at, ingest_sequence
    ) AS delivery_rank
  FROM event_fixture
)
SELECT occurred_day, COUNT(*) AS event_count
FROM ranked
WHERE delivery_rank = 1
GROUP BY occurred_day
ORDER BY occurred_day;

Set a correction policy readers can understand

Choose a provisional period based on observed lateness and business needs. An illustrative policy might refresh recent days automatically and route older arrivals to a correction queue. The interval should come from actual delivery patterns and reporting commitments, not a universal rule about how late events ought to be.

Show report freshness and the date through which data is considered settled under that policy. Settled does not mean physically impossible to change. An offline client or historical backfill can still reveal older activity. Decide whether those arrivals revise published history, create an adjustment, or remain excluded from a specific contractual report.

A watermark in a streaming system expresses an estimate of event-time progress; it is not proof that no earlier event can appear. A batch correction window serves a related operational purpose without requiring a streaming framework. Both designs need explicit treatment for data that arrives beyond the normal window.

Separate corrections from repeated delivery

A duplicate repeats the same fact. A correction changes a fact previously believed to be true. If a source changes an account identifier or occurrence time, use a revision identifier, a correction event, or another documented mutation protocol. Otherwise a deduplication rule may discard the correction or accept contradictory history unpredictably.

Recompute all affected partitions when a correction moves an event from one day or account to another. Updating only the new destination leaves the old total inflated. Test deletions and privacy-driven removals through the same dependency chain so derived reports do not retain facts the canonical layer has removed.

In Redshift, stage deduplicated changes before merging into a canonical table, then update affected aggregates consistently. Do not assume a declared key alone guarantees uniqueness during ingestion. Keep the original delivery evidence long enough to explain how the canonical record was chosen, subject to the applicable retention policy.

Verify replay does not change the answer

Run the fixture in arrival order, replay every delivery, and confirm the final daily totals remain two and one. Then add a correction that moves e2 to October 31 and verify both days change. Test a failure between fact storage and aggregate refresh to establish that retry repairs the report.

The PostHog and Redshift reconciliation guide explains cross-destination comparisons, while the Customer.io replay guide keeps data repairs separate from messages. Software implementation can encode these time, identity, and correction rules in the pipeline and the reports it serves.

FROM INSIGHT TO ACTIONCustom software development

Make late data a supported reporting behavior.

We can implement event identity, arrival checkpoints, canonical deduplication, and repeatable aggregate corrections across your pipeline.

Explore software developmentDiscuss your build

Start with your goals.
We’ll work through the next step together.