S3 uploads that remain correct after retries
A presigned URL authorizes a storage operation; it does not complete the application’s upload workflow. Persist upload intent, assign the object key on the server, verify the exact uploaded object or version, and make finalization and processing repeat-safe. Treat notifications as duplicate-prone signals rather than proof that the content is approved.
Inside this article 6 sections +
A browser reporting “upload complete” means it received a successful storage response. It does not prove that the application has associated the file with the correct tenant, verified its content, or started processing exactly one intended version. Those are separate durable decisions.
Consider a supplier-catalog upload. The client requests permission, uploads directly to S3, and tells the API to start an import. The network can fail at any step, the client can retry, and S3 notifications can arrive more than once. Design the lifecycle around a stable upload intent rather than a sequence of optimistic browser callbacks.
Create an upload intent before signing access
Authenticate the caller and verify access to the target account. Persist an upload record containing a server-generated identifier, tenant, intended purpose, expected constraints, and expiry. Choose the object key on the server so a client cannot request a URL for another customer's known key.
Use a unique key for each intended upload generation. Do not use the original filename as the only identity. Preserve the filename as display metadata after validation, while storage identity remains opaque and scoped. If the user replaces a file, create a new generation or intent instead of silently reusing an accepted object's identity.
- Intent createdThe tenant, purpose, key, and expiry are recorded before the URL is issued.
- Object observedThe server identifies the uploaded object and its exact version where versioning is used.
- Content verifiedSize, integrity, format, and required scanning checks produce an explicit result.
- Processing admittedA guarded transition schedules work for that verified object identity.
- Outcome recordedCompletion, rejection, expiry, or unresolved processing remains inspectable.
Understand what the URL authorizes
A presigned URL is a bearer capability for the signed method, key, and conditions during its validity. Keep it out of logs and public analytics. Choose a lifetime appropriate to the upload and recognize that temporary signing credentials can expire before the requested URL lifetime.
The URL is not inherently single-use. Repeating a PUT to the same key can replace the current object. If the application verifies a key and later reads “whatever is current,” a subsequent upload can change the bytes between verification and processing.
Close that gap explicitly. One design enables versioning and binds finalization to the verified version ID, then makes workers read that version. Another uses a supported enforced conditional-write policy to prevent replacement of an accepted key. The chosen mechanism must apply at the storage boundary; a browser promise not to retry is insufficient.
Verify the exact object before accepting it
On finalization, load the upload record under the authenticated tenant and inspect the object through server credentials scoped to the intended storage area. Verify key, version, size, and the integrity metadata required by the protocol. Do not trust a client-supplied object URL as proof of ownership.
Illustrative accepted-upload identity:
upload_id: upload-203
scope: tenant-19 / supplier-catalog
object_key: server-assigned opaque key
object_version: verified immutable version identifier
checksum: verified checksum under the chosen upload protocol
verification_state: accepted
processing_operation: import-203
A checksum establishes integrity relative to an expected value; it does not establish that a file is safe or semantically valid. Content type supplied by the client is also only a claim. Apply format parsing, size and expansion limits, and any required malware checks before exposing or processing the file.
Do not assume an ETag is always a simple content MD5. Multipart uploads and encryption can change its interpretation. Use the supported checksum mechanism for the chosen upload path and record what was actually verified. The worker should reject a mismatch instead of quietly switching to a newer object version.
Make finalization and notification handling converge
The browser completion call and an S3 notification may both discover the object. Route both through the same guarded application transition. If the upload is already accepted, return or observe the existing result. If it is expired, rejected, or belongs to another generation, do not reopen it merely because another event arrived.
S3 notifications can be duplicated and arrive out of order. Deduplicate by the relevant object and business identity, and compare the event with the upload record before acting. A notification for an older version must not overwrite acceptance of a newer explicitly selected generation.
Persist processing intent with the acceptance transition where possible, then dispatch through a durable mechanism. A crash after acceptance but before enqueue should not strand the file. A repeated dispatch should reference the same processing operation so it cannot create two imports from one accepted upload.
Bound abandoned uploads and incomplete work
Expire intents that never produce an eligible object, and remove orphaned objects according to a deliberate retention policy. For multipart uploads, configure and test cleanup of incomplete uploads as well as completed objects. A browser tab closing does not notify every supporting resource reliably.
Keep cleanup aware of accepted and running work. Deleting an object because its original upload URL expired can break a valid import that starts later. Storage retention should follow the application's processing and recovery requirements, not the short lifetime of the upload credential.
Separate rejected content from accepted content through access policy and lifecycle. An unverified object should not become publicly downloadable simply because it exists in the bucket. Record the rejection reason without retaining unnecessary sensitive file contents in logs.
Exercise the races, not only the upload
Test a lost PUT response, repeated finalization, duplicate notification, delayed old-version notification, replacement attempt after verification, and a crash before processing dispatch. Assert the exact bytes or version processed and the number of business imports created.
The resumable import guide develops processing checkpoints, and idempotency design establishes repeated intent. S3 integration implementation should connect storage permissions to a complete, explainable application lifecycle.