Object storage SDKs (S3, Azure Blob, GCS)

Object storage (Amazon S3, Azure Blob Storage, Google Cloud Storage) keeps files as objects in buckets, addressed by a key. The SDK lets your code upload, download, list and sign access to them over HTTP.

A mental model

A huge coat check. You hand over an item and get a ticket (the key); you cannot edit the coat in place, only replace it with a new one or ask for it back.

How it works

You create a client once (S3Client.builder().region(Region.EU_WEST_1).build()) and reuse it: clients are thread safe and hold HTTP connections.

putObject(PutObjectRequest.builder().bucket(b).key(k).build(), RequestBody.fromFile(path)) uploads; getObject returns a stream you must read and close. listObjectsV2 returns pages of keys.

Large files use multipart upload (the transfer manager does it automatically) so parts upload in parallel and a failed part is retried alone.

A presigned URL grants temporary access to one object without sharing credentials, so browsers upload and download directly instead of through your server.

Going deeper

There are no real folders. A key such as invoices/2026/a.pdf is just a name; listing "a folder" is a prefix query. Renaming means copy then delete, and it is not atomic.

Always close the response stream of getObject. An unclosed stream holds a pooled HTTP connection and eventually starves the client.

Read-after-write is strongly consistent in S3 today, but overwriting keeps no history unless versioning is on. Set lifecycle rules for expiry and cheaper storage classes, or costs grow forever.

Under the hood

Every call is a signed HTTPS request (Signature V4: a hash of the request combined with a key derived from the secret and the date). The SDK retries throttling and 5xx errors with exponential backoff and jitter, and the async client uses non-blocking Netty I/O.

Common mistakes

• Creating a new client per request.
• Not closing the response stream.
• Loading big files into memory instead of streaming.
• Public buckets or long-lived presigned URLs.