Random and probably boring stuff from me.

How does a PUT to a swift object server look like.

I have been trying lately to get a better understanding of the Swift code base, and I found the best way to know it was to read it from top to bottom and document it along the way. Here is some of my notes, hopefully more will come.

I am starting with an object PUT when the request is coming from the proxy server. The request in the log-file will look like this :

For large file sync which is over the configuration variable bytes_per_sync it will do a fdatasync(2) and drop the kernel buffer caches (so we are not filling up too much the kernel memory).

if we have a content-length in the client headers that doesn’t match the calculated upload_size we return a 499 Client Disconnected as it means we had a problem somewhere during the upload.

It will bail out if we have a etag in the client headers that doesn’t match the calculated etag.

And now we are starting defining our metadatas that we are going to store with the file :

metadata = { ‘X-Timestamp’: timestamp generated from the proxy_server. ‘Content-Type‘: defined by the user or ‘guessed’ by the proxy server ‘ETag‘: calculated value from the request. ‘Content–Length‘: an fstat(2) on the file to get the proper value of what is stored on the disk. }

It will add to the metadata every headers starting by ‘x-object-meta-‘.

It will add to the metadata the allowed headers to be stored which is defined in the config variable allowed_headers (default: allowed_headers = Content-Disposition, Content-Encoding, X-Delete-At, X-Object-Manifest).

It will write the file using the put method of the DiskFile class, which finalise the writing on the file on disk and renames it from the temp file to the real location:

It will write the metadata using the xattr(1) feature which is stored directly with the file.

If there is a Content-Length with the metada it will drop the kernel cache of that metadata length.

It will get the headers X-Container-{Host,Partition,Device} from the original headers which is defined by the proxy to know on which container server it going to update. Every different PUT will have assigned a different container to each their own.

It will use the async_update method (by self since it’s part of the same class) to make an asynchronous request:

Passing the aforementioned build headers and req.path.

If the request success (between 200 to 300) it will return to the main (PUT) method.

the request didn’t succeed it will create a async_pending file locally in the tmp dir which is going to be picked-up by the replication process to update the container listing when the container is not too busy.