Secure way to serve protected files with Nginx Sendfile (X-Accel-Redirect)

If you are running some kind of e-commerce store, it’s quite possible that your product images are protected with watermark. At the same time original uploaded images are not available for public access.

All nice and good! But what if some of your partners request access to original images without watermarks?

You can’t just create new url ( something like /images/partner_hidden_1234/i/image.jpg ) and give it to the partner, as it can potentially (and most likely will) leak out.

The other downside is you have no “metering”, meaning you have no idea who access those new urls and how often. Some people may say you have access logs and you may parse those to create stats. Well, yes – you can get some info this way, but only some..

Is there better way? You bet.

Meet Nginx X-Accel-Redirect

First we need to define new url for those original images without watermarks. Let say it will look like this:/original_imgs/awesome_pic.jpg (note that your actual images may be stored in totally different location on your server, or could even be served from remote locations like S3. This is purely virtual path.)

Then we generate keys for every partner that require access to images. You partners will need to provide key in two ways, through GET param or direct embed into URI (you can think of more ways if you want, header etc):/original_imgs/awesome_pic.jpg?key=ab234ab or /original_imgs/ab234ab/awesome_pic.jpg

Next in Nginx we create new location block, where we rewrite urls to /original_imgs to some back end script:

Inside of our meter.php script we check if the key is correct, store access info (in Redis for example), throw statsd (or Datadog) metric and if everything looks good – return X-Accel-Redirect header with original image location.Here is some pseudocode:

Please note that Content-type header might be necessary, otherwise Nginx may guess your Content-type incorrectly. The other option is to remove Content-type from response headers and leave it to Nginx to decide.

Final step is to create internal location in nginx that will be serving protected files ( /unprotected_originals in our example ) to the clients.

location ~ /unprotected_originals {
internal;
}

It’s important to note here, that this example is super simple. In reality /unprotected_originals location may use aliases or event point to another server/service.

The main point here is that when someone tries to access /unprotected_originals directly, they will get 404 error from Nginx because our location is internal.

Conclusion

At this point our original files should be protected from unauthorized access and we have full control and visibility over access patterns, which is VERY important.

If you have some big files, you can apply Sendfile approach to offload file serving to Nginx, which is optimized for the task and release backend resources. Another idea is to provide some temp urls for paid downloads.

As always let me know if you have any questions/suggestions in the comments section below!

I helped to build and maintain the infrastructure for Game of Thrones, the biggest and most popular show in the world.

Do you want to know the single most important thing that I learned over the years?

NONE OF IT REALLY MATTERED…

Yes, it was fun for a while.

Yes, like most of us engineers I was making good money.

But at the end of the day, I would still have to show up at work and sell my time.

Sometimes I would come in, sit in my cubicle and dream about things I could do instead of staring at the screen all day long…

I could go to the beach with my wife and my son.

I could fly to El Classico game in Barcelona with my brother and watch Messi scoring amazing goals.

I could organize a surfing trip to South Africa and other awesome places around the world. Places I’ve never seen.

I could work on my own projects that would make the impact in the world or at the very least, make me some money.

Hell, I could just sit home and do absolutely nothing!

And yet there I was still in my cubicle 12 years later with big hopes and dreams and pretty much nothing to show for…

Sounds familiar?

The tipping point for me was when I started buying games on Steam and GoG and playing them in my mind.

Nothing to install, no need to upgrade video cards, no need to feel bad in front of my wife, no time to waste…

You are right, I was spiraling down and needed a break, but more so I felt like I needed some radical changes in my life.

I’m sure you heard this saying before: “Insanity: doing the same thing over and over again and expecting different results”

It became clear that the road I was walking on would lead me to mediocre life.

The problem was that I didn’t want to be mediocre. I wanted my life to be awesome, full of fun, happiness and excitement!

I wanted to make a difference in the world, leave a legacy, make my kids proud, live without regrets, discover my true purpose.

So about a year ago, I set out on my new journey…

I left my old comfortable job, attended multiple high profile non-technical events (including Tony Robbins UPW), joined an expensive business program, hired a personal coach and mentor, met a bunch of people who were able to disconnect from the Matrix and never looked back.

And let me tell you – there is another world out there, something we technical guys don’t get to experience!

There is hope.

Now, here is my question for you:

Do you want to continue to be just a tool in someone else’s hands or you want to upgrade yourself and become a Rain Maker?

If you want to find out who you really are, take full control of your life, step outside your comfort zone in order to grow physically, mentally and financially and help others along the way, then the Red pill is for you. Just drop your email in the field below and we’ll be in touch.

Take a Blue pill and you will forget that we ever met. You will close this popup and continue reading articles about Nginx, Kubernetes, Docker, secretly dreaming of life that you could have… (or pathetically thinking that you will have it one day just by perfecting technical skills)