Getting Started with Nginx & Drupal 7

This started as a discussion, but I figured we could add to it and collaborate a bit on it as a wiki instead.

So you want to use Nginx with Drupal. And you have no idea how to get started. Hopefully this’ll help.

Our lofty goal is to get Perusio’s Nginx config up and running. Hopefully in the process we'll understand more about how nginx works and how we can debug things.

Unfortunately I don’t understand all of it, so there’s going to be a few understanding holes here and there, but we’ll still be able to get it running.

There are plenty other ways to set-up Nginx, PHP and Drupal, this is just how I’ve done it.

I’d love to have people add to this. Comment things I’ve missed or got wrong and I’ll update and add them in.

What skill level is this guide for?

Someone with very little experience in nginx. (i.e. me).

I’ll try and explain as much as possible, but if you’re completely unfamiliar with the Linux server administration then you might need to brush up a little. A good starting point? Check out the linode library. Work through theirLEMP guides to get comfortable then come back and switch out their nginx (which isn't secure) for the one we'll go through here.

What this doesn’t cover?

PHP-FPM debugging. I’m not very good at this. If anyone more experienced would like to write a guide with me on this, I’d be super happy to do this because it’d be really useful for me and anyone else just starting out.

Microcaching. I haven’t tried it so the chances of me getting it right without using it are minimal.

Boost. Again I haven’t used it, although it’s probably easier to set-up than microcaching.

UNIX sockets. Same again, I’ve not used them.

Why Nginx?

There’s a great thread here on Drupal answers which there’s no point re-creating.

Because I had virtually no experience in either, I decided to pick Nginx mainly because it allows you to do more with less and I have a poor mans VPS with low RAM to run my sites.

Let’s Get Started: Installing Nginx

Ubuntu/Debian

We could use the following command to install it:

sudo apt-get install nginx

But the version in the Ubuntu repository is going to be old (probably 1.1.19) and we want brand new Nginx.

Useful Command:sudo nginx –V

This will show tell you what version of nginx you’re currently running and all the modules that you've installed with it.

So instead let’s use the official repositories. We can use either the stable, or the main-line repository. Either is fine.

Warning: I’ve had problems getting mixed mode SSL set-up, the redirects caused a number of problems. This problem was solved (thanks to some consulting from Perusio) and I know the solution I’ll use works for the mainline version (1.5.7 as of this time).

It may also work for the stable version 1.4.7 but I haven’t tested it.

Instructions for getting the repositories can be found here. I’ve given an example below:

First we add a public key to authenticate the repository, this means we know (and Linux knows) it’s genuine. Download the key from here to your server and then run the following command:

Then you’ll need to update your distributions and then you can install nginx. Finally you check you’ve got the right version.

sudo apt-get updatesudo apt-get install nginxsudo nginx –V

Now going to your IP or servers web address should give an Nginx Welcome page.

If you’ve already got a version of nginx on your server you’ll be asked a bunch of questions about whether or not you want to keep your config files. It doesn’t matter as we’ll be deleting it all anyway.

CentOS/RHEL

NGINX maintains an RPM repository where you can get the latest version of NGINX.

Depdending on whether you have RHEL or CentOS, you'll need to add this to /etc/yum.repos.d/nginx.repo:

Installing (downloading) Perusio’s Config

Before we get to some understanding let’s put the config in place. Then we’ll use that to work through and hopefully get a handle on how Nginx, PHP and Drupal can all work happily together.

Perusio’s config is here. Perusio himself has a set of instructions on installing it and that might be enough. If you’re not quite up the task then don’t worry we’ll run through it all here and the next few sections.

Once you’ve followed this guide go back and re-read all of Perusio’s documentation. It’s extensive and really useful.

First rename your current nginx config and remake the directory:

sudo mv /etc/nginx /etc/nginx_old, sudo mkdir /etc/nginx

Into that we place the Drupal 7 branch of Perusio’s config. You can use git, or if you’re unsure about that select the Drupal 7 branch at the top of the page, download it as a zip and extract it into the new directory you just made.

If you’ve done it right the following path should exist:

/etc/nginx/sites-available/000-default

Now we’ve got the config, we now need to set it up for us.

How to think about Nginx Configs

There are a lot of files in Perusio’s config and it’s a little scary. Especially at my skill level (and probably yours if you’re reading this) debugging is very hard, mostly because we have no idea what is going on.

We’re so un-confident in our knowledge of nginx, we don’t know where to turn or what’s broken when it doesn’t work and our attempts to fix it often can make more problems down the line.

Nginx reads one config file we’ll care about.

/etc/nginx/nginx.conf

Everything else comes from this, it will call all the other config files we care about. Nginx might read other files (e.g. koi-win), but for 90% of the time, this file and two other config files which nginx.conf calls are what we'll focus on. If we need to look at one of the other files, one of these three will tell us.

There are a lot of other config files called by these three files, however we’ll rarely, if ever, touch them, except usually to comment out the line and stop it from being included.

One Config Fits All

This would be lovely, unfortunately it’s not really an option, so what Perusio has done is provided us many options and different versions of files for any possible config we could have.

Remember we downloaded the Drupal 7 branch? Well that’s because he has an entire version of this config for Drupal 6.

So what we’ll do is work through his config and pick the parts that fit our set-up.

In this guide we’ll be installing the standard Drupal 7 version. If you have special characters in your URL’s, like ? or + then you’ll have to use the drupal_escaped versions of all the standard config files I include. You'll see them as we work our way through the config files.

So let’s start at the top of

nginx.conf

This is how we’ll do it. We’ll go through the config line by line. I’ll clarify anything I struggled to understand or that is particularly important.

Perusio’s config is very well documented and if there are instructions which apply to you and I don’t mention go ahead and do them. I probably haven’t mentioned them because he explained it fine.

...Access_log pathError_log path...

Note these paths. If things go badly, we’ll be visiting these guys. Mostly the error log. We'll define more error logs in example.com.conf. They're usually the first stop. But if you can't find any records in there, head to this one.

How could that happen? Perhaps the config in example.com.conf is never hit, in that case this log will tell you why it's being missed. (Perhaps you forgot to symlink sites-enabled for example).

...Client_max_body_size 10m;...

This sets the maximum upload size. Set as you need. If you have post_max_size in your config (it’s not in Perusio’s by default) then that’ll need to be equal or bigger.

...Gzip on;Gzip…;......

A lot of gzip related stats. We don’t really need to touch these, but nice to know where they are.

...ssl_session_cache shared:SSL:30m;ssl_session_timeout 1d;...

The first part says whether or not SSL session can be cached (it can), and how large that cache should be at maximum (30m). The second part says how long someone should be able to use that cache for.

So if I log on as a user over SSL, I’ll be able to re-use that connection for a day, unless something else shuts it down.

...#Map_hash_bucket_size 192;...

We usually need to touch this to deal with a particular error when trying to start:

...nginx: [emerg]: could not build the map_hash, you should increase map_hash_bucket_size...

I’ve solved this error, not by un-commenting the line above but by adding the following line:

...variables_hash_max_size 1024;...

I think the error is something to do with variable names being too long when hashed and so we increase the maximum size for hashed variables.

...upload_progress uploads 1m;...

If we want to see upload progress this needs to be un-commented. I’m unsure if the size reserved for tracking uploads has to be as large as the maximum upload.

This line blocks click-jacking attacks. The top is used if you’re using frames, the bottom if you’re not. Although I don’t have frames, I’ve found using the bottom line blocks my ability to upload files with Drupal, so I’m using the top one.

So Nginx has to talk to PHP-FPM (remember they're separate) and to do that it uses sockets, either TCP or UNIX.

We’ll be going with TCP because I managed to get that one working first… although I’m not sure it makes much difference.

So comment out the unix.conf and un-comment the tcp.conf

...include php_fpm_status_allowed_hosts.conf;...

Un-comment this, it’ll give us a good way to check on PHP-FPM.

...include apps/drupal/cron_allowed_hosts.conf;...

Perusio’s config is very secure, because it blocks all execution of PHP unless it’s happening from index.php. This means you can’t run cron using drupals inbuilt cron or a crontab.

If you’re using Drush for cron (which is great) then this is no problem, if you’re not un-comment this and in cron_allowed_hosts.conf choose the IP’s you’d like to be able to run cron.

You need the ngx_http_geo_module for this which isn’t installed with the standard install. You’ll need to remove nginx and install nginx-extras with sudo apt-get install nginx-extras unless you compile if from source.

...include map_cache.conf;...

Because I’m using Varnish rather than micro-caching (see below), I leave this un-commented. I’m not sure if this is necessary though.

I commented out these lines because I’m not using micro-caching I’m actually using Varnish.

I’m pretty sure microcaching with Nginx is a very similar concept and has similar results. I was already slightly comfortable with Varnish and I didn’t use it because I was already having trouble getting nginx to work.

Hopefully after reading this nginx is working for you however and so I would definitely give it a go, the less pieces you have, the less can go wrong.

Great we’re done with nginx.conf. Time to move on:

example.com.conf

Remember as a webserver nginx’s primary job is make sure requests go where they’re supposed to.

To do that it defines blocks of certain levels. So pretty much everything goes through the HTTP block.

The Server block is then slightly more specific, only effecting certain addresses (usually a website). It doesn’t as the name might imply effect the whole server, you could have multiple server blocks (for multiple websites) on a single server. The location blocks, only effect a certain location in a website.

This is the port that this server block will listen on. If your server supports IPv6 then fill in the IPv6 line otherwise leave it out.

...server name example.com;...

This defines a web address for this server. So a request through port 80 to address example.com will come into this server block.

...access_log patherror_log path...

Note these paths. When things go wrong we’ll be coming back to these. These are the more specific log files I mentioned earlier.

...root /var/www/mysiteindex index.php...

This is probably the most important. Root is the location of the root of your drupal website.
Index is the first file to run. For Drupal it’ll always be index.php.

...include apps/drupal/drupal.conf;...

This is the inclusion of the last important config file we’ll examine.

...include apps/drupal/drupal_cron_update.conf;...

If you use update.php externally (i.e. you go to the address in the browser bar rather than using drush, then uncomment this). Perusio's config blocks pretty much all php execution which isn't through index.php.

...include apps/drupal/drupal_install.conf;...

If you’re going to install Drupal onto this server (i.e. not migrate it) then you’ll need to un-comment this.

...include php_fpm_status_vhost.conf;...

We’re using php-fpm so let’s uncomment this so we can get some more diagnostic information on it later.

...Return (no rewrite) server block. server { .... }...

Again skip this whole server chunk for the moment. We’ll come back to it.

...Server { #listen 443 ssl listen 443 ssl spdy;...

So we set up port 80. But if you want HTTPS you’re going to need to define a port for it and that means creating a whole new server block. That’s what we’re doing here.

We’re also enabling spdy, Googles protocol for speeding up HTTPS. This module comes bundled with later versions of Nginx we’ve installed.

....ssl_certificate /etc/ssl/…ssl_certificate_key /etc/ssl/…...

Fairly self-explanatory. Here we put the paths for our ssl certificate and key. Nginx will tell us if we have these wrong.

Everything else is identical to the other server block.

Return (no rewrite) server block

These blocks are for re-directing sub-domains to the correct address. The two crucial lines are:

Here we’re taking all requests to www.example.com and re-directing them to example.com, because www is a subdomain. Having both will create duplicate content, which Google is not a fan of.

...Return 301 $scheme://example.com$request_uri;...

This is the line that does the re-directs. 301 is the re-direct code and then we’re providing the correct URL.

$request_uri means that if we go to www.example.com/blog we’ll be redirected to example.com/blog not just example.com.

I’ve had problems getting these blocks to work on local sites i.e. my virtual box. I think this is because I’m forced to put an IP address in the above line and this creates a re-direct loop. But it works great on my actual site. So if you’re on a local server comment these two blocks out.

Drupal.conf

Time for the last large config file:

location / { ... }

This block will catch everything hitting the server block, although it’s directives will be run after the server blocks ones. If you scroll down to the end of this block you’ll spot two lines:

I’m not quite sure why, here’s my guess: all URL’s hit this directive. I’m presuming they find nothing if you have clean URL’s enabled and in that case the URI is passed to @drupal and that doesn’t contain the “?q=” which then returns a file.

Anyway back to the top of the location / { block.

...Location ^~ /private { internal;}...

This section says that any folder named private, will only be accessible by the server. I.e. it’s what we use to define our private files folder in Drupal.

I mention these two blocks because they caused me a lot of pain. These location blocks are to integrate with the AdvAgg module. You’ll need to change them I think if you’re running AdvAgg 7.x 2.2 or later (if it hasn’t been updated in Perusio’s config yet) to the following:

It means that the location @drupal (if you remember we pass all requests through here with the try_files directive above), has permission to execute PHP. But it limits it only to requests passing through index.php.

If you’re using any of the modules or options it describes you’ll need to un-comment various files.

And that’s pretty much the end of the three most important config files we need to touch.

Now the sharp eyed amongst you (or anyone who used nginx before) might notice we’ve only changed /etc/nginx/sites-available/example.com.conf

And nginx.conf calls from the path:

include /etc/nginx/sites-enabled/*;

So we need to create sites-enabled and symlink our config from sites-available.

PHP-FPM Set-Up

Again Perusio has a default config for this so we’ll do the following:

sudo mv /etc/php5/fpm /etc/php5/fpm_old

We’ll then copy in Perusio’s default config for FPM.

Then we need to create the sym-link so PHP pulls in all the various config files:

sudo ln –s /etc/php5/conf.d /etc/php5/fpm/conf.d

Unfortunately my understanding of PHP-FPM is not great, so this section is going to be little light on explanations or debugging skill.

Basically what we’ve done, is we’ve said to PHP-FPM, we’re going to define three pools that you can send PHP requests from Drupal to.

When a request arrives a child is spawned which deals with it (like some sort of miniature slave labour camp). When the process is finished the child stays alive and is ready to take another request.

Php-fpm.conf defines some basic rules for your pools, the most important thing to note is:

error_log = /var/log/php5-fpm.log

There’s a high chance we’ll be needing this log.

It includes the pool config files with the following command:

include=/etc/php5/fpm/pool.d/*.conf

php.ini is the rules that PHP-FPM will run php with. (I.e. it’s not the rules effecting the pools but the rules effecting the actual php code itself. The pool configs are extra rules atop that.)

We then need to configure our pools. By default 0 and 1 are normal. 2 is a backup.

So let’s open /etc/php5/fpm/pool.d/www0.conf and look at the important parts:

...listen = 127.0.0.1:9001...

Remember the whole TCP UNIX thing? Well this is defining a TCP socket to listen on.

...user = www-datagroup = www-data...

This sets which user and group will run the PHP processes.

The next set of options are the most important ones and the ones which will cause PHP-FPM to fail or not. When I say fail, it don’t mean all the time. It’s unlikely to break completely. But you might start getting 502’s or other weird errors.

Unfortunately getting these numbers right depends completely on your own environment. On Perusio’s page he has an estimation on how you should set them up but the best way to get this right is using the debug logs to sort out your problems and tune them.

Another unfortunately. I’m not really sure how to do this. I would love to know, I would love to write a guide about it, please message or comment if you do and lets put something together for the community.

Then we can note that our two php streams are named phpcgi and phpcgi_backup.

Open up /etc/nginx/php_fpm_status_vhost.conf.

In order to get the config to work I had to change

fastcgi_pass www0 to fastcgi_pass phpcgi, comment out the chunks involving www1 and change phpcgi to phpcgi_backup in the final chunk.

I.e. we’re naming our upstreams as opposed to the pools.

I’m not sure if I’ve done this correctly, but if you’re running into errors in php_fpm_status_vhost.conf then this is what it is.

Now we can test our config using:

sudo nginx –tsudo php5-fpm -t

And hopefully we get no errors. We can then restart and head to our webpage. Congratulations you’ve successfully got nginx up and running! Hopefully quickly as well. If you have errors then head to the debugging section:

Debugging

I’ve run into loads of errors trying to set-up Nginx. So many errors. I’ll list them and the various things I’ve found caused them, they’re in no particular order:

Remember after changing a lot of these you’ll need to restart nginx and or php-fpm:

sudo service nginx restartsudo service php5-fpm restart

Fatal error: Class 'PDO' not found in ...includes/database/database.inc on line 184

OR

GD and PDO missing when you try to install Drupal, even though you’ve installed them.

It usually means you’ve created the /etc/php5/fpm folder but have forgotten to create the symlink to include all the other php config files that php needs.

Head to etc/php5/fpm then create it:

sudo ln –s ../conf.d conf.d

Images Broken

Can you only see small crosses where your pictures should be even though you’re website is running?

I got this error because my PHP FastCGI was misconfigured.

Open /etc/nginx/fastcgi_params and make sure the SCRIPT_FILENAME line looks like below:

#add_header X-Frame-Options SAMEORIGIN;## For sites not using frames uncomment the line below. add_header X-Frame-Options DENY;

Comment out DENY and uncomment SAMEORIGIN.

add_header X-Frame-Options SAMEORIGIN;## For sites not using frames uncomment the line below.#add_header X-Frame-Options DENY;

I’m not entirely satisfied with this, because I didn’t think Drupal uploads used frames so it shouldn’t affect anything. But perhaps the upload window counts a frame? Could it be something to do with the headers varnish adds? I’m not really sure.

[emerg]: could not build the map_hash, you should increase map_hash_bucket_size

Add the following line to your nginx.conf file. To keep it neat put it near the other hash line which is commented out. I talk about this earlier.

variables_hash_max_size 1024;

nginx: [emerg] no port in upstream "www0" in etc/nginx/php_fpm_status_vhost.conf:16

I got this error because I’d not changed the www0 , www1 and phpcgi in /etc/nginx/php_fpm_status_vhost.conf.

I also commented out the chunks with www1 in and changed phpcgi to phpcgi_backup. So the fastcgi_pass #values# corresponded with the upstreams defined in: /etc/nginx/upstream_phpcgi_tcp.conf.

File not found

Not a 404 error. But file not found in small words in the top left of the browser. This error means that PHP FastCGI is running correctly but it’s paths are wrong.

I managed this because I’d set the chroot in my php pool and PHP FastCGI was navigating to the root, once nginx had already sent it to the root,. This mean the full path to my page that php was searching for looked something like : /var/www/mysite/var/www/mysite/mypage.

Looking into your nginx and PHP logs, you should be able to find some evidence of this.

Run through the section on setting up PHP-FPM above, make sure you’ve got all the variables set as I go through.

UPDATE:

Since trying to re-create my dev environment on a new server I ran into a new cause of this error:

If you're migrating a drupal installation then something in settings.php may break this. I'm not entirely sure what though. Re-creating settings.php solved this problem for me.

Re-direct loops

Two possible causes:

You’ve left the server re-write blocks enabled on a localhost environment like a Virtual Box.

Head to your vhost config, /etc/nginx/sites-available/mysite.com.conf and comment out the two small server blocks that appear above each of the main ones for HTTP and HTTPS.

Each is labelled: ## Return (no rewrite) server block.

You’re using mixed mode SSL and secure pages.

Remember the mixed mode SSL problems I mentioned above?

Nginx Mixed Mode SSL

Here’s the solution to getting mixed mode SSL working. (Courtesy of Perusio). Create a new file in your nginx config:

You can find more information by checking /var/log/pool name.log.slow.This is the log I mentioned earlier which will backtrace anything over a certain length.

I’m still getting these errors so I’m probably not the best person to go to for help with this, but here is a little of what I have managed.

Possible Solutions

Upping pm.max_children in the pool config files has helped a little (bumped 6 to 20), but I’m still getting them.

Upping request_terminate_timeout also in the pool config, increases how long php scripts can run for before the execution times out. As far as I can tell having this filled in overrides the max_execution_time value in /etc/php5/fpm/php.ini.

Unfortunately for me increasing the request_terminate_timeout, just means the processes run for longer and then get killed. But that might help.

Which makes no sense, as my termination time is 400 seconds and 16599 seconds is four hours. No answers from me yet. But maybe that helped a little.

There are a lot of other errors nginx can throw, however most of them have messages which should make it immediately explanatory what has gone wrong. E.g. Your SSL keys aren’t working, or you’re missing a bracket etc.

But either way that’s pretty much it. Hopefully this has helped you get up and running with nginx and have a better idea of what everything is doing. Any questions or corrections throw them below.

As I mentioned at the top. I’m having problems with PHP-FPM, I also don’t really understand it (so I have problems, go figure). I’d love to write a guide about how to go about debugging PHP-FPM, what sort of problems are common, what process you should use to debug it etc. (teach a man to fish and he won’t fill up the nginx group with trouble shooting).

So if you’ve got experience in this contact me or leave a comment, it’d be great to write up a resource for people just starting.

anyone here figured out how to get rid of
nginx: [emerg] unknown "not_allowed_cron" variable

it happens after I uncomment:

#

### Configuration for updating the site via update.php and running
### cron externally. If you don't use drush for running cron use
### the configuration below.
#################################################################
include apps/drupal/drupal_cron_update.conf;

apparently if this line is not commented out modules can only be installed via drush. funny enough I can run updates just fine so the comments on this directive make no sense to me at all.

anyone here figured out how to get rid of
nginx: [emerg] unknown "not_allowed_cron" variable

it happens after I uncomment:

#

### Configuration for updating the site via update.php and running
### cron externally. If you don't use drush for running cron use
### the configuration below.
#################################################################
include apps/drupal/drupal_cron_update.conf;

apparently if this line is not commented out modules can only be installed via drush. funny enough I can run updates just fine so the comments on this directive make no sense to me at all.

## If you want to run cron using Drupal cron.php. i.e., you're not## using drush then uncomment the line below. Specify in## cron_allowed_hosts.conf which hosts can invole cron.include cron_allowed_hosts.conf;

Which provide this variable.
And set the IPs allowed to run the cron.php

Hi I'm using the openresty source, and was using basically perusio conf , but without include apps/drupal/drupal.conf; enabled and was using location @rewrite's instead. When I use drupal.conf (@drupal) , I can not access my twilio php file which is just a bit of html, js, php api to twilio. It sits in a directory /var/www/mysite/twilio/hello-monkey.php ... everything else works G8 , until I try to access the hello-monekey.php I get a 404 I put this file in another site same conf but without the include apps/drupal/drupal.conf; and by changing chmod 755 can see hello-monkey.php in another directory (site) made the library path absolute, still no joy ?
The only other thing I have done is install Memcache and drupal module. I'm actually getting a xml message from my database saying leave a message. I also tried to disable the last section pps/drupal/drupal.conf; to allow php access . that just let me download the file so I put it back.