I'll be doin' all I can, If I die an honest man.

Main menu

Category Archives: Webdev

If you read my posts on building a local dev server, you will have seen that I have been using .dev as a local-only TLD. Well .dev was bought by Google and they will apparently be selling domains on it. And to make matters worse, both Chrome and now Firefox have preloaded their browsers with unalterable HSTS settings that require HTTPS for any request to a .dev domain. If you type “http://foo.dev” into your address bar it will ignore what you typed and request, “https://foo.dev” instead. That killed my local sites. More details here:

I want to host a copy of all the websites on which I work, right on the computer where I do my coding. I don’t want to depend on a server on my LAN that won’t be there when I am working from out of the office. I don’t want to work on a remote server that requires a slow (S)FTP loop to try out every change. And I also don’t want to work entirely from the command-line on a remote server. So I set up my macbook with wildcard DNS that points any hostname *.test to localhost (127.0.0.1). If I am working on example.com’s website, I can use “example.test” as a hostname that points right back to my machine. Now I need to set up Apache to host example.test. But adding a virtual host config for every website on which I work is going to be extremely laborious. Luckily Apache supports no-config mass virtual hosts. All I’ll need to do to add a new website for example.test is create an “example/” directory in the right spot.

Apache includes cool “mass virtual hosting” features that will allow it to suss out the DocumentRoot from the request hostname. First, let’s create a directory where our virtual hosts will live:

mkdir ~/Documents/vhosts

I put mine in a folder inside my Documents folder. For my login, that’s /Users/haroldp/Documents/vhosts. But you can put it just about anywhere. I added the following to my /etc/apache2/httpd.conf:

First, note that /Users/haroldp/Documents/vhosts makes sense for me on my computer, but it’s going to be different for everyone, so you can’t just copy & paste. Note that I tacked an /htdocs directory onto the end of my vhosts VirtualDocumentRoot directive. This is not necessary at all, but I like to have directories associated with a website, but outside the webspace. You don’t have to do that if you don’t want. Note too that I setup PHP, because I’ll be using that. You may or may not want those directives.

Then I uncommented the mod_vhost_alias module to enable apache’s mass vhosting directives:

LoadModule vhost_alias_module libexec/apache2/mod_vhost_alias.so

While I was in there I told apache to Listen only on localhost:

Listen 127.0.0.1:80

Because I don’t want anyone else to be able to hit the webserver on my laptop. Just me.

Next I enabled the PHP module by uncommenting:

LoadModule php5_module libexec/apache2/libphp5.so

I set the ServerName so Apache won’t whine about it:

ServerName www.test

I set the ServerAdmin so I know whose fault it is when something doesn’t work:

ServerAdmin haroldp@internal.org

And finally, I changed the user that Apache runs as to my login:

User haroldp
Group staff

WARNING, HIGH VOLTAGE!! This is very dangerous. I’m setting up apache to do its work, including running PHP scripts as my own UID. This means that a naughty script could do anything on my machine that I could, including very bad stuff. I am doing this so things like WordPress will create files with my login instead of the web user, which avoids a lot of hassles, and makes upgrades much easier. I am not worried too much about the security implications because I am running my own code, and the server is only available on localhost. If you are already on the machine, there are easier ways to do bad things.

Ok, let’s check our work:

sudo apachectl configtest

Fix any errors and rerun until Apache starts without issue. Then simply:

sudo apachectl start

If that worked, you should get a (404) page if you go to http://127.0.0.1/ . But let’s try out our virtual hosting:

We want to develop websites or other internet services on our OSX computer. It’s convenient to point wildcard DNS for a whole (imaginary) top level domain to localhost, so we can invent as many domains as we want without having to edit any config files or do any work.

I develop websites on my laptop using a local web server. Often those sites have functions that send out email, and that needs to be tested, along with everything else. It can be a problem when some function sends out lots of emails to customers, admins, affiliates – a bunch of people. If I’m working with a copy of the “live” database to debug some problem, it might try to send emails to places I don’t want (real customers). What would be nice is if it generated those emails, but just wrote them to a file on disk where I could look at them.

This is pretty easy to do with Postfix, my MTA of choice and the one that ships with my OSX laptop. First, add a line to the end of /etc/postfix/master.cf:

It should run as user _www, which is the UID my web server runs as. The mail files created with have 0600 permissions, so only the owner can read them. More on that later.

The pipe argv is set to tee (tee has a man page you can read), to split output to a file. And that file is in a directory in my websites folder. Each file will be named using the postfix queue ID and the recipient. I thought that would be sufficiently unique for my needs.

When that is saved, we need to create the directory to collect those emails and make it writable by the fs_mail process:

If you are setting this up on your own computer, you will want to adjust the directory location to your suit your needs.

Now we need to tell postfix to use our new service for all outgoing email. Edit /etc/postfix/mail.cf adding the following:

default_transport = fs_mail

That should do it. Restart postfix and check your mail log for any errors:

sudo postfix stop
sudo postfix start
tail /var/log/mail.log

If that all looks good we can test by sending an email from the command line:

% mail haroldp@internal.org
Subject: test #42
This is a test message. End it by typing a period (.) on its own line, and hitting return.
.
EOT

Check your mail.log again to see that it worked without error. Check your new spool directory to see if there is a mail file in there.

If that is working, then you are done! But remember that we saved those messages as UID _www? That is the default user ID of apache web processes on OSX, so my local web server can read those files. For extra credit build a web page to view the 10 newest emails in your spool dir:

I’ve finally got a working config for Apache with PHP-FPM, per-vhost pools, UIDs and chroots. There seem to be a lot of tutorials around the net to help set up FPM with nginx, but very little with Apache. The following instructions are for FreeBSD, but they would be easy to adapt to most any OS. This document is still evolving, but I wanted to get it out to people in FreeNode #php-fpm who have been asking for help.

Why am I setting up PHP like this??

I have been using Apache with mod_php for years, and it works, but it has a number of problems, especially in a virtual hosting situation. All PHP scripts will run with the webserver’s UID, which is crummy for security. Users’ scripts can see the whole file system. When Apache services a non-PHP requests, such as an image or style sheet, it still has to load the whole PHP interpreter, using a bunch of memory.

This setup addresses each of these issues, hopefully making PHP sites more secure, and less memory hungry. Instead of including the mod_php interpreter in Apache uses the “FastCGI” protocol to parcel requests to a long-running “PHP-FPM” server. Each website I’m hosting is has its own configuration. Each runs under its own UID. Each is chroot-ed in the owner’s home directory. Only PHP requests are handled my PHP-FPM. Everything else stays in Apache.

Let’s get to the details

We’re going to install and configure a bunch of stuff software, and then set up a chroot environment.

Install Apache 2.2

cd /usr/ports/www/apache22
make install clean

Be sure to enable suexec in the Apache options dialog.

Enable Apache

Add apache22_enable=”YES” to /etc/rc.conf and start it up

service apache22 start

Install PHP-FPM

cd /usr/ports/lang/php5
make install clean

Do NOT build the Apache module.

DO build the FPM version

Building the CGI and CLI versions is fine as well

I add the mailhead patch too

Install the PHP extensions

This is a bit of a FreeBSD-ism, that you won’t have to do on most other OSs. FreeBSD strips the PHP port down to a bare minimum, and moves all the plugins – including the default ones – into their own ports. The php5-extensions meta-port collects them all into one place.

Now FPM needs some configuration. Create a directory to store per-vhost fpm configs:

mkdir /usr/local/etc/fpm.d

Then edit the global php-fpm.conf, un-commenting:

include=etc/fpm.d/*.conf

switching the listen statement from a tcp port to:

listen = /tmp/php-fpm.sock

and changing the pm to:

pm = ondemand

There are a couple different types of process manager (pm). On demand will prefork zero (0) processes. They will only forked when needed. I chose this for lots of small sites. You may want a model that suits your setup better.

Now lets create a vhost. Given a site named “example.com” owned by user “luser”, here’s my template:

Living in a chroot
So PHP’s mail() function invokes your system’s sendmail binary, usually /usr/sbin/sendmail. From within a chroot, that won’t be available. However, there is the further problem that even if you copied sendmail and any libraries it needs into the chroot, it will want to write files to /var/spool, and again, that won’t be available. We need a work around. Install mini_sendmail. It is a sendmail workalike that you can easily copy into a chroot, and instead of writing to /var/spool, it will make an SMTP connection to localhost. Be sure to set the -f envelope-sender in your fpm pool config, or mini_sendmail will use your username out of the environment when PHP or mini_sendmail was compiled, at the machine name. PHP scripts can still override it using the mail() functions additional_parameters argument.

PHP will need a /tmp directory. If you are using MySQL, you will need to hardlink your mysql.sock into there or use TCP connections. If you link the socket, you need to redo that EVERY time you restart MySQL. (I should include my rc script here). Hard link mini_sendmail into the chroot. And finally, PHP needs a shell to invoke sendmail. Yes this sucks. You can copy /bin/sh in, but chances are, it needs libraries that aren’t in the chroot. I could copy those too, but I just copied the crunched binary from FreeBSD’s /rescue dir. Yes, this sucks even more because it includes stuff I don’t want or need, and I need a better solution. TODO: crunch my own sh with a couple other useful items. Maybe use busybox for this?

Set the tmp dir in php.ini to

upload_tmp_dir = /tmp

Update #1

I had a problem with a number of server variables not getting properly translated for use within the chroot, so I added a php prepend directive to the php-fpm conf files like:

Update #2

PHP’s streams tools (like file_get_contents()) rely on openssl for HTTPS URLs, and many other plugins (like SOAP) in turn rely on those streams. Curl seems to function just fine in a chroot, but PHP’s openssl streams require certain device nodes to function. You will have to mount /dev inside your chroot in order to use them. More on this when I get a good system in place.