"Our initial running theory was correct--the server that hosted
the apachecon.com (dv35.apachecon.com) website had been compromised. The machine was running CentOS, and we
suspect they may have used the recent local root exploits patched in RHSA-2009-1222 to escalate their privileges on this machine. The attackers fully compromised
this machine, including gaining root privileges, and destroyed most of
the logs, making it difficult for us to confirm the details of
everything that happened on the machine.

This machine is owned by the ApacheCon conference production company,
not by
the Apache Software Foundation. However, members of the ASF
infrastructure team had accounts on this machine, including one used to
create backups.

The
attackers attempted unsuccessfully to use passwords from the compromised ApacheCon
host to log on to our production webservers. Later, using the SSH Key of the backup account, they were able to access
people.apache.org (minotaur.apache.org). This account was an unprivileged user, used
to create backups from the ApacheCon host.

minotaur.apache.org
runs FreeBSD 7-STABLE, and acts as the staging machine for our mirror
network. It is
our primary shell account server, and provides many other services for
Apache developers. None of our Subversion (version control) data is
kept on this machine, and there was never any risk to any Apache source
code.

Once
the attackers had gained shell access, they added CGI scripts to the document root folders of
several of our websites. A regular, scheduled rsync process copied these scripts to our
production web server, eos.apache.org, where they became externally
visible. The CGI scripts were used to obtain remote shells, with information sent using HTTP POST commands."