we already are slowly implementing managed configuration of the systems via puppet, but the next step will be to automate the service lifecycle. Currently we have an in-house developed solution that basically consists of cronjobs on the various servers that read data from MySQL and deploy a new domain:

99.9% of the times we add domains, 0.1% of the times we remove them or shuffle them around in some way - the latter is currently done by hand via ssh. Some systems are already spread over multiple servers (e.g. domains spread over more than one mailbox server) and some will be in the future (eg. MySQL).

We're gonna develop a simple web interface for our customers to manage their domains and get new ones and we obviously would like to eliminate all human work wherever possible. The web interface (programmers, think view) will not contain any business logic and will talk (probably via RESTful-like HTTP) to another application that'll do just that (programmers, think controller).

Would you interface the business logic app to puppet and use that to manage the services? Or let's talk directly with the systems? (eg, mysql config is done via SQL, proftpd accounts + mail relay data + antispam settings are stored in the db, apache vhosts could be handled by a simple agent that could do create/config/purge and similar stuff)?

How do the big hosting shops manage this kind of stuff? Also, any other advice would be welcome.