Resources for A-Z Continuous Delivery How-To

I'm looking for a guide/resources/even a book that can cover specifics on best practices from taking raw web app code from Github, testing it, and deploying it on platforms like Heroku or Appfog. Then, how to best monitor for performance, availability, and security while covering things like disaster recovery, database maintenance, and such.

For instance, the toolchain in my mind would be:- code in Github (of course)- building of tests using BDD (cucumber, etc)- Codeship.io integration for testing harness- app deployed to stage and production in Heroku- database in either Heroku DB or one of the addon services- database snapshots to S3?- uptime monitoring by ?- performance monitoring by ?- best practices for maintaining database "state" and integrating that into the rebuild/recover process- How to best integrate all of the above?

I know that the language of the web-app heavily influences the specific tools and services used, but I'm ok with it being a soup-to-nuts howto with a different language just so I can use it as a reference model and swap out comparable parts if needed. I keep running into blogs and such that talk specifics about each tool or service, but I'm trying to get a "big picture, state of the art" view of how to build and run a modern web application using the best of the free/cheap tools and services out there. There seems to be an explosion in the usefulness and capabilities of these offerings, but I'm finding it hard to navigate and wrap my head around integrating all of these.

Am I crazy in that I can't find this on my own yet despite a lot of looking? Or is having this level of continuous delivery the "secret sauce" that people are holding close to the chest right now?

I'm looking for a guide/resources/even a book that can cover specifics on best practices from taking raw web app code from Github, testing it, and deploying it on platforms like Heroku or Appfog. Then, how to best monitor for performance, availability, and security while covering things like disaster recovery, database maintenance, and such.

<snip>

Am I crazy in that I can't find this on my own yet despite a lot of looking? Or is having this level of continuous delivery the "secret sauce" that people are holding close to the chest right now?

It's because the secret sauce is different for every organization implementing Continuous Deployment (I think that's what you mean, not Continuous Delivery).

Somethings to think about:What's your budget? There are some good free tools, but you will probably need to buy/subscribe some services (like monitoring).How many people do you have working on this? Will you have enough people to specialize in certain aspects or are you wearing a number of hats?Does your website need to be available while a new deploy is going on, or can it be down for a period of time?

Some tools to look at:Performance monitoring: Take a look at New Relic. Since it sounds like you are using Ruby, you'll be glad to know that there's a gem you install that instruments your Ruby on Rails app and gives you nice charts on performance characteristics. There are iOS/Android apps that will allow you to see what your website is doing while on the go.

Monitoring: Take a look at https://www.pingdom.com/. New Relic does offer some monitoring, but I don't find it very good.

Edit:You'll want some sort of continuous integration (to run unit tests and your lettuce tests), take a look at http://jenkins-ci.org/. There are plugins available that supports RoR tasks.

You asked about budget and team size and skill sets. Ignore that for a second. I'm looking for one or more reference models. Ideally, someone who does this for themselves would say "here's how I pull all this together." Then maybe another person says "here's how my company does it all" and finally, someone from a big shop says "this is how we do it on a large scale."

And they all outline their services used and some of the extra "glue" needed to make it all work. They would show how they use testing services, how they deploy without downtime, how they rollback, how they handle the database, what metrics they follow, what and how they monitor, etc.

I'm just not finding this kind of thing handy in my searching, so before I start blazing my own path to achieve release nirvana, I'd want to see if I can follow footsteps of people smarter than me or if I have to sharpen my machete.

You asked about budget and team size and skill sets. Ignore that for a second. I'm looking for one or more reference models. Ideally, someone who does this for themselves would say "here's how I pull all this together." Then maybe another person says "here's how my company does it all" and finally, someone from a big shop says "this is how we do it on a large scale."

And they all outline their services used and some of the extra "glue" needed to make it all work. They would show how they use testing services, how they deploy without downtime, how they rollback, how they handle the database, what metrics they follow, what and how they monitor, etc.

I'm just not finding this kind of thing handy in my searching, so before I start blazing my own path to achieve release nirvana, I'd want to see if I can follow footsteps of people smarter than me or if I have to sharpen my machete.

Like I mentioned before, the secret sauce is different for each organization as each organization has different resources and requirements. What you'll see in common are automation (configuration management via Puppet/Chef/other tool, automated builds using Jenkins/Bamboo/Buildbot, automated deploys via scripts/Capistrano/Fabric/Maven) and best practices (using an agile process instead of waterfall, having proper dev, QA, staging, prod environments, etc).

The continuous deployment process will vary a great deal based on your budget and available skill set. Commercial versions of Puppet and Chef are nicer/more featureful compared to the free version. Can you afford the money a consultant will cost to implement db optimization since you don't have a DBA on staff? You want to implement services in NEW_LANGUAGE but only know OLD_LANGUAGE, can you afford to hire someone or pay for training?

For example code deployment can be different based on:1) Number of servers -- how you deploy to 2 servers is different from deploying to 20 which is different when deploying to 200 which is different when deploying to 10,000+.2) Infrastructure -- Do you have to push code to each individual server or are you using a central network mount and just updating symlinks on the servers?3) Physical versus virtual -- Deployment to physical servers versus to virtual servers can be very different. There are techniques you can use with virtual servers (for example: deploy to a new set of VMs and swap out old versus new VMs after verifying the new VMs are good to go) that you can't duplicate on physical servers without spending a lot of money on hardware).

Dredphul,I know there are a dozen different options for each of the steps in the CI process, and they all depend on the organization and their constraints. But, it's beneficial to see someone's entire toolchain and hear why they use each item and how they flow together. That first video is what I was looking for. So, I'll be looking for more like them (hopefully ones that use online services).