Bhavani Polimetla's Notes

Release Management

Wrong Practices observed:
1. Bringing system down and applying patches. (There is no guarantee that team can complete apply fixes/changes in given time. Also tough to roll back if something goes wrong)
2. Applying patches/fixes without informing management/customers
3. Not taking file/database backup before applying patches/fixes.
4. Not doing enough testing before releasing fixes/patches
5. Manually pushing changes by hand and updating Prod data whenever they wish
6. Doing restarts during business hours
7. Not following deployment patterns.

Like this:

No fear to release large/new features (Release new features with enable/disable through config properties)

No fear to feel user’s acceptance

Release Mechanism – Web / Standalone

Release new features / fixes / patches to small amount customer base and slowly increase based on product stability.

Week 1 – 5% of Users
End of Week 3 – 10% of Users
End of Week 6 – 25% of Users
End of Week 12 – 50% of Users
End of Week 18 – 100% of Users

This % and Release schedules to be finalized based on project needs.

Practically seen following:
1. Company A deployed project A and at load balancer level they are controlling % of traffic to specific server. On new site they solicit feed back and offer gift cards / raffle tickets, ..etc
2. Company B is having 40 web instances. They will deploy instance by instance slowly with days gap
3. Company C is building standalone software. It release software to free community edition first. Based on feed back they release to commercial customers.
4. WordPress / Google and other release Beta to free customers first to avoid liability. Based on it is success they will release later to commercial customers.
5. Company D is having large customers in USA and small customers in other small countries like Italy. They first release software in Italy and after some time they release in USA.

Like this:

I had seen following deployment techniques used in Cloud for Java Applications

1. Hot Deployment (Partial Deployment / Quick Fixes / Live Fixes / Hot Fix / Dirty Fix)
When there is problem with one static component (JS/CSS/Image) this is the best technique.
People used to put class files, jsps too, but that too affects end user.

2. Hot Swap Deployment (Complete code swap, JVM Swap, VM Swap)
To achieve zero down time, we need to use commercial tools.
When there is no DB Changes, install new software on S2. Bring S2 to Up. In load balancer remove active S1 server and point to S2. This way S1 is passive and helps to bring it back, if there are any issues in S1 deployment.

3. Rolling Restart
When there are few servers (<5), it is easy to bring down one, install and bring it up. When we do one by one, we can minimize down time to end user without any new hardware.

4. Hard Stop Deployment (Bring all server(s) down and bring them back)
For major changes without backward compatibility, we need to stop all servers. Install and bring them back. There is huge down time and risks involved.

5. Promote Deployment
Many companies deploy software on many servers. For example Company A is deploying software on 100 servers. First they deploy on 1% servers. Test with target IP Address. If everything is fine they will promote same code to 5% of servers. They get feedback from users. Also there is another technique – switching the 5% of traffic irrespective of servers. This helps company to gain confidence on fixes/changes and also in users.