Dynamic FOG Replicator transfer rates

Note: this code has not been completely tested. As it stands right now its more of a proof of concept than a functioning system

I have a desire to have different replication rates between storage nodes based on time of day. During the day time I want to cap transfer rates between the FOG Master node and Storage nodes to 1Mb/s and then during the night time transfer as fast as possible (based on the WAN link speed).

So hacking about for a while I came up with this solution. Its a bit complex but simple at the same time. The first step is to update the bitrate values in the FOG database and then restart the FOG replicator service.

For my project I only need 2 different bit rates day and night. You might expand this concept to any number of transfer rates per day. I might suggest against having a more than 2 transfer periods, but that decision is based on speculation.

For this hack I needed to create 2 bash scripts. One for daytime rates and one for nighttime rates.

Lets start by making a place to put our bash scripts. For this example I’m placing my files in a new directory in the /opt/fog directory.sudo mkdir /opt/fog/cron

The notable bits in this is the line declare -A StorageNodes=( [ATLNode]=1000 NYCNode]=1000 [LANode]=600 ) This is a key value pair that will be used to update the FOG database with the new bit rates. You can add as many or as few storage nodes to this key value array as necessary. Just ensure you keep the proper formatting, capitalization and spacing. The key value must match exactly the name of the storage node from the FOG management GUI. The value must be an integer value. Again watch your spacing adding or removing storage nodes. There may be better ways to go about this, but “it works for me” YMMV.
4) Next we need to create our nighttime bash scriptsudo touch /opt/fog/cron/replicator.nighttimesudo chmod 755 /opt/fog/cron/replicator.nighttimesudo vi /opt/fog/cron/replicator.nighttime
5) In the replicator.nighttime we’ll paste the following code

This concludes the setup required to change the replication rates based on time of day. Is this an ideal solution, no. Will it work, yes. Are there some caveats in the design, yes. So in short it worked for me.

Any image that isn’t completed when the replicator gets restarted, the image transfer too gets restarted.

I is why I think that rsync would have been a better tool for image replication than ftp. Then it could possibly limit the number of NAS devices supported, because not all NAS devices have rsync as an option. But that is a topic of another thread.

If you look at the above concept from a purest standpoint. At 5am I want daytime transfer rate rules applied, I’ll have to live with the results of having to send one of the partitions (hopefully just a partition) that was in transit over again. I guess that is an OK trade off. I absolutely want to ensure that full WAN bandwidth is available as the workforce starts at 6am.

I like it but there are issues with how the image replicator works with this approach.

Any image that isn’t completed when the replicator gets restarted, the image transfer too gets restarted. Tom made this change due to a bug that was discovered (re-discovered by me) in how lftp mirrors changes in files. It didn’t do it right - hashing the source and destination image files and comparing the hashes showed they were not the same. The solution was - whenever it was determined that replication needed to happen, delete the destination file first.

If this issue with lftp could be checked on, and if that’s solved and the FOG replicator code can be tweaked, this stuff you’ve wrote here would be that much better.