Tag Archives: S3

Some updates from personal life…

I have joined Fremont based IP backbone & colocation provider – Hurricane Electric and would be working on some cool things at AS6939. 🙂

Updates on blog…

I have changed theme and entire look of blog and re-designed it with new plugins, more tweaking etc. As of now blog has more cleaner while theme which gives more space for posting, improved security with some ACLs, forced HTTPS to avoid telcos from injecting iframe in readers on 3G networks (which is very bad and worrying). Also, with use of bunch of plugins, now my I am hosting all static media content on AWS S3 to avoid local storage on server, it’s backup etc. Running it on AWS S3 with Geo replication + Cloudfront for CDN/efficient delivery made much more sense. Though sad that there’s no easy way for integration of Google Cloud storage with wordpress. S3 being more mature product makes it easier.

Just wrote a quick script to pull dump of Picasaweb albums backup on my server & further to Amazon S3. Overall I have good trust on Google for data but it’s always a poor idea to leave all eggs in single bucket.

OK here’s the script (poorly written code. Literally spent 10mins on this, thus suggestions to improve my coding are more then welcome!)

How to use

Simply download Google Cli scripts, and get your Google account working with the installed stack. Also if you need Amazon S3 backup support then install & configure s3cmd. Once you have both of these configured with your account, simple give executable bit to the script & run!

Code logic

Yes it’s super crappy code but anyways it does the work.

I couldn’t find an easy to way to download entire album base from Picasa. There seems to be some bug with Google Cli tools in directory creation and hence google picasa get .* . fails right after 1st album pull up. Google Cli offers pullup of album names (along with hyperlinks) with list-albums parameter. Thus first part of code is to pull that list and cut the first part of output using comma as delimiter. Next. the output is taken on a txt file which is read line by line in a loop. And the loop has simple code for download of each album one by one. Once download is completed, tar runs to create compress archive followed by gpg to encrypt the tar. This encrypted file is then uploaded to Amazon S3 using s3cmd tool and lastly all downloaded files are just deleted!

On Amazon S3 I have a bucket expiry rule which takes care of rotation and removal of old data. I can spend few more mins to make it more complex but this one just works! 😉