Tuesday, March 7, 2017

Abstract

I always get this question – how can I download Azure
blob storage files in Azure Linux VM? When I say use Azure CLI (Command Line
Interface) then next question asked is – Do you have step by step guide?

Well, this blog post is the answer to both questions.

The high level approach is outlined below –

Provision Linux Azure VM in a subnet. [Of course this step is out of scope of this article. For detailed steps refer -
this
guide
.]

Install Azure CLI in Linux VM

level Upload sample files to azure storage and then download them in a folder in Linux VM.

Prepare you Azure Linux VM

I have provisioned an
Azure Linux VM with OS as CentOs 7.2. Added this VM in a Subnet, with NSG
having only port 22 inbound open. Also I have attached a public IP to this VM
so that I can make SSH to this VM from anywhere over port no 22. Step by step
guide link is already shared above.

If you are having
different OS than CentOS then commands in below steps will change however high
level approach remains same.

Install CLI in Azure CentOS VM

First make SSH to
your Linux VM and run command “sudo su” [without double quotes]. So in
subsequent steps we will not face awesome “access denied” or “permission denied”
errors. Or we don’t have to add “sudo” word in every command we run.

There are two
versions of Azure CLI –

1.0
–
This stuff is written in node.js and supports both Classic [old way of doing
things on Azure] and ARM [new fancy way of doing things on Azure].

2.0
–
To make this version impressive Microsoft calls it “Next generation CLI” and is
written in python. Only supports ARM mode.

I will be using 2.0
version. Hence I need Python as well installed on the Linux Azure VM. So let’s
first install python latest version on Azure CentOS VM.

Let’s make sure that
yum is up to date by running below command –

sudo yum -y update

-y flag tells system
that “relax, we are aware that we are making changes, hence do not prompt for
confirmation and save our valuable time”. This command execution will take good
amount of time.

Next install
yum-utils using below command –

sudo yum -y install
yum-utils

Now we need install
IUS (Inline with upstream stable). Don’t get scared by name. This is community
project which will ensure that whatever version we install for Python 3, we will
get the most stable version. Run below command to install IUS –

sudo yum -y install
https://centos7.iuscommunity.org/ius-release.rpm

After IUS now we can
install recent version of Python. As of writing, the recent version is 3.6 but
I will install 3.5 to be on safer side. In python 3.5 version I see 3.5.3 is
the latest so let’s install it.

sudo yum -y install
python35u-3.5.3 python35u-pip

To verify, simple
run below command and it’s output should be 3.5.3.

python3.5 -V

Now install the
required prerequisites on CentOS using below command –

Finally, back to installation of CLI 2.0 -

curl -L
https://aka.ms/InstallAzureCli | bash

This may prompt you
to download CLI in which directory. Press enter to keep the default path of
installation which would be “/root/lib/azure-cli”. Similarly keep pressing
enter if more prompts are displayed.

Restart command shell
to take changes effect –

Exec -l $SHELL

Just type “az”[without
quotes] and it should you Azure cli commands information in CentOs. This means
installation of Azure CLI 2.0 on Linux is successful.

Run below command to
list storage related commands.

az storage -h

Upload sample files to Azure Storage

This step is straight
forward. Use Azure portal and create one standard [not premium] ARM based storage
account. Create container and upload 4 sample files in the container. It would
look like below –

Add Azure account to CLI

Run command as shown below. It will prompt you with a code
and link to enter the code. After this you will be asked for login using
existing azure related credentials. Successful login will show you the
subscriptions associated to your account as below –

Set storage account and download the blob

Now set credentials for
storage account.

export AZURE_STORAGE_ACCOUNT=YourStorageAccount

export AZURE_STORAGE
ACCESS_KEY=YourStorageAccountKey

Create a directory
named as test1 using the command. This is the directory in which we will
download blob contents. -

mkdir test1/

After this run below
command to download the blob file in test1 folder

az storage blob
download -c sample -n File1.txt -f /test1/File1.txt

az storage blob
download -c sample /test1

Change directory to
test by command cd test1 and run ls -l. This should list File1.txt as shown
below.

Limitation

Using Azure CLI you
can’t download all the blob from a container. You have to download each and
every blob individually; bulk download of azure blobs is not supported.

Resolution to Bulk download

To download all
blobs from a container instead of Azure CLI, we will need to use Azure XPlat
CLI. Or we can also use Powershell as it is open source now [although I have not tried yet]. It’s common to refer many approaches to achieve one task when you are in open
source.
J

Azure XPlat CLI is a
project that provides cross platform command line interface to manage Azure. Refer
documentation here - https://github.com/Azure/azure-xplat-cli.
But this is another blog on another day.

Conclusion

So I hope now you understand how easy it is to download
azure blob storage contents in Linux Virtual Machine.

Followers

About Me

I am Kunal Chandratre. Working as Cloud Solution Architect @Microsoft. My speciality is Microsoft Azure Cloud platform.
Awarded as Most Valuable Professional (MVP) in Microsoft Azure for consecutive 3 years. Passionate speaker, trainer...In free time (which I don't get usually)I write blogs and answers the forum questions. I was doing it just for timespass but now I have got addicted to blogging...Apart from work, I do variety of things which I can't tell here:).. I am trekker, singer, actor, painter, f1 racer, super hero in my dreams.. ...and now trying my luck with technologies...Keep posting...

Visitors

Disclaimer:

The information shared in this blog is the result of my personal experience with various technology platforms. In no way it represents the company I work for.
The information provided here is "AS IS" with no warranties, and confers no rights. This blog does not represent the thoughts, intentions, plans or strategies of my current employer or past empolyers or any other forums or community I belong to. It is fully my own opinion. Inappropriate comments will be deleted at the authors discretion. All code samples are provided "AS IS" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.I have full rights to edit/modify/delete any content of this blog without any prior notice to public/followers/RSS readers of this blog.