Cloud Servers

Have Feedback?

Use task states with server imaging

Last updated on: 2015-06-03

Authored by: Brian Rosmaita

Some new task states have been exposed in the OpenStack Server Extended
Status Extension that provide more fine-grained visibility into server
state during the image-create (or “snapshot”) process. In this article,
I’ll tell you what they are and make some suggestions for how you can
make use of them.

Before the OpenStack Grizzly release, when you requested a create image
action on a server, the server would go into a special “task state” of
image_snapshot, and it would stay in this task state until the image
was completed. This single task state hid the fact that there are three
distinct phases to a snapshot operation:

The hypervisor creates an image of the server’s virtual hard disk.

The hypervisor packages the image and prepares it for upload to the
image store.

The hypervisor uploads the packaged image to the image store.

By far, the phase that takes the longest is the third phase, in which
the upload occurs. Further, in both phases 2 and 3, while the
hypervisor is working on your server’s behalf, it isn’t doing anything
with your virtual hard disk. So only during phase 1 is it important
that you avoid any operations that would modify data on the server’s
virtual hard disk. (Otherwise, the recorded snapshot might include
inconsistencies from which certain application programs on your
server–I’m thinking primarily of databases here–might not be able to
recover when you boot from the image.)

With the OpenStack Grizzly release, the semantics of the image_snapshot
task state were modified slightly and two new task states were added.
So now the task states that your server will go through while
processing an image create action are:

image_snapshot: the hypervisor is creating an image of the server’s
virtual hard disk

image_pending_upload: the hypervisor is packaging the image and
preparing it for upload

image_uploading: the hypervisor is uploading the image to the image
store

While your server is any of these task states, you can’t issue another
create image action on that server. As you can see from the task state
descriptions, the hypervisor is involved in all three phases of the
image create action, so all the extra bookkeeping resources the
hypervisor has allocated to your server are in use. You have to wait
until the entire snapshot process has completed and these resources are
released before you can create another snapshot.

Notice, however, that once the first phase is completed, you no longer
have to worry about operations occuring on your server that might
interfere with the effectiveness of your snapshot. Unfortunately,
server task states are not currently exposed in the control panel. You
can, however, easily check them by using the API or the
python-novaclient.

You’re looking for the element named “OS-EXT-STS:task_state”. (Since a
JSON object is unordered, it might not be right near the top of the
response, as it is in this example.) From the value displayed in this
example, you can see that the hypervisor has completed creating the
image of the server’s virtual hard disk and is now packaging and
preparing the image for upload.

Using the Python-Novaclient to Check Server Task State

The python-novaclient is a handy program you can run from the command
line. If you haven’t used it before, here are some How-To
articles to check out:

In this example, you can see that there’s no task state for the server,
so it could accept an image-create request.

Polling to Check Server Task State

The use case for server task states we’re discussing here is the
ability to:

Stop activities on the server that would affect the
quality of the disk image (e.g., stop a database
management system).

Issue an image-create command (via API, novaclient, or
control panel) for the server.

Monitor the server to see when it exits the
‘image_snapshot’ task state.

Restart the activities stopped before you took the
snapshot (e.g., bring your database management system
back up).

You can write a simple bash script to monitor your server.
How elaborate you want the script to be is up to you, here’s a sample
of the most relevant part. (Please read through and make sure you know
what it’s doing before using it.) It uses four programs (curl, egrep,
sed, and date) that are installed by default on most linux systems.
This fragment is pretty primitive, you have to control-C to stop the
script.