New Image Tools for SmartOS

Thank You

Thank you for contacting us. We will get back to you shortly.

October 22, 2012 - by Derek Crudgington

Share:

Starting with our newer machine images we've introduced a few customer specific utilities to help you in the SmartOS world: Quickbackup, Smtools, and Dtracetools. These are packages updated frequently in which anyone can install and use immediately to reap the benefits and make your life easier.

Smtools - A collection of utilities to handle various tasks around images such as changing the hostname, setting your timezone, monitoring resources, printing image information, listing creating removing databases and users, automatic tuning of MySQL, installing a blog, and more.

Dtracetools - Many DTrace scripts put in to a package for tracing and real time debugging of your image.

First we'll go over the basics of Quickbackup which is only available for Percona, MongoDB images and comes pre-installed.

Quickbackup (Percona)

Quickbackup is a command line tool wrapped around Percona Xtrabackup which provides a very simplistic, one command approach to backing up your data so you don't have to do any of the messy behind the scene configuration for Xtrabackup. When you provision a Percona image Quickbackup is already installed for you and you can begin backing up your MySQL data manually or on a schedule.

For backing up on a schedule Quickbackup is already installed as a SMF service which properties must be configured to specify when you want to backup. By default the service is set to backup every night at midnight but comes disabled by default. To enable this all you need to do is enable the service:

# svcadm enable mysql-quickbackup

Your backups will begin collecting at /backups every night at midnight. To set the service to perform a backup every 15 minutes of every hour Monday, Tuesday, Wednesday, Thursday, and Friday, you would use the following configuration commands to Quickbackup:

The only settings you should be concerned about are day, hour, minute, expiredays (age of backup files to be rotated), backupdir (specify where to store backups), and stream. The stream method is set to xbstream by default which allows super fast backups. In our testing we have done 10G backups in less than a minute!

Quickbackup (MongoDB)

Quickbackup for MongoDB is very similar to the Percona version, only that it is powered on the back end by mongodump and mongorestore. The service comes installed by default on MongoDB images which has the SMF name svc:/pkgsrc/quickbackup-mongodb and provides backing up on a schedule. All of the commands and configuration are the same as above.

sm-list-dbs Lists MySQL or PostgreSQL databases

sm-create-db Creates MySQL or PostgreSQL databases

# sm-create-db mysql testdb1
* Created MySQL database testdb1.

sm-tune-mysql Automatic database tuning of MySQL or Percona

# sm-tune-mysql
* Tuning MySQL, this can take up to 60 seconds..
Your tuned configuration is at /mysql/my.cnf.tuned
There is more to just copying this file over your old one and
restarting. Review the configuration file first to make any
necessary changes, and because 'innodb_log_file_size' has likely
been modified you need to remove the old log files first before
starting. If you don't do this properly then MySQL won't start.

sm-rebuild-pkgsrc Upgrade pkgsrc to the latest release

# sm-rebuild-pkgsrc
This script upgrades userland software in /opt/local, using packages
from the following package set:
http://pkgsrc.joyent.com/sdc6/2012Q2/x86_64
In the process, your /opt/local path will be backed up, new bootstrap image
will be downloaded and packages that you had installed, will be re-installed
from the new package set.
Consequently, if you had any software or files installed under /opt/local
that did not come from pkgsrc packages (when this machine was provisioned,
or installed later on), it will be removed and only preserved in the backup
compressed tarball.
? You do want to proceed with initial checks? y/N

Dtracetools

The Dtracetools is a pkgsrc package available since 2011Q4 which contains many (over 100!) DTrace scripts you can use to trace and debug issues in your Image. Dtracetools is available by default on all Images, but can be installed through pkgsrc with:

# pkgin install dtracetools

The tools will be installed at /opt/local/bin with the prefix dtrace-*. There are many useful Dscripts which provide tracing of PHP, Ruby, Perl, Python, Bash, Tcl, MongoDB, MySQL, and more. Below are a few examples:

As you can see we can easily get information from our applications using the Dtracetools package. This package is still under extreme development where some scripts may need minor tweaking. We are constantly updating http://docs.joyent.com/jpc/Collecting+Data+with+DTraceTools with updates as it develops.