[Updated] esxcli plugin to run shell commands (new version)

[Please note: I have updated this blog post to reflect a new version of my esxcli-shell plugin and the changes in there. What's new in version 1.1 (of 2012-01-12):- supports for advanced shell function like pipelines, I/O redirection- execute multiple shell commands at once (or a single command spanning multiple lines)- logging of shell commands to /var/log/shell.log ]

The esxcli command is a very powerful tool to query and configure various aspects of an ESXi host's configuration, and you can not only use it in an ESXi shell, but also remotely through the (perl based) vSphere CLI and the (PowerShell based) vSphere PowerCLI.

The esxcli commands are organized in so-called namespaces (e.g. hardware, software, network etc.) for managing the sub-components of the server, and - out-of-the-box - you are limited to the commands that are exposed through these namespaces. But you cannot run arbitrary ESXi shell commands through it ... until now!

I have released an esxcli plugin (for ESXi 5.x) that allows to run any ESXi shell command through esxcli. It implements the esxcli namespace shell:

esxcli shell help

What is this good for? It allows you to run any ESXi shell command remotely without enabling and using ssh! This means e.g. that you can start e.g. shell scripts (like William Lam's ghettoVCB.sh script) through a remote esxcli or a PowerCLI script.

Installation

The esxcli-shell package is of the AcceptanceLevel CommunitySupported, so you need to change your host's configuration to this level before you can install the package. You can do this via the vSphere Client (see Host Configuration / Security Profile / Host Image Profile Acceptance Level: Edit...) or by running the esxcli commandesxcli software acceptance set --level CommunitySupported

Download the latest version of the esxcli-shell Offline Bundle from the V-Front Online Depot and upload it to a datastore of your ESXi host. Then install it with the following esxcli command:esxcli software vib install -d /vmfs/volumes/datastore1/esxcli-shell-X.X.X-Y-offline_bundle.zip

(X.X.X-Y is the current version number of the plugin. Replace datastore1 with the name of the datastore that you have uploaded the bundle to.)

In order to make the installation effective you need to either reboot the host or restart the hostd daemon by running/etc/init.d/hostd restart

in an ESXi shell. Please note: This will not interrupt any VMs, but will drop any management connections like vSphere client and remote esxcli sessions!

You can also deploy esxcli-shell through Update Manager by importing the Offline Bundle and adding it to a baseline. The hosts' acceptance level still needs to be changed (like described above) though, and you need to manually reboot the hosts or restart hostd on them, because the package does not force a reboot.

Usage with vSphere CLI

Obviously it doesn't make any sense to use esxcli shell commands in a local or remote (ssh) ESXi shell, because there you can run any shell commands directly. However, you can also run esxcli commands remotely from a Windows or Linux machine by installing the vSphere CLI on it. To run an ESXi shell command you then need to use the following syntaxesxcli -s hostname -u username -p password shell cmd -c "command1" [-c "command2" ...]

hostname is the name of the ESXi host that you want to run the command on. username is the name of a privileged user on this host (normally root), and password is the password of this user. command1, command2 etc. are the shell commands that you want to execute (at least one is required). If a command contains spaces then enclose it in quotes (like "ls -la").

Since version 1.1 you can use any character in a command line that is also available at a regular shell prompt. This includes the pipeline (|) and input (<) and output (>) redirection symbols. For easy output redirection of all specified commands you can also use the options -l (or --logfile=) and -e (or --errlog=) to specify log files for redirecting the standard and error output channels to. If you do not want to overwrite existing log files, but append to them, you need to use the flag -a (or --lappend). Finally you can make esxcli change the working directory before executing the command by specifying it with -d (or --workdir=). The default working directory is /var/log/vmware. So the complete syntax like shown in the screenshot above is:

Passing options to an esxcli command is a bit tricky in PowerCLI: You always need to list all the possible arguments in a specific order - and this is the alphabetical order of the long option names. The --help screen (as shown in the screenshot above) will show the options in exactly the right order:

command(s) to run (--command), specify as array if more than one: ("cmd1","cmd2", ...)

name of error log file (--errlog)

flag for append mode (--lappend): $true, $false or $null for the default (=$false)

name of log file (--logfile)

name of working directory (--workdir)

If you do not want to use an optional parameter then just specify the value $null for it.

Syslogging / Auditing

For auditing reasons any command that you type in a regular shell is logged to /var/log/shell.log. In version 1.1 of the esxcli shell plugin I added code that will also log all executed commands to this file. Unfortunately all the commands that are run through esxcli will appear as if they have been run by the user root - even if you used another account for the remote esxcli connection -, because they are run through the hostd daemon that runs in the security context of root. I haven't found a way to safely determine the remote user's account.Defeating command timeouts
If the shell command(s) that you execute through the esxcli shell plugin in PowerCLI take a very long time to complete then the command might fail with an error message "The operation timed out". You then need to raise the PowerCLI timeout for web operations with the command

Set-PowerCLIConfiguration -WebOperationTimeoutSeconds n

with n being the new timeout value (in seconds). The default is 300 sec. (= 5 min.). You can also disable the timeout by using a negative value for n, but I would not recommend this.

Please note that this feature requires PowerCLI 5.1.The Making of ...

I created this esxcli plugin after reverse-engineering already existing plugins. It turned out to be relatively easy and only requires some understanding of XML and Linux shell code. I will soon post more details about this ...

Yes, you can deploy through Update Manager. You still need to change the hosts' acceptance level though (like described in the post), and you need to manually reboot the hosts or restart hostd on them after the remediation, because the package does not force a reboot.

Have you actually tested deployment of custom VIBs via VUM? From my testing, even after setting the acceptance level to CommunitySupported, it seems that VUM still prevents custom VIBs to be staged & installed.