api

I was working on one of my vSAN Clusters a few weeks back and I had noticed a bunch of vSAN Objects being listed under the "Other" category within the vSAN Virtual Objects Health view as shown in the screenshot below.

I could not figure out what files or VMs these vSAN objects were actually associated to and it was especially strange since all VMs that were deployed on my vSAN Cluster were already properly showing up under this view and I could not account for these "Other" vSAN Objects. I had reached out to a few folks to see if anyone knew how to identify these objects and the only suggestion I had received back was try to run this python vSAN Health Status script located on one of the ESXi hosts participating in the vSAN Cluster to see if it provided what I needed.

The script is located at /usr/lib/vmware/vsan/bin/vsan-health-status.pyc and you run it like the following:

python /usr/lib/vmware/vsan/bin/vsan-health-status.pyc > /tmp/output

The above command just runs the script and stores its output (which is quite extensive) to /tmp/output. Once the script finishes, you can then open up the file using vi and search for the specific vSAN Object UUID in question. I was able to eventually identify what these vSAN Object UUIDs were mapped to (more on this later), but the overall experience was not ideal and it required SSH access to ESXi host which most customers disable by default. In addition, the process was pretty manual and tedious if you wanted to check multiple vSAN Object UUIDs.

So what did I do, well I looked for a better way of course! It turns out the output produced by vsan-health-status.pyc is actually all available using the vSAN Management API. Not only can you obtain this information programmatically and remotely but you can also retrieve this information by simply going to vCenter Server rather than having to directly connect to an ESXI host which was huge negative for me regarding the previous solution.

Last week during lunch I learned about a cool little project that my colleague Alan Renouf was working on in his spare time at night. He was interested in learning more about Docker and thought the best way to learn about something new was by using it, which is normally how I learn as well. He came up with a nifty idea to create a Docker Image that would house a bunch of useful VMware tools which included several VMware open source projects as well.

UPDATE (11/23/16)- The Docker Container has now been updated with all the latest vSphere 6.5 SDK/CLI/Tools. We also plan to make this new version of the Docker Image available on Docker Hub, so stay tuned for those details shortly.

Some customers in the past have built similar offerings by using a free VMware Appliance called vMA (vSphere Management Assistance). vMA is nothing more than a stripped down version of SLES that has the vSphere CLI (vCLI) pre-installed. In my opinion, vMA is pretty limited and you can not install additional packages without voiding official support. Even if you decide to ignore support and install custom packages, I have often seen this break existing dependencies. When I talk to customers about their use of vMA, most used it because it was just there, but the majority prefer to use their own harden distribution of Linux and install their own admin utilities and packages which may also include non-VMware tools.

I personally have no problem building my own VM appliance that contains the various VMware packages, utilities and scripts that I use on a daily basis. However, not everyone is comfortable with this idea. Wow could this be further simplified and automated? Well, enter vmware-utils a Docker Image that allows you to automatically build a new image that contains some of the most popular and widely used VMware Utilities.

I wanted to enhance the awesome work that Alan had done with couple more VMware open source tools that I thought might be useful to VMware Administrators, which I actually wrote about here in my List of VMware CLIs, SDKS and DevOps Tools article. I have already submitted a pull request for my changes here. If there are other tools or packages you think that are useful and wish to contribute back, feel free to clone the repository and submit a pull request!

The latest vmware-utils now contains the following:

vSphere CLI 6.5

PowerCLI Core 1.0

vSphere Management SDK 6.5

vSphere SDK for Perl 6.5

vSphere SDK for Ruby (rbvmomi)

vSphere SDK for Python (pyvmomi)

vSphere Automation SDK for Ruby 6.5

vSphere Automation SDK for Python 6.5

vSphere Automation SDK for Perl 6.5

vSphere Automation SDK for Java 6.5

VSAN Management SDK for Ruby 6.5

VSAN Management SDK for Python 6.5

VSAN Management SDK for Java 6.5

VSAN Management SDK for Perl 6.5

Virtual Disk Development Kit (VDDK) 6.5

OVFTool 4.2

PowerCLI Community Repository

PowerCLI Core Docker Container Samples

William Lam's vGhetto Script Repository

Pyvmomi Community Samples

Docker Client v1.12.3

Docker Compose v1.8.1

For those of you who are new to Docker, a great way to quickly get started is by using an awesome tool called boot2docker which allows you to run Docker Containers on either a Windows or Mac OS X system. This also helps remove any barriers if you do not want to setup a Linux machine to get Docker of if you are like me, running on Mac OS X and rather not have to spin up a VM just to use Docker. Below are the steps on getting boot2docker working and building your own vmware-utils Docker Image.

Step 2 - Take a look at the vmware-utils README, I spent some time updating it to make it more consumable for new users of Docker and follow the "How" section which will have you download the 4 VMware utilities as well as the vmware-utils DockerFile which we will need to build the Docker Container.

Step 3 - Create a directory and place all files into that directory. In this example, I have called the directory "vmware-utils".

Step 3 - We are now ready to build our vmware-utils Docker Image. Change into the "vmware-utils" directory that contains the files you downloaded earlier we will need to specify a "tag" for our image as part of the build command. In this example, I have called my image "lamw/vmware-utils" and to start the build process run the following command:

docker build -t lamw/vmware-utils .

Step 4 - The build itself may take some time depending on the speed of your internet connection. You will know when it has successfully completed when it states "Successfully built X" where X will be some unique ID as seen in the screenshot below.

Step 5 - Once the Docker Image has finished building, you can then run and connect to the Container by running the following command:

docker run --rm -it lamw/vmware-utils

At this point, you are now logged into the vmware-utils Docker Container that you have just built! It contains all the VMware Utilities that I have listed earlier and for more details on what has been installed and the location of the utilities, take a look at the vmware-utils Github documentation. If there are other tools you would like to see, feel free to contribute back by cloning the repository and submitting a pull request. I am definitely looking forward to seeing how this project evolves and providing a more dynamic way of creating a vMA-like experience without the current limitations. Keep up the awesome work Alan!

A frequent question that I get asked from customers and new developers that are looking to get started with VMware Automation is what CLIs (command-line interfaces) or SDKs (programing/scripting specific languages) are available for them to use? I know it is not always easy to find out what is available on the VMware.com website and some times I even have trouble browsing for all the right information as it is located in many different places. I figure it would be helpful not only for myself but also for others if I put together a list of the various VMware CLIs, SDKs and even some of the new DevOps Tools that VMware has been working on that are available to our customers/developers and partners. I have also included a couple of community tools that I think are pretty interesting. If there are others that you think I should add to the list, feel free and leave a comment.

Note: This is by no means a comprehensive list of every single VMware product/toolkit as there are many many more, including gated SDKs specifically for our partner eco-system. For more details on those API/SDKs, be sure to check out the VMware Developer Center.

One common piece of feedback that we have heard from customers and partners over the years is how difficult it can be to get started with using VMware's API and SDKs. The biggest challenge is not actually using the API and SDKs, but actually finding out what is available for a particular product or the solution you are trying to develop. The other challenge of course is where to download the API/SDK, language support and their corresponding documentation.

In the past, this information would be scoured throughout VMware's website and Google searches will usually lead folks to the VMware Developer Center. The site has been in need of a major revamp for awhile now and I am glad to see that VMware has taken this feedback very seriously as I have provided similar feedback when I was a customer as well as employee.

I am happy see that we have re-launched the VMware Developer Center which now contains a wealth of useful information that has never been available before! I was one of the lucky few who received early access to the new VMware Developer Center to help provide feedback on how we can better improve our user experience with the site. This will continue to be a work in progress based on on-going feedback from customers and partners.

Here are a few things worth checking out ...

Development & Certification

The first thing I think everyone will just LOVE is that VMware is now listing EVERY single API/SDK that is available which includes public API/SDKs as well as what is known as "Gated API/SDK" that usually are available to partners only. The reason this is a big deal is if you are a new partner, how do you know what is available to you and how can you add value on top of the VMware eco-system? You now can get a list of all VMware API/SDKs under the Developement & Certification section and it provides you with a high level of what each API/SDKs provides. For gated API/SDK, you will see a tiny lock symbol next to it. When you login to MyVMware and you are supposed to have access to these API/SDKs, the icons will be unlocked and you will be able to get more details including downloads and documentation. You can also filter the views by specific API/SDKs as well as scripting/programming language.

vCloud Suite Library

Earlier I mentioned that it was quite difficult for someone to figure out exactly what API/SDKs are available for a particular product, especially within the vCloud Suite. One really neat feature is under the "Library" section which provides you with a high level diagram of the VMware vCloud Suite and it is an interactive diagram which you can hover over and it provides information about each product. If you click on one of the products, it will provide more information on the API/SDKs that can be used to integrate or access that particular product. This is a great way to learn about VMware products from a scripting/programmatic perspective and we also have one for the Horizon Suite.

VMware Standards

One other cool page to visit is the "Standards" page which currently just contains the UX design standards for the vSphere Web Client. If you want to learn more about the design principles used to create the vSphere Web Client or looking to build your own vSphere Web Client plugin, you may be interested in this section. I suspect over time we will be publishing more design standards that can be shared with the general community and hopefully useful for our partners and integrators.

Be sure to explore the rest of the site, though this is only the initial release of the VMware Developer Center, if you have any feedback feel free to leave a comment either on my blog or in the VMTN community forums. I know the team responsible for the VMware Developer Center has so much more planned including more sample code for each of our API/SDK as well as more content and interactions from each of the engineering teams. So stay tuned and check back often!

The VMware VIX API in my opinion is still one of the most powerful and undervalued API's that is available to customers and partners for Virtual Machine guest operating system Automation. The VIX API allows you to perform guest operations such as starting/stopping an application, file/directory manipulation, uploading/downloading files all within the guest operating system without requiring any network connectivity to the Virtual Machine. This is all made possible through the use of VMware Tools that is running inside of the Virtual Machine and operations are only performed after a user of the guestOS is properly authenticated.

The VIX API was first introduced as a separate client API supporting VMware's hosted products such as VMware Fusion, Workstation and Player and later supported VMware vSphere. The API was quite popular for the hosted products and with the release of vSphere 5.0, the VIX API was finally integrated into the vSphere API to provide a single API that could manage all aspects of vSphere as well as these new guest operations APIs for your Virtual Machines. With this integration, these new APIs are now known as the vSphere Guest Operations API.

If you are familiar with the VIX API and would like to move or migrate to using the new Guest Operations API within vSphere, there is a really useful whitepaper that I recently came across called Transporting VIX Guest Operations to the vSphere API that provides a nice mapping of the API methods between the VIX and new vSphere Guest Operations API. The whitepaper also includes various code samples using Java, PowerCLI cmdlets and vSphere SDK for Perl to demonstrate the new Guest Operations APIs.

I think every vSphere administrator or developer should be familiar with the capabilities of the VIX and Guest Operations API and how it can help them further automate and manage your guestOSes and the applications that run inside of them.

Creating reports is a common task that every vSphere administrator must deal with at least once if not many more times in their career. Whether you are tasked to provide an inventory report of all your virtual machines and their configurations to your manager or to provide a compliance report for your security team to ensure that all virtual machines are hardened according to the vSphere Security Hardening Guide, report creations can be a challenge.

The vSphere platform provides a very powerful and rich set of APIs (Application Programming Interface) that can be consumed by both vSphere administrators as well as developers. However, there is a high learning curve when using the API and it takes quite a bit of time to learn and of course your manager is expecting the report to be done in the next 5 minutes. Even with abstraction tools such as PowerCLI, quickly building a robust, scalable and performant script is not always a trivial task, not to mention the maintenance and updates to the script because your manager wants to continually add more things to the report.

So how can we make reporting so easy that vSphere administrators will no longer have to spend time digging through API documentation and instead they will be able to quickly put together reports within minutes? Well, this is something that the CloudPhysics team has been working on as part of their CloudPhysics card platform and they have built a very unique card solution to help solve this problem. I had the opportunity to get an early preview of this new card and I have to say that I am very impressed at how easy and intuitive the interface is to build simple to very complex reports using this new card solution. The coolest part of this solution is that no programming or scripting skills are required!

To give you an example on how easy it is to use the interface, I recently helped a customer with a script to identify all virtual machines that had a virtual disk using the 2gbsparse disk format. I would like to think I know the vSphere API pretty well, so putting the script together took just a few minutes because I knew exactly where to look for this information. That evening I decided to go through the same scenario, but using the new CloudPhysics card solution and I was literally able to create the report in seconds! It probably took me longer to name the report than to actually create it. As you can see, I am pretty excited about the new card solution and it will be interesting to see all the cool new reports customers can now create and share with each other.

Here is a sneak peak at what the interface looks like when creating your own custom reports:

Here is the final report that is produced:

As you can see, I have quickly narrowed down the specific virtual machine that contains a 2gbsparse VMDK and I am able to see exactly which virtual disk that is.

If you would like to learn more about the new card solution, Irfan Ahmad, CTO of CloudPhysics will be hosting a live webinar to go over this new solution next Tuesday, June 25th at 9am PST and I would highly recommend you register for it to learn more.

Below is a bit more details on what you can expect from the webinar and you can register here.

While vSphere, the best-of-class virtualization platform, brings great efficiencies to the datacenter, reporting still presents challenges and pain to sys admins on a daily basis. CloudPhysics offers a radical new way to complete reporting for your virtual infrastructure. In addition to 20 high-impact reports, you can easily build your own and share the report template and output. When asked for asset reports, trending, activity, auditing and more, you’re never more than a few clicks away from delivery.

In addition to best practices and secrets to amazing mashups, you’ll learn to:

Primary Sidebar

Search this website

Author

William Lam is a Staff Solutions Architect working in the VMware Cloud on AWS team within the Cloud Platform Business Unit (CPBU) at VMware. He focuses on Automation, Integration and Operation of the VMware Software Defined Datacenter (SDDC).