Building a Single-Node VSAN

I was wondering if it possible to speed up my Intel NUC based ESXi with Virtual SAN. The idea is that compared against vSphere Flash Read Cache, Virtual SAN can use the SSD not only as read cache but also as write buffer. This post explains how you can create a Virtual SAN Datastore on a single ESXi host from the command-line without a vCenter Server.

It goes without saying that this is neither the idea behind Virtual SAN nor officially supported by VMware. It also violates VMware's EULA if you are running Virtual SAN without a VSAN license. To assign a licence you need a vCenter Server and wrap the single ESXi into a Cluster.

The guide starts with a fresh ESXi installation in evaluation mode and I will not cover licensing. According to this post it also works with the free ESXi Hypervisor license, but to be compliant with VMware's EULA you have to add the host to a vCenter Server and assign a Virtual SAN license.

First step is to enable VSAN traffic for the VMkernel interface

~ # esxcli vsan network ipv4 add -i vmk0

Next step is to identify the devices to be used for VSAN. Both, SSD and HDD, must not contain any data or partitions. The easiest way to identify devices is with the vdq -q command.

There is a link to your article where you are doing something similar with the free ESXi. I've seen it while searching for an answer to the license questions. I didn't know that bootstrapping article because we never had this problem as we are not running the vCenter in the VSAN itself ;-)

Nice post.... Me too I was expecting that you'll be "nesting" the complete VSAN solution, for learning purposes. And actually, this makes me wonder (and gives you an idea for a follow-up post perhaps) if All-Flash VSAN would work the same way... ? -:)... Let say you replace the 5400 rpm drive with a SSD...

There are some quick performance values in the article. I think it's hard to give an actual statemant about the performance. It's an unsupported configuration of course and you can't compare the performance against a supported enterprise-grade server.

The performance with VSAN (1 SSD + 1 HDD) is faster than against the HDD itself. I think the performance is "OK".
There are some critical arguments against VSAN with AHCI. There was a bug in 5.5 which has been resolved in 5.5 U2.

mkdir doesn't work against VSAN datastores. The VSAN datastore, as an object store, is not a filesystem. A lot of the filesystem-related things don't work because there isn't really any such thing as a "directory" or a "file" with VSAN.

All "directories" on VSAN are actually special-purpose objects that are subsequently formatted with a VMFS variant to provide a real filesystem for small files (like VMX config files, VMDK descriptors, etc.).

If you want to create a directory in the VSAN datastore, you actually have to create a VSAN namespace object, and it is a different command:
# /usr/lib/vmware/osfs/bin/osfs-mkdir /vmfs/volumes/vsan\:*/

Once that is created, you can then run mkdir for subdirectories, or touch files, or whatever else because the namespace object is formatted with a filesystem.

In the single-node case, the default policy for namespace objects will need to be set to hostFailuresToTolerate=0 or forcedProvisioning=1 (as outlined in the article above).

@fgrehl - the creation of the vsandatastore works for me following your method, and I can browse the datastore, but I cannot create any machines or svmotion anything onto it. I am also running v6.0 Update 2