Why can I not have a purely virtualized SAN in vSphere 5? I cannot enable the iSCSI port binding option (it is greyed out and unchecked) or add the VMkernel to the iSCSI software host bus adapter unless there is a physical NIC associated with the vSwitch.

Yes, i did create a new vSwitch. I found NFS performance to be slower when compared to iSCSI. I bought a $700 SSD in order to use it as the L2ARC cache on a ZFS box that I am using to setup iSCSI Target on, so I am trying to get the most performance for my money.
–
cmaduroMar 6 '12 at 18:41

Are you attempting to create an all-in-one system with virtualized ZFS storage? If so, I'd go the NFS route. iSCSI may not make sense for intra-host communications.
–
ewwhiteMar 6 '12 at 18:47

How can I get 200+Mb/s 10000+ IOPS performance out of NFS?
–
cmaduroMar 6 '12 at 19:37

1

Depending on how your ZFS box handles iSCSI. The Solaris kernel with it's COMSTAR iSCSI target is implemented as files on top of ZFS, so essentially you're adding one more layer which should have higher latency than doing NFS which is really only a thin layer around the syscalls that do the real work underneath. 10K IOPS (without a pattern and IO size this is really meaningless) is easily doable with modern hardware and the appropriate block device(s). For random small IOs iSCSI will probably be only a few % better.
–
pfoMar 6 '12 at 19:44