Maintenance

Advanced

Blog

Site Information

iSCSIWhat is iSCSI?

iSCSI or internet SCSI, is really not that difficult to understand. Once you get your head around the basics, you can use this great technology just about on every system out there. Most vendors build it into their operating systems so you can immediately use it after installation.

In this section I will cover the basics on internet SCSI, the components that make it up and then how to install and use it on a Solaris 10 system.

What is iSCSI?

Simply put, it's storage that is made available over the normal IP network to hosts or servers. That's really all there is to it.

Isn't that what NFS and CIFS does as well? Yep. that's exactly what NFS and CIFS does, but these storage models are what they call, file centric models. Stop frowning, I'll explain it in more detail.

You see, with NFS and CIFS, files and directory's are shared over the network. You share some files on the server side and then on the client side you mount these files and then immediately start to store mp3's or documents on it.

You don't have any control over what type of filesystem you just mounted, you just mount and use it.

With iSCSI, a disk is presented to the host instead of a mount point. The disk looks like a normal disk. You format it like a normal disk. You partition it like a normal disk and create filesystems on it, like a normal disk.

It's like connecting a USB drive to you system, only it's done over the IP network. You can either directly connect to the system that makes this storage available, or you can access the storage via standard Ethernet switches.

No need to invest in expensive fibre channel switches or infrastructure. It's all there, just use it.

The storage system that makes the storage avaiable is called the iSCSI target, and the system that uses this storage is called the iSCSI initiator. Clever hey. Below is a picture of this model.

Internet SCSI

Later on this page I'll show you how to setup a target and initiator on Solaris 10 and Windows 7. On windows 7, I'll just show you how to configure the initiator. The Solaris 10 system will be configured as the target.

Components of internet SCSI

Let's now look at the different components that internet SCSI is made up of. I don't want to spend too much time on this. It can become very complicated and that's not what I'm aiming for here. But you have to understand the theory behind this technology to use it correctly.

I will list the most common components here, with an explanation.

What is a iSCSI initiator?The initiator is usually the device or port that will make use of whatever storage is presented to it. Initiators is not just found in internet SCSI, you will here this term with fibre channel and normal SCSI as well.

It could be a host bus adapter (HBA), hardware initiator, or software running on a server. Software initiators are very common and is, of course, much cheaper. They install as part of the internet SCSI software and no further software needs to be installed.

Hardware initiators are usually used when the system wants to boot from an internet SCSI target. But you can boot from a software initiator as well.

An initiator is like a boss in a company. The boss always initiates the work. He checks what needs to be done in order for the company or division to run smoothly. So, the initiator will make use of the storage or lun presented to it and use it to make the system run smoothly.

What is a iSCSI target?A target makes some storage available to the initiator. This target could be a lun on a storage array or a lun made available from a normal server where some kind of storage is attached to it.

Most companies producing storage arrays have the internet SCSI target capability built in to their appliances. Some well known brands include, NetApp, Sun ZFS storage 7000, Sun Storage 2510 series and Fujitsu Eternus DX series. These are just a few, there are lot's of companies that make these internet SCSI storage devices.

What is an IQN?IQN or iSCSI Qualified Name. This has to do with addressing the storage. You create a lun on the target and make this lun available to initiators. For the initiator to access the luns it has to communicate to a target.

It's very similar to fibre channel world wide names (WWN). Every port on fibre channel has a WWN, and the systems use these WWN's to map storage to it.

If you directly connect servers to storage then this is not much of a problem, cause the server can see only that storage connected to it. But, what if the storage is connected to several server? Why can we identify which luns should go where?

There is actually 2 ways that internet SCSI identifies targets and initiators. You can either use IQN's or Extended Unique Identifier (EUI). The IQN is most commonly used.

An example IQN would look like this: iqn.1986-03.com.sun:01:ea7ec2f5ffff.4d9aa3d2

It looks more complicated than it really is.

iqn identifies the type of addressing. In this case it's iqn. You could also have eui.

1986-03 is the date string. This is the date that the domain was registered.

com.sun is the domain name of the company, reversed.

This first part, up to the : is assigned by the standards organization to a company that uses internet SCSI. This is fixed and should not be changed.

After the colon, :, you can use whatever you want to. The above example is from an Oracle Sun server, so they used their own type of naming scheme as shown above. You may want to use a more descriptive name after the colon. Maybe something like the server name and port, or something similar.

There is a RFC (Request For Comments) for iSCSI, RFC3720.

Below is an extract from the document where they show you how you how the naming scheme for IQN and EUI works:

There are three discovery methods:- Static discovery- Send targets- iSNS

Static discovery is a one on one mapping from the initiator to the target. You map to a specific iqn.Send targets is where the initiator checks the iqn of a target to see what luns are available.iSNS (Internet Storage Name service) is similar to a DNS (Domain Name Servive). The initiator will query the iSNS server to see what target luns are available.

The most common discovery is send targets and static. For larger environments you might want to have a look at iSNS.

What is CHAP?

Challenge Handshake Authentication Protocol (CHAP) has to do with security. If you want the initiator to authenticate itself to the target, you would setup CHAP.

You set a user and password on the target so that when the initiator connects to the target you specify the correct user and password in order to use the storage.

There is much more going on in the back ground but that has to do with the authentication protocol that CHAP uses. I'm not going into details with this. Read the RFC if you need more details.

General considerations

When setting up an internet SCSI network, use a separate network for internet SCSI. Lot's of data will travel through this network. You DO NOT want to use your normal public LAN for internet SCSI.

I'm going to say it again, use a separate network when using internet SCSI.

Don't use 10Mbit or 100Mbit networks. Take my word for it, it's too slow. You would need at least 1Gbit or higher.

For redundancy and speed I would also suggest you use some kind of multi pathing such as Internet Protocol Multi Pathing (IPMP), Link aggregation or Windows MPIO (Multi Path IO), if using a Windows client.

Use as many links as you can. Don't think that one 1Gbit channel is enough. Use two or more and the biggest you can afford. If you just want to play with internet SCSI then using one link is fine, but for serious server environments I would suggest that you do some research and planning.

The cool thing is that iSCSI uses normal IP networks. So, if you are familiar with networks, then you already have an advantage when setting up the network.

IF you need more information on internet SCSI then read the RFC 3720 (Opens in a new window) on it. It's a bit technical but if you are into reading RFC's then give it a go. It's very long and technical but it gives a lot of detail on this technology.

internet SCSI examples

My sample setup consists of two Solaris 10 systems. These are Oracle VM VirtualBox virtual machines with Solaris 10 installed on it. The one system will be my target and the other the initiator. I will use various types of backing store luns.

The backing store is where the actual storage is that you want to make available via internet SCSI. Here is a list of devices that you can make available as targets:

Files - You can create a file using mkfile, and then make this available as a target.

Block devices - Devices such as /dev/dsk/c0d0

SVM Solaris Volume Manager volumes - Devices like /dev/md/dsk/d100

ZFS volumes - This is the easiest way to make targets available. You create a pool, create a volume with the -V option and set the shareiscsi=on property. If you are going to use a server with a jbod connected to it, then I suggest you use ZFS to create the targets with. It's easy and fast.

We use the iscsitadm, in Solaris 10, to administrate targets, and iscsiadm to administrate initiators.

So, here goes.

Create target luns

First of all we need to create a base directory where the information for the targets will be kept. It's like an administration area that internet SCSI uses to keep configuration information. I will use a directory called, /iscsibase.

You could also create the base directory and then create targets in the base directory by just specifying the size and name of the target. This is a nice and simple setup cause you don't need to worry about where you created the targets. Later I will show you an example to do this.

Let's have a look at the command. You create targets with the iscsitadm create target command. The -b option is for the backing store. This is the actual piece of storage you want to make available as a target. In this case it's /dev/dsk/c0d1s6. I partitioned all the space in slice 6, so that's the slice I used. The last operand is the name of the target. In this example I called it block1

I then used the list target command to display the target name, iqn and whether there's any connections to it.

We then used the list target -v options to get a bit more information about the target. With the -v option, you also get information on the ACL, lun number and some other information about the target.

Using send targets on initiators to map targets

So now the target has been created and we need to discover it from the client system that will use this target.

We need to specify a discovery method to actually see the targets on the initiator.

On the client system, we'll use the iscsiadm command to configure the initiator.

I will use the send targets discovery method first just to show you how to easily discover luns or targets.

Please Note: I had to split the command into three lines cause it doesn't fit into my page layout. The create command should be entered continuously on the command line. The backslash (\) above, just indicates that the command continues on the next line.

Okay, we see it. We can now go and partition and use the lun. With static discovery, you can specify which luns you want to add to the host. With send targets, all luns that are configured on the target will be seen by the initiator.

Using static discovery might be a good idea if you only need to discover one or two luns. If there are lots of luns from the the, send targets might be a good idea. It's all about planning. I cannot tell you which method to use. But the previous examples might help you decide.

Using internet SCSI with ZFS volumes. The easy way

If you find it difficult to use normal block devices or files to create targets, then I suggest using ZFS when creating targets. I would suggest it anyway if you are going to deploy internet SCSI using normal servers with storage connected to it. It's really easy creating internet SCSI targets using ZFS.

Let's get going.

I'll create a mirror pool with two disks and then create volumes out of this pool that will be used as targets on my initiator system.

I have only showed you the basics on how to create target's and initiators. There's still a lot of stuff that you can do. I don't want to overwhelm you with options and operands. The information on this page should give you a good understanding of internet SCSI and how it works.

The best way to learn internet SCSI is to actually do it. Get yourself a system or two and start to create targets and initiators. Try to setup CHAP between the targets and initiators. Use some other features like multi pathing, ACL's and Target Portal Group Tag (TPGT).

Use the man pages and internet to get more information if you need to. I think that you would probably use some kind of hardware that will be used as the target and servers as initiators.