Netapp test cases

Netapp Full Labs

Would you like full HTML/PDF FilerView documentation to be installed [yes]:

Continue with installation? [no]: yes

Creating /sim

Unpacking sim.tgz to /sim

Configured the simulators mac address to be [00:50:56:17:5:bf]

Please ensure the simulator is not running.

Your simulator has 3 disk(s). How many more would you like to add? [0]: 40

Too high. Must be between 0 and 25.

Your simulator has 3 disk(s). How many more would you like to add? [0]: 25

The following disk types are available in MB:

Real (Usable)

a – 43 ( 14)

b – 62 ( 30)

c – 78 ( 45)

d – 129 ( 90)

e – 535 (450)

f – 1024 (900)

If you are unsure choose the default option a

What disk size would you like to use? [a]:

Disk adapter to put disks on? [0]:

Use DHCP on first boot? [yes]:

Ask for floppy boot? [no]:

Checking the default route…

You have a single network interface called eth0 (default route) . You will not be able to access the simulator from this Linux host. If this interface is marked DOWN in ifconfig then your simulator will crash.

Which network interface should the simulator use? [default]:

Your system has 1803MB of free memory. The smallest simulator memory you should choose is 110MB. The maximum simulator memory is 1763MB.

1. What is iSCSI?

It is a network storage protocol above TCP/IP. This protocol encapsulates SCSI data into TCP packets. iSCSI allows us to connect a host to a storage array via a simple Ethernet connection (tape drive). This solution is cheaper than the Fibre Channel SAN (Fibre channel HBAs and switches are expensive). From the host view the user sees the storage array LUNs like a local disks. iSCSI devices should not be confused with the NAS devices (for example NFS). The most important difference is that NFS volumes can be accessed by multiple hosts, but one iSCSI volume can by accessed by one host. It is similar to SCSIi protocol: usually only one host has access to one SCSI disk (the difference is the cluster enviroment). The iSCSI protocol is defined in the RFC3720 document by the IETF (Internet Engineering Task Force).

Some critics said that iSCSI has a worse performance comparing to Fibre Channel and causes high CPU load at the host machines. I think if we use Gigabit ethernet, the speed can be enough. To overcome the high CPU load, some vendors developed the iSCSI TOE-s (TCP Offload Engine). It means that the card has a built in network chip, which creates and computes the tcp frames. The Linux kernel doesn’t support directly this and the card vendors write their own drivers for the OS.

The most important iscsi terms:

Initiator:

The initiator is the name of the iSCSI client. The iSCSI client has a block level access to the iSCSI devices, which can be a disk, tape drive, DVD/CD writer. One client can use multiple iSCSI devices.

Target:

The target is the name of the iSCSI server. The iSCSI server offers its devices (disks, tape, dvd/cd … etc.) to the clients. One device can by accessed by one client.

Discovery:

Discovery is the process which shows the targets for the initiator.

Discovery method:

Describes the way in which the iSCSI targets can be found.The methods are currently available:

Internet Storage Name Service (iSNS) – Potential targets are discovered by interacting with one or more iSNS servers.

SendTargets – Potential targets are discovered by using a discovery-address.

SLP – Discover targets via Service Location protocol (RFC 4018)

Static – Static target adress is specified.

iSCSI naming:

The RFC document also covers the iSCSI names.The iSCSI name consists of two parts: type string and unique name string.

The type string can be the following:

iqn. : iscsi qualifiled name

eui. : eui-64 bit identifier

Most of the implementations use the iqn format. Let’S see our initiator name: iqn.1993-08.org.debian:01.35ef13adb6d

iqn : we use iSCSI qualified name adress.1993-08 : the year of the month on which the naming authority acquired the domain name which is used in the iSCSI name.org.debian : reversed dns name which defines the organizational naming authority.01.35ef13adb6d : this string is defined by the naming authority.

Our target name is similar (iqn.1992-08.com.netapp:sn.84211978). The difference is that contains the serial number of Netapp filer. Both names are user editable (initiator,target). We need also two ip adresses for the the target and for the initator, too.

The following figure shows our demo environment. It consists of one Debian host which is the iSCSI initiator, and accesses the
iSCSI disk via /dev/sdb device. The Netapp filer is our iSCSI target device, which offers /vol/iscsivol/tesztlun0 disk or lun for the Debian Linux host. The iSCSI session consists of login phase, then the data exchange phase.

2. iSCSI support on other Unix platforms

The Cisco iSCSI Driver is one of the earliest software iSCSI initiator implementations.This driver supports all of the major commercial Unix systems and their versions (HPUX:10.20,11,11i, AIX:4.3.3,5.1,5.2, Solaris: 2.6,7,8,9). The earliest release can be dated back to 2001. Currently each Unix vendor implements its own driver, and we investigate these drivers.

Solaris:

Multiple sessions to one target support: this feature enables that one client can create more iSCSI sessions to one target as needed, and it increases the performance.

Multipathing: with the help of Solaris Mpxio or IPMP feature we can create redundant paths to the targets.

2 Tb disks and CHAP authentication are also supported. The Solaris driver can use the three discovery methods (SLP can’t). iSCSI disks can be
accessed by the format program.

HPUX:

HP supported the iSCSI from the HP11i v1 os. This driver can discover the targets via SLP (Service Location Protocol) which is also defined by IETF (RFC 4018). This means that the iSCSI initiator and targets register themselves at the SLP Directory agent. After the registration the iSCSIi initiator queries only the Directory agent. HPUX driver implements all of the discovery methods. The CHAP authentication is also implemented and the OS multipath tools (PVLinks) also supported. The HPUX driver provides transport statistics, too.

AIX:

From 5.2 AIX supports iSCSI.The driver implements the static target discovery only.We can use the iSCSI disks with AIX multi pathing called MPIO. The CHAP authentication is also supported.

None of the drivers allows us to boot from iSCSI. This can be a next step in the driver development.

3. iscsi Linux implementations

Initiator implementations:

Cisco also released Linux driver but it is quite old.

The Intel iSCSI implementation contains both target and initiator drivers and a handy tool for generating workloads.

UNH-iSCSI is an initiator and target implementation of the University of New hampshire.

The Open-iSCSI project is the newest implementation.It can be used with 2.6.11 kernels and up. We will test this driver with the Debian host. It contains kernel modules and an iscsid daemon.

The iscsid can be started with the following command:

/etc/init.d/open-scsi start

The iSCSI operations can be controlled with the iscsiadm command. The command can discover the targets, login/logout to the target, and displays the session information.

The configuration files are under the /etc/iscsi directory:

iscsid.conf: Configuration file for the iscsi daemon. It is read at startup.

initiatorname.iscsi: The name of initator, which the daemon reds at the startup.

The storage array manufacturers offer also a native support for iSCSI (EMC, Netapp, etc.).

We have chosen Netapp FAS filer for the testing, but you can test it with a free software. There is a link at the bottom of the article which shows how can we do it with Openfiler.

4. Setting up the iSCSI Linux demo environment

Our demo environment contains one Debian Linux host and one Netapp filer. The Debain host is the initiator, and the Netapp filer is the target.

The setup process is the following briefly:

We should set up the tcp/ip connection between Debian and Netapp filer. The initiator and target must ping each other. We assume that the open-iscsi package is already installed on Debian.

The Debian host must discover the Netapp targets. It is called a “discovery” process. Then the target sends the target lists.

The target must enable to the initator to access the LUN. On the Netapp side, it means that we should create one initiator group, which is a logical binding between the hosts and the luns. The initiator group contains the lun and one Debian host which can access this lun.

When the initator gets the target lists, it must “login” to the target.

When the “login” process completes successfully and Netapp filer allows the access, the initiator can use the iSCSI disk as the normal disk. It appears under /dev/sdx devices and you can format, mount it like the normal disk.

5. We can use it as the normal disk. You can create one partion, and you can easily mount it.

debian:~# fdisk /dev/sdb

debian:~# mkfs /dev/sdb1 ; mount /dev/sdb1 /mnt

If you want to use sdb after the next reboot, you should change the following entry:

node.conn[0].startup = manual to automatic

in the /etc/iscsi/nodes/<iscsi target name>/<ip address> file. After you change it the iSCSI daemon will login to this target. Adding an automatic mount entry (/dev/sdb1 /mnt) in the /etc/fstab file doesn’t work, because the open-iscsi daemon will start later than the mounting of filesystems. One simple script can solve this problem, which does the automatic mounting after the iSCSI daemon starts.

The open-iscsi initiator implementation tolerates network errors well. If you disconnect the Ethernet cable and connect it again, you must start the io process again, but the reconnection occurs automatically.

Another good solution is for the network failures, if you create multiple paths for the one LUN (For example: /dev/sdb, /dev/sdc), the initator logs in to two locations (two RAID controllers) and you make the two disks as a single logical disk using Linux multipath software (dmsetup).

I recommend another alternative for iSCSI target implementation: Openfiler (if you cant test on the Netapp box). It is a free Linux based NAS sofware, which can be managed with a web based GUI.

The iSCSI setup process is quite similar in the case of other Unix implementations.

5. Summary and results

iSCSI is a good solution for a cheap disaster recovery site.You shouldn’t buy an expensive Fibre Channel card at the disaster recovery site, you can use the Ethernet and iSCSI. You can also use it for connecting hosts to disk arrays without Fibre Channel host adapters (if the arrays are iSCSI capable).

During the test I ran the Debian host in the Vmware player program, and my network connection was 100 Mbit/s. I cannot reach more than 15 MB/s read/write performance but it isn’t relevant. With Gigabit Ethernet you can reach much better performance, the only drawback is that it increases the CPU load (CPU must build and compute TCP frames).

redhat es5中安装使用netapp simulator

redhat es5中安装使用netapp simulator
1、Linux中安装netapp-simulator
netapp-simulator的安装过程并不复杂，可以直接按照提示一步步进行安装。
安装完成后，执行以下命令
Password:
netapp01> Wed Mar 1 00:08:51 IST [console_login_mgr:info]: root logged in from console
//*This will create an aggregate of 8 GB comprising of 8 disks each of 1 GB.
It will take some while(around 15 mins) to initialize the disks. *//netapp01> aggr create aggr1 -t raid4 8
Creation of an aggregate with 8 disks has been initiated. The disks need
to be zeroed before addition to the aggregate. The process has been initiated
and you will be notified via the system log as disks are added.
//*Below you can view aggregate is getting initialized.
You can take a coffee break here !!!*//netapp01> aggr status aggr1
Aggr State Status Options
aggr1 creating raid4, aggr snapshot_autodelete=off,
initializing lost_write_protect=off
Volumes:
Plex /aggr1/plex0: offline, empty, active
//*Once the aggregate is initialized following would be the status.*//netapp01> aggr status aggr1
Aggr State Status Options
aggr1 online raid4, aggr
Volumes:
Plex /aggr1/plex0: online, normal, active
RAID group /aggr1/plex0/rg0: normal
//*You can disable snaps creatation to maximize the performance.*//netapp01> aggr options aggr1 nosnap on
netapp01> aggr status aggr1
Aggr State Status Options
aggr1 online raid4, aggr nosnap=on
Volumes:
Plex /aggr1/plex0: online, normal, active
RAID group /aggr1/plex0/rg0: normal
//*Now you create the volume(vol1) of 2GB. This can dynamically extended as per your requirement.
2GB would be sufficient for creating the seed database.*//netapp01> vol create vol1 aggr1 2g
Creation of volume ‘vol1’ with size 2g on containing aggregate
‘aggr1’ has completed.
//*Status of the volume can be checked with following command.*//netapp01> vol status vol1
Volume State Status Options
vol1 online raid4, flex create_ucode=on,
convert_ucode=on
Containing aggregate: ‘aggr1’
netapp01> lun setup
This setup will take you through the steps needed to create LUNs
and to make them accessible by initiators. You can type ^C (Control-C)
at any time to abort the setup and no unconfirmed changes will be made
to the system.
Do you want to create a LUN? [y]: y
Multiprotocol type of LUN (image/solaris/windows/hpux/aix/linux/netware/vmware)
[image]: linux
A LUN path must be absolute. A LUN can only reside in a volume or
qtree root. For example, to create a LUN with name “lun0” in the
qtree root /vol/vol1/q0, specify the path as “/vol/vol1/q0/lun0”.
Enter LUN path: /vol/vol1/lun0
A LUN can be created with or without space reservations being enabled.
Space reservation guarantees that data writes to that LUN will never
fail.
Do you want the LUN to be space reserved? [y]: y
Size for a LUN is specified in bytes. You can use single-character
multiplier suffixes: b(sectors), k(KB), m(MB), g(GB) or t(TB).
Enter LUN size: 2g
You can add a comment string to describe the contents of the LUN.
Please type a string (without quotes), or hit ENTER if you don’t
want to supply a comment.
Enter comment string: oradata01
The LUN will be accessible to an initiator group. You can use an
existing group name, or supply a new name to create a new initiator
group. Enter ‘?’ to see existing initiator group names.
Name of initiator group []: ora
Type of initiator group ora (FCP/iSCSI) [iSCSI]: iSCSI
An iSCSI initiator group is a collection of initiator node names.Each
node name can begin with either ‘eui.’ or ‘iqn.’ and should be in the
following formats: eui.{EUI-64 address} or iqn.yyyy-mm.{reversed domain
name}:{optional string composed of alphanumeric characters, ‘-‘, ‘.’
and ‘:’}
Eg: iqn.2001-04.com.acme:storage.tape.sys1.xyz or eui.02004567A425678D
You can separate node names by commas. Enter ‘?’ to display a list of
connected initiators. Hit ENTER when you are done adding node names to
this group.
Enter comma separated nodenames: iqn.1987-05.com.cisco:calvin
Enter comma separated nodenames: iqn.1987-05.com.cisco:hobbes
Enter comma separated nodenames:
The initiator group has an associated OS type. The following are
currently supported: solaris, windows, hpux, aix, linux, netware, vmware
or default.
OS type of initiator group “ora” [linux]:
The LUN will be accessible to all the initiators in the
initiator group. Enter ‘?’ to display LUNs already in use
by one or more initiators in group “ora”.
LUN ID at which initiator group “ora” sees “/vol/vol1/lun0” [0]:
LUN Path : /vol/vol1/lun0
OS Type : linux
Size : 2g (2097152000)
Comment : oradata01
Initiator Group : ora
Initiator Group Type : iSCSI
Initiator Group Members : iqn.1987-05.com.cisco:calvin
: iqn.1987-05.com.cisco:hobbes
Mapped to LUN-ID : 0
Do you want to accept this configuration? [y]: y
Do you want to create another LUN? [n]: n
//*Check the status of the iscsi service on the NetApp Filer.*//netapp01> iscsi status
iSCSI service is not running
//*Start the service on the NetApp Filer.*//netapp01> iscsi start
Wed Mar 1 23:13:42 IST [iscsi.adapter.online:notice]: ISCSI: iswta, Adapter brought online.
iSCSI service started
Wed Mar 1 23:13:42 IST [iscsi.service.startup:info]: iSCSI service startup
//*Recheck the status of the iscsi service.*//netapp01> iscsi status
iSCSI service is running
//*Check the status of the LUN on the NetApp filer.*//netapp01> lun show
/vol/vol1/lun0 1.6g (1712324608) (r/w, online, mapped)
2、在客户端连接iscsi
在redhat es5版本中连接iscsi与之前版本有较大差异：
# rpm -ivh iscsi-initiator-utils-6.2.0.742-0.5.el5.rpm
(RHEL5的光盘（1/5）中有这个文件)
# chkconfig iscsid start
# /etc/init.d/iscsid start
# iscsiadm -m discovery -t sendtargets -p 192.168.122.1:3260
172.16.122.1:3260,1 iqn.2001-04.com.example:storage.disk2.sys1.xyz
(iqn.2001……….这部份应该和iscsi服务器中ietd.conf中的标记一样)
# iscsiadm -m node -T iqn.2001-04.com.example:storage.disk2.sys1.xyz -p 172.16.122.1:3260 -l
(以上两个iscsiadm的操作只在第一次连接iscsi服务器时要执行，以后每当iscsid启动都会自动连接)
# fdisk -l
(应该可以看到多出来一个/dev/sdx的分区)
明天要弄明白的是：如果两台机器同时连上了之后，oracle rac是如何安装的呢？

Parallel NFS (pNFS) is an extension to NFS v4 that allows clients to access storage devices directly and in parallel thus eliminating the scalability and performance issues associated with NFS servers in deployment today. This is achieved by the separation of data and metadata, and moving the metadata server out of the data path.

There are two options in the “exportfs” command.One is “root=”, the other is “anon=”.

1.If you use “root=<hostname or ip address>” option in the “exportfs” command, then the filer will assign UID 0 to the root user of the client host which is included in the “root=<>” option.

2.If you don’t use the “root=” option while exportfs,but you use “anon=501”, then the filer will assign the UID 501 to the root user logined from eny clients

3.If you use “root=” and also use “anon=” at the same time, the root user who login from the host included in the “root=” option will be assigned with the UID 0,and the root user who login from clients not included in the “root=” will be assigned with UID <anon>.

4.If you don’t use the “root=” and “anon=” options , the root user of all clients will be assigned with UID 65534

5.all none-root user will be assigned with the original UID of the client’s UID in the filer.

options 中NFS版本相关项目

options nfs.v2.df_2gb_lim

Causes the storage system to return replies to the “file system statistics” NFS v2 request that shows no more than (2**31)-1 (or 2,147,483,647) total, free, or available bytes (i.e., 2 GB) on the file system.

options nfs.v3.enable

When enabled, the NFS server supports NFS v3. Disable this option if there is a problem with some client when using NFS v3, and that client cannot be configured to use NFS v2. Valid values for this option are on (enabled) or off (disabled).

options nfs.v4.enable

When enabled, the NFS server supports NFS v4. NFS v4 support is only over the TCP protocol. Valid values for this option are on (enabled) or off (disabled).

options nfs.v4.id.domain

This option controls the domain portion of the string form of user and group names as defined in the NFS v4 protocol. The domain name is normally taken from the NIS domain in use, or otherwise from the Domain Name System (DNS). However if this option is set, it will override this default behavior. When the option nfs.webnfs.enable is ‘on’, the NFS server supports WebNFS lookups. Valid values for this option are on (enabled) or off (disabled).

options nfs.v4.read_delegation

Read delegations allow NFS v4 clients to do read operations locally without contacting the server. These include open for read, read locks, and file read operations. Both the server and client must support read delegations for this feature to work. When enabled, read delegations are supported for NFS v4. This feature is not supported for NFS v2 or v3.

options nfs.v4.write_delegation

Write delegations allow NFS v4 clients to do write operations locally without contacting the server. These include open for write, write locks, and writing to files. Both the server and client must support write delegations for this feature to work. When enabled, write delegations are supported for NFS v4. This feature is not supported for NFS v 2 and v3. Valid values for this option are on (enabled) or off (disabled).

The sysstat command is used to report storage system aggregate performance statistics. The report includes the current CPU utilization, amount of network input/output traffic, disk I/O, and tape I/O. When invoked without any arguments, the printout is repeated every 15 seconds.

Examples

sysstat 1

Displays the default output every second; requires control-C to terminate.

sysstat -s 1

Displays the default output every second; upon control-C termination, prints out the summary statistics.

sysstat -c 10

Displays the default output every 15 seconds, stopping after the 10th iteration.

sysstat –u –c 10 –s 2

Displays the utilization output format every 2 seconds, stopping after the 10th iteration; upon completion, prints out the summary statistics.

sysstat –x –s 5

Displays the extended (full) output every 5 seconds; upon control-C termination, prints out the summary statistics

这些命令都有众多的参数，看一个人是否有丰富的经验也可以通过对参数的灵活应用来看哈。

平时用的不多也记不住，用的时候man看一下就是了。

NFS的性能调整（七）

nfsstat

The nfsstat command displays statistical information about the NFSand remote procedure call (RPC) interfaces to the kernel. It can also be usedto reinitialize this information. If no options are given the default is

NetApp>nfsstat, whichwillrequest the listing of every statistical data

The following table describes the output from the nfsstat command as server RPC displaysfields with separate values for TCP and User Datagram Protocol (UDP):

Header

Description

calls

Total RPC calls received

badcalls

Total number of calls rejected by the RPC layer

nullrecv

Number of times an RPC call was not available even though it was believed to have been received

badlen

Number of RPC calls with a length shorter than that allowed for RPC calls

xdrcall

Number of RPC calls whose header could not be decoded by External Data Representation (XDR)

The server NFS display shows the number ofNFS calls received (calls) and rejected (badcalls) and the countsand percentages for the various calls that were made.

nfsstat 同样可以带很多不同的参数，其中最有用的是 nfsstat -d

Command to Decode Mount and Export problemsThe nfsstat –d

command displays reply cache statistics as well as incomingmessages, including allocated mbufs. This diagnostic option allows fordebugging of all NFS-related traffic on the network.

NFS的性能调整（八）

ifstat

Displays device-level statistics for network interfaces

Syntax ifstat [ -z ] -a | interface_name

The ifstat command displays statistics about packets received and sent on a specified network interface or on all network interfaces. The statistics are cumulative since the storage system was booted.

The -z argument clears the statistics. The -a argument displays statistics for all network interfaces including the virtual host and the loopback address. If you don’t use the -a argument, specify the name of a network interface.

nfs_hist

nfs_hist is a priv set advanced command that displays information on the processing time for NFS requests which could not be processed immediately by WAFL (i.e., they were suspended at least once). The information is given in the form of a delay time distribution, which gives the number of messages of a given type that were delayed for that amount of time (in milliseconds).

In addition, delay distributions are given, which indicate the time various messages spent waiting to be processed in the system. Separate distributions are given for messages waiting on a queue to be processed by WAFL and those waiting on a queue to be processed by all other processes. While these delay distributions include non-NFS messages, they may be helpful in understanding how the system is working when one is attempting to understand NFS performance issues.

The information displayed includes messages processed since the delay distributions were last zeroed with the -z option (or since reboot if the delay distributions have not been zeroed).

netdiag

Performs network diagnostics

Syntax netdiag [ -s|v|d ] [ -nbate ] [ -p [ -I interface ] ]

The netdiag command analyzes the statistics continuously gathered by the network protocol code and (if required) performs various tests to ensure the sanity of operation of the protocol code. It displays the results of the analysis (and any tests performed) along with suggested remedial actions (if any problems are found). It analyzes almost all of the statistics displayed by the many forms of the netstat command for aberrant values.

The first form presented allows the user to specify what subset(s) of the networking subsystem to address and what kind of output to produce. The various options that influence this command form are described in detail below.

A Windows workgroup is a simple, logical group of networked machines (computers) that share resources, such as folders and files.

Each machine has its own Security Accounts Manager database (for Windows NT) or a local security database (for Windows 2000 or later) that is used to perform user authentication and user authorization.

Each user that wants to access resources on a machine must have a user account on that machine.

貌似WIN2K安装好了默认的工作组名字叫”WORKGROUP”.

Joining a Workgroup

A machine “joins” a workgroup by broadcasting its identity to machines on the same subnet. Machine-C joins the workgroup by broadcasting its identity to Machine-A and Machine-B. The master browser in Machine-A responds by capturing the broadcast, updating the master browse list that contains all workgroup machine names, and then broadcasting to all workgroup machines that the updated list is available.

There is a delay (up to 15 minutes) from when the master browser receives the new identity and broadcasts to the workgroup. Workgroup Machine-B and Machine-C then pull the updated master browse list to their local machine browse list. A user can find other machine names on the subnet in the browse list.

Machine-Name Resolution in a Workgroup

How does workgroup machine-name resolution work?

A user broadcasts a name query with the requested machine name to the other machines on the subnet. Machine-C broadcasts a query for the IP address of Machine-B. Machine-B responds to the name query by broadcasting its IP address to Machine-C.

User Authentication in a Workgroup

Users are added locally to each machine.Local-user accounts are created with user names, passwords, group information,and user rights. Machine-C user has a local-user logon.

User authentication is performed locallyon each machine. The local-user logon requires a user name and password. Usersession authentication is performed with the user name and password. When auser wants to access another machine for resources (as in a client-serverrelationship), successful user authentication establishes a session.

Machine-B user requests user sessionauthentication with Machine-C. Machine-C authenticates Machine-B user by usinghis user name and password found in the Machine-C local-user account. AfterMachine-B user successfully authenticates, a session is established withMachine-B user and Machine-C.

Storage System Joins a Workgroup

When a storage system “joins” aworkgroup, it becomes a server that provides services to clients. In thisexample, the storage system broadcasts its identity to machines on the samesubnet.

The master browser captures the storagesystem machine-name broadcast, updates the master browse list, and broadcaststo all workgroup machines that the updated list is available.

Workgroup machines pull the updatedmaster browse list to their local machine. The storage system does not pull themaster browse list because its role is always a server and it does not need tofind other machines.

User Authentication on a Storage System in a Workgroup

Users (local-user accounts) are added toa storage system and user authentication is performed locally on the storagesystem. User session authentication with a user name and password authenticatesa user in order to establish a session with the storage system.

Data access on a storage system requiresa network logon to the storage system. A user can administer a storage systemthrough the network (for example, a Telnet session) using a local account onthe storage system; however, a user cannot logon locally to a storage system toaccess data.

Machine-B user requests user sessionauthentication with the storage system. The storage system authenticatesMachine-B user by using his user name and password found in the storage systemlocal-user account. After Machine-B user successfully authenticates, a sessionis established with Machine-B user and the storage system.

Authenticated users can browse a storagesystem for available resources, but must be authorized to access a share andresources in a share.

Workgroup Disadvantages

The disadvantages of a workgroup are the following:

A user must have a user account on each machine where he wants to gain access.

Any changes to a user account (for example, passwords) must be made on each machine in the workgroup.

Machines that join or leave (for example, machine shutdown) a workgroup must be broadcast by the master browser.

There is a delay (up to 15 minutes) from when the master browser receives a machine broadcast and then broadcasts the availability of the updated list to the workgroup.

A browse list cannot span subnets.

The reliance on broadcast messaging is what restricts workgroups to the local subnet because broadcast traffic is not passed to remote subnets.

What Is a Domain?

A Windows domain is a logical group ofnetworked machines that share a central directory database located on a domaincontroller.

The domain controller centralizes themanagement of a user’s access to the network including:

User logon

User authentication

User group information foraccess to directory and shared resources.

Machine-Name Resolution in a Domain

Machine names and their IP addresses areadded to (or registered with) a machine-name-resolution server. This may beperformed dynamically or manually by a system administrator.

The machine-name-resolution serverresolves machine names to IP addresses. When a user wants to access a share orresources on a machine, he sends the machine name to themachine-name-resolution server and the server returns the IP address of themachine.

Machine-B user wants to access anothermachine and sends the machine name query to the Machine-Name Resolution Server.This server sends the resolved IP address of the machine to the Machine-B user.

Joining a Domain

Machine-B joins a domain by going to adomain controller. The domain controller adds the machine account with themachine name to the directory database.

User Authentication in a Domain

Users are added to the central directory database in the domain controller for user logon and user session authentication.

User authentication is performed on the domain controller. The domain controller checks the directory for the user name and password to authenticate a user when a user requests the following:

User logon access (using the domain-user account).

User session authentication to establish a session.

Machine-B user requests user logon access to Machine-B. The domain controller checks the Machine-B user name and password in the domain-user account. When the user information authenticates successfully, then Machine-B user has logon access to Machine-B.

Machine-A user requests user session authentication for a session with Machine-B. Machine-B goes to the domain controller to authenticate Machine-A user. The domain controller successfully authenticates Machine-A user and a session is established with Machine-A user and Machine-B.

Typical Machines in a Domain

A typical Windows domain has the following types of machines:

Clients: Machines that request services of a server.

Member servers: Windows servers that are not configured as domain controllers, but are members of the domain and configured to provide resources to clients.

Domain controllers (DCs): Windows servers configured to store and maintain a copy of a directory that has the following:

Machine accounts (machine names) registered with the domain.

User names and passwords for authenticating domain users.

Domain machine-name-resolution servers: Windows servers configured to resolve machine names to IP addresses. They provide the following types of services:

WINS (Windows Internet Naming Service) for Windows NT 4.0 domains.

DNS (Domain Name System) for Windows 2000 (or later) domains.

Storage System Joins a Domain

When a storage system joins a domain, it becomes a member server that provides services to clients. The storage system (member server) goes to a domain controller and the domain controller adds the machine account to the directory database.

User Authentication on a Storage System in a Domain

Domain users (already added to the domain controller) can browse the storage system for available shares and then request access to the storage system and its shares and resources in a share.

User session authentication with a user name and password is performed centrally on the domain controller; this establishes a user session with the storage system. Users must be authorized to access a share and resources in a share.

Data access on a storage system requires a network logon to the storage system. A user can administer a storage system through the network (for example, a Telnet session) using a local account on the storage system; however, a user cannot logon locally to a storage system to access data.

Client-B user requests user session authentication with the member server (storage system). The member server goes to the domain controller to authenticate Client-B user. The domain controller authenticates Client-B user and a session is established with Client-B user and the member server (storage system).

Storage System Joins Windows NT 4.0 Domain 古董??

1. The storage system registers with the Windows Internet Naming Service (WINS) server.

2. The storage system asks WINS for the name and IP address of the Primary Domain Controller (PDC).

3. The system administrator pre-creates a machine account for the storage system on the PDC before the storage system joins the domain.

4. The storage system goes to the domain controller and joins the domain.

5. The domain controller returns to the storage system its credentials.

The storage system acts as a domain member server.

Storage System Joins Windows 2000 (or later) Domain

1. The storage system registers with the Domain Name System (DNS) server or a system administrator can manually add it.

2. The storage system asks the DNS for the name and IP address of the following; domain controller in its domain, LDAP server, and Kerberos? server.

3. The storage system goes to the domain controller and joins the domain.

4. The domain controller returns to the storage system its credentials.

The storage system acts as a domain member server.

User Authentication on a Storage System in a Windows 2000 (or later) Domain

The storage system acts as a domain member server. The storage system asks the DNS server to locate the domain controller in its domain. The domain controller performs the Active Directory user authentication using Kerberos. Kerberos V5 is an Internet standard security protocol for handling authentication of a user or system identity.

Both WINS and DNS can run at the same time in a Windows 2000 (or later) Active Directory-based domain.

The older Windows clients rely on WINS only to resolve machine names. These are NetBIOS-based clients: Windows 95, Windows 98, and Windows NT.

Non-Windows Workgroup

A non-Windows workgroup is a logical group of networked machines that share resources with Windows client users; the networked machines are neither members of a Windows workgroup nor a Windows domain.

This server environment is also called:

UNIX clear text password workgroup

/etc/passwd-style workgroup

这个说法只看到NETAPP的资料里是这样

Storage System as a Non-Windows Workgroup Server

When a storage system becomes a non-Windows workgroup server, it provides services to clients. An example is an all-UNIX work environment with many UNIX workstations and a few Windows clients with users that need CIFS resources. Note that any UNX reference also includes LINUX.

Servers functioning in the role of a directory store for user information (user names, passwords, and group information):

Storage system’s local /etc/password file

Network Information Services (NIS) server

Lightweight Directory Access Protocol (LDAP) server

Servers that can provide machine (host) name resolution:

Storage system’s local /etc/hosts file

NIS server

Domain Name System (DNS) server

NIS Server vs. LDAP and DNS Servers

An NIS server provides machine-name resolution to IP addresses, and user names, passwords, and group information for user authentication and authorization.

In an LDAP and DNS server environment, a DNS server provides machine-name resolution to IP addresses, and an LDAP server provides user names, passwords, and group information for user authentication and authorization.

NOTE: An LDAP server requires a DNS server to handle machine name resolution.

用NIS一个就行,LDAP要和DNS搭配才行

User Authentication on a Storage System in a Non-Windows Workgroup

The storage system performs user authentication with user information.

User and group information can be added to one or both of the following:

Locally to the storage system /etc/passwd file (user names and passwords) and /etc/group (group information).

Centrally to an NIS or LDAP server.

The /etc/nsswitch.conf file sets the order of precedence for where a storage system goes to search for user and group information.

三种方式不同的优点

Workgroup Advantages

Does not require a machinerunning a Windows server to hold centralized security information.

Saves money by not having tobuy Microsoft server licenses.

Simple to design and implement.

Convenient for a limited numberof machines in close proximity.

Storage system access limitedto 96 local clients.

Domain Advantages

Centralized administration ofall user information.

User information is modified ina central location rather than being replicated throughout the environment.

A single authentication sitefor all domain account logons for local and user session authentication; a usermust also have permission to access network resources.

More scalability because ofcentralized administration.

Non-Windows Workgroup Advantages

In an all-UNIX environment,CIFS can be licensed on a storage system with CIFS shares made available to thefew Windows client users.

User authentication can beperformed with existing authentication mechanisms in a UNIX environment, usingan NIS or LDAPserver, or locally in the /etc/passwd file as a directory store for user andgroup information.

Machine name resolution can behandled by an NISor DNS server.

CIFS Features in Windows Workgroups and Domains

The following are some CIFS features available in a Windows workgroup and domain:

Network browsing to locate machines within a domain or workgroup (provided by a browse list) and shares that are available on each machine (provided by that machine).

This example demonstrates client-server communications for session, share access, and file authorization. The following are the basic steps.

1. The client contacts the server and requests CIFS dialect.

2. The server responds with supported CIFS dialect and next logon step.

3. The client responds with username and password.

4. The server sends a UID (User ID) if the username and password are accepted or an error if not accepted.

5. The client requests access to a share.

The storage system caches all security IDs (SIDs) and usernames received from the domain controller at boot time.

6
The server responds with a treeID to the requested share (if access is allowed).
7.
The client requests to open afile on a share.
8.
If access is allowed, theserver responds with the ID of the requested file.
9.
The client requests that theserver read the data and return its contents.
10.
The server sends the requesteddata.
During this process, the Access ControlLists (ACLs) are checked for permissions.

Using a lookup service, LDAP (Lightweight Data Access Protocol), NIS (Network Information Service), or a local storage system, Data ONTAP determines who is trying to access the resource, and verifies that the permission list indicates operations that the user may perform.

Files Created to Support the CIFS Environment

Files Created to Support the CIFS EnvironmentDuring the CLI cifs setup script or FilerView CIFS Setup Wizard, CIFS support and configuration files are created in the /etc directory. The number and content of the files are dependent on the environment.

The following are files that are common to all environments:

/etc/cifsconfig_setup.cfg (stores the CIFS setup configuration)

/etc/usermap.cfg (multiprotocol support for mapping users of NFS and CIFS)

/etc/passwd (multiprotocol and UNIX workgroup)

/etc/cifsconfig_shares.cfg (default shares definitions)

Additional files are created depending on the environment as in a workgroup (Windows/non-Windows) or a Windows domain.

The administration host is given root access to the filer’s /etc files for system administration. To allow /etc root access to all NFS clients enter RETURN below. Please enter the name or IP address of the administration host:

Purpose of a Domain Administrator and Local Administrator Accounts

On the storage system, the domain administrators group and the local administrator account are part of the BUILTIN\Administrators group. They can do the following:

Provide a text editor to edit configuration files.Data ONTAP does not include an editor.

Provide the ability to administer a storage system and hence have access to the root file system (C$ and ETC$).

Modify the share access for C$ and ETC$ to grant additional users access.

The local administrator can setup local users on the storage system with the useradmin user add command.

The -g option specifies that the user is the name of a UNIX group. Use this command when you have

A UNIX group and a UNIX user or an NT user or group with the same name.

Displaying CIFS Shares

R2*> cifs shares

Name Mount Point Description

—- ———– ———–

ETC$ /etc Remote Administration

BUILTIN\Administrators / Full Control

HOME /vol/vol0/home Default Share

everyone / Full Control

C$ / Remote Administration

BUILTIN\Administrators / Full Control

Creating a Share

When you create a share, you must provide these items:

The complete path name of an existing folder, qtree, or volume to be shared

The name of the share entered by users when they connect to the share

The permission for the share

Optionally, a description of the share

When creating a share from the Data ONTAP CLI, you can specify a variety of share properties, including group membership for files in the share, support for wide symbolic links, and disabling of virus scanning when files in the share are first opened.

Virus scanning occurs when files are opened, renamed, and closed after being modified.

After you have created a share, you can specify these share properties:

Maximum number of users who can simultaneously access the share

If you do not specify a number, the number of users is limited by storage system memory.

The share name ‘testshare’ will not be accessible by some MS-DOS workstations

Are you sure you want to use this share name? [n]:y

R2*> cifs shares

Name Mount Point Description

—- ———– ———–

ETC$ /etc Remote Administration

BUILTIN\Administrators / Full Control

HOME /vol/vol0/home Default Share

everyone / Full Control

C$ / Remote Administration

BUILTIN\Administrators / Full Control

testshare /vol/myforvol001 test cifs share command

everyone / Full Control

默认的权限是

Full Control for Everyone

cifs access修改用户

R2*> qtree status

Volume Tree Style Oplocks Status

——– ——– —– ——– ———

vol0 unix enabled normal

myforvol001 unix enabled normal

myforvol001 testshare unix enabled normal

R2*> cifs access testshare Administrator Full Control

1 share(s) have been successfully modified

R2*> cifs shares

Name Mount Point Description

—- ———– ———–

ETC$ /etc Remote Administration

BUILTIN\Administrators / Full Control

HOME /vol/vol0/home Default Share

everyone / Full Control

C$ / Remote Administration

BUILTIN\Administrators / Full Control

testshare /vol/myforvol001 test cifs share command

everyone / Full Control

R2\administrator / Full Control

R2*> cifs access -delete testshare everyone

1 share(s) have been successfully modified

R2*> cifs shares

Name Mount Point Description

—- ———– ———–

ETC$ /etc Remote Administration

BUILTIN\Administrators / Full Control

HOME /vol/vol0/home Default Share

everyone / Full Control

C$ / Remote Administration

BUILTIN\Administrators / Full Control

testshare /vol/myforvol001 test cifs share command

R2\administrator / Full Control

修改后普通用户testcifs就不可以访问testshare这个文件夹了.

CIFS Sessions

R2*> cifs sessions

Server Registers as ‘R2’ in workgroup ‘WORKGROUP’

Filer is using en_US for DOS users

Using Local Users authentication

====================================================

PC IP(PC Name) (user) #shares #files

192.168.222.1() (R2\testcifs – pcuser)

2 0

R2*> cifs sessions *

users

shares/files opened

192.168.222.1() (R2\testcifs – pcuser)

HOME

testshare

R2*> cifs sessions -s testcifs

users

Security Information

192.168.222.1() (R2\testcifs – pcuser)

***************

UNIX uid = 65534

NT membership

R2\testcifs

BUILTIN\Users

User is also a member of Everyone, Network Users,

Authenticated Users

***************

Options

The -t optiondisplays the total count of CIFS sessions, open shares and open files.

If you include the user argument, the command displays information about the specified user, along with the names and access level of files that user has opened. If you use * as the specified user, the command lists all users.

Specifying the -c option with a user argument, will display the names of open directories and the number of active ChangeNotify requests against the directory.

The -s option displays security information for a specified connected user. If you do not specify a user or workstation name, the command displays security information for all users.

Stopping and Restarting CIFS Services

To terminate CIFS service (a complete shutdown) where all CIFS sessions are ended:

You can create user home directories on the storage system and configure Data ONTAP to offer each user a home directory share automatically. Each user can connect to the user’s home directory only, not to the home directories of other users.

这个功能应该来说比较有用.免的用户乱存东西到别人的目录下.

To specify the naming style used for matching home directories to users:

To specify whether members of the storage-system Builtin\Administrators group can connect to the CIFS home directories of other users:

options cifs.homedirs_public_for_admin on

When you create a user’s folder for their home directory, Data ONTAP automatically searches the paths in the cifs_homedir.cfg file for the user name that matches the login name, and dynamically creates the share for that user

Auditing CIFS Events

以前看到过有个贴子问怎么对用户行为做audit,测试了一下.

You can enable auditing for the following categories of events:

Logon and logoff events

File access events

These are the prerequisites for auditing file access events:

The file or directory can be audited in a mixed or NTFS volume or qtree.

If the cifs.audit.nfs.enable option is “on,” you can audit events for files in UNIX security- style qtrees.

You must activate auditing for individual files and directories according to your Windows documentation.

An opportunistic lock (also called an oplock) is a lock placed by a client on a file residing on a server. In most cases, a client requests an opportunistic lock so it can cache data locally, thus reducing network traffic and improving apparent response time. Opportunistic locks are used by network redirectors on clients with remote servers, as well as by client applications on local servers.

Opportunistic locks coordinate data caching and coherency between clients and servers and among multiple clients. Data that is coherent is data that is the same across the network. In other words, if data is coherent, data on the server and all the clients is synchronized.

Opportunistic locks are not commands by the client to the server. They are requests from the client to the server. From the point of view of the client, they are opportunistic. In other words, the server grants such locks whenever other factors make the locks possible.

When a local application requests access to a remote file, the implementation of opportunistic locks is transparent to the application. The network redirector and the server involved open and close the opportunistic locks automatically. However, opportunistic locks can also be used when a local application requests access to a local file, and access by other applications and processes must be delegated to prevent corruption of the file. In this case, the local application directly requests an opportunistic lock from the local file system and caches the file locally. When used in this way, the opportunistic lock is effectively a semaphore managed by the local server, and is mainly used for the purposes of data coherency in the file and file access notification.

默认情况下NETAPP FILER中options cifs.oplocks.enable on是打开的.

但在几种情况下需关闭

Using a databaseapplication whose documentation recommends oplocks be turned off.

On-access virus scanning means that a file is scanned before a CIFS client is allowed to open it.

1. The scanner (Windows server) registers with the storage system, so no storage system configuration is required.

2. At the storage system prompt, type the vscan on command to enable scanning.

3. The scanner waits for requests to come from the storage system.

Several scanners can register with the storage system. This is recommended for performance and reliability.

A single scanner can scan multiple storage systems.

4. The scanner pings the storage system from time to time to detect and recover from reboots and takeovers.

CIFS性能分析工具之cifs stat

The cifs stat command has two main forms.

Ifyou specify the interval, the command continues displaying a summary ofCIFS activity until interrupted. The information is for the preceding intervalseconds. (The header line is repeated periodically.) The interval must be >=1.

If you do not specify the interval, the command displayscounts and percentages of all CIFS operations as well as a number of internalstatistics that may be of use when diagnosing performance and other problems.

Bydefault, the statistics displayed are cumulative for all clients. However, ifthe cifs.per_client_stats.enable option is on, a subset of the clients may beselected using the -u and/or -h options

cifs_stat Options

-u <user> If per-client stats are being gathered, selects a user account to match for stats reporting. More than one -u <user> option may be supplied. If more than one client matches the user, the values reported are the sum of all matching clients.

The user specified may have a domain, which restricts matching to that domain, or the domain may be “*” or left blank to match any domain. The user account may be specified, or may be “*” to match any user.

-h <host> If per-client stats are being gathered, specifies a host to match for stats reporting. More than one -h <host> option may be supplied. If more than one client matches the host, the values reported are the sum of all matching clients.

The host may be an IP address in dot notation, or it may be any host name found using DNS if that is enabled on the storage system.

-v[v] If per-client stats are being reported using the -u or -h options, it may be desirable to know which clients contributed to the total stats being reported. If -v is given, the count of the number of matching clients is printed prior to the stats themselves. If -vv is given, the actual matching clients are also printed prior to printing the stats themselves.

-c Displays counts and percentages for non_blocking CIFS operations as well as block_ing, which is the default. This option is not available in combination with the perclient options.

The cifs top command is used to display CIFS client activity based on a number of different criteria. It can display which clients are generating large amounts of load, as well as help identify clients that may be behaving suspiciously.

The default output is a sorted list of clients, one per line, showing the number of I/Os, number and size of READ and WRITE requests, the number of “suspicious” events, and the IP address and user account of the client. The statistics are normalized to values per second. A single client may have more than one entry if it is multiplexing multiple users on a single connection, as is frequently the case when a Windows Terminal Server connects to the storage system.

This command relies on data collected when the cifs.per_client_stats.enable option is “on”, so it must be used in conjunction with that option. Administrators should be aware that there is overhead associated with collecting the per-client stats. This overhead may noticeably affect the storage system performance.

A filer can be configured for multiprotocol access, or as an NTFS-only filer. Since NFS, DAFS, VLD, FCP, and iSCSI are not licensed on this filer, we recommend that you configure this filer as an NTFS-only filer

The process of creating a CIFS cred from a user ID (UID), or a UNIX cred from a Windows account, always involves checking a user mapping file called /etc/usermap.cfg. The user mapping process allows much flexibility, but it also must be used carefully because it is possible to create confusing scenarios.

Network access to NAS on the storageappliance is typically via TCP/IP.

SAN provides block access to LUNs (logical unit numbers), whichare treated as local disks by both Windows and UNIX-based operating systems. Thestorage system views a LUN as a logical representation of physical storage. Networkaccess to LUNs is via SCSI over Fibre Channel (FCP) or SCSI over TCP/IP (iSCSI).

如果对别的厂家的阵列比较熟悉，从上个帖子的图中就可以发现NETAPP对SAN的支持要比别的厂家多了一个层次wafl。

Initiator & Target

Initiator & Target

主机是发起请求到存储设备，FILER自己访问扩展柜的时候就变成了发起（Initiator)。那个FA的端口的属性是可以改的。

how requests move from the host (initiator) to thestorage system (target). First, an application sends a request to the filesystem. The file system issues Input/Output (I/O) calls to the operatingsystem. The operating system then sends the I/O through its storage stack (SCSIdriver) to issue the SCSI commands. Next, these commands are encapsulated inFibre Channel frames or iSCSI IP packets. Once the request is received by thestorage system target, the Data ONTAP operating system converts requests fromthe initiator. Data ONTAP turns SCSI commands into WAFL operations. WAFL sendsthe request to the ONTAP RAID subsystem where RAID manages the data on thephysical disks where the LUN is located. Once processed, request responses moveback through the FC fabric or iSCSI-based。

WWNN & WWPN

做过SAN的人一定接触过WWN

When discussing World Wide Names it is important to cover themat a high level before going into too much detail. The FC specification for thenaming of nodes and ports on those nodes can be fairly complicated. Each device is given a globally unique WorldWide Node Name (WWNN) and an associated World Wide Port Name (WWPN) for each port on the node. World Wide Namesare 64-bit addresses made up of 16 hexadecimal digits grouped together in twos witha colon separating each pair (e.g. 21:00:00:2b:34:26:a6:54).

Thefirst number in the WWN defines what the other numbers in the WWN represent,according to the FC specification. The first number is generally a 1, 2, or 5.In the example of QLogic initiator HBAs, the first number is generally a 2. ForEmulex initiator HBAs, the first number is generally a 1.

Initially, Fibre Channel (FC) Point-to-Point topologies were seen as a replacement for the parallel SCSI bus, to overcome bandwidth and distance limitations. FC at 100Mb/sec was superior to SCSI at 10–20Mb/sec, and as SCSI progressed to 40, 80, then 160Mb/sec, FC stayed ahead with 200Mb/sec then 400Mb/sec. SCSI bandwidth was reaching a ceiling where FC at 200Mb/sec was just getting started. FC Point-to-Point also overcame the severe distance limitations of SCSI, but one limitation remained: It connected one initiator to one target, supporting only the simplest topology. This provides limited connectivity.

采用这种方式和DAS实际就是一回事了，不过把DAS多用的SCSI的线缆换成了FC的光纤。

FCP Protocol Topologies 续

Fibre Channel Arbitrated Loop (FCAL)

To overcome the connection limitation, the second generation of FC was developed, FC Arbitrated Loop (FCAL). FCAL could connect up to 127 ports on a shared loop using a two-place hexadecimal addressing scheme. Eventually, even this 127-port address was seen as too much of a limitation, and the bandwidth was shared.

In an arbitrated loop, when devices are added or removed, all activity on the loop is disrupted. This occurs because there is a break in the ring. A FC hub may be used to connect devices and therefore bypass failed ports

磁带库多是这种情况接入SAN。

FCP Protocol Topologies

Switched Fabric

The third generation of FC, called Switched Fabric, used a 24-bit addressing scheme with 64-bit WWPN and WWNN. This scheme has a possible 12 million addresses, and the initiator-target pair got a dedicated non-blocking path to ensure full bandwidth.

In this configuration, all devices or loops are connected to FC switches.

最常见的情况。

这里说一下存储内部DAE的采用的多是switch方式，但NETAPP的叫ESH。

图示SAN TOPO

文字描述的东西始终不如一副图看的明白。

NETAPP FILER支持的SAN TOPO

Network Appliance differentiates between three basic SAN topologies which are possible when connecting Network Appliance Storage Appliances and server systems with FC:

A FC channel zone consists of a group of FC ports or nodesthat can communicate with each other. It can be thought of as a logical fabricsubset. Two FC nodes can communicate with one another only when they arecontained in the same zone. A node can be contained in multiple zones. Thereare two types of zoning, “hard” and “soft” zoning.

Hard ZoningHard zoning physically restricts communication in aswitched fabric. Hard zoning is considered secure because it prevents zoningbreaches caused by bypassing the fabric name service.

Soft ZoningSoftzoning limits visibility of Fibre Channel nodes across zones. When a nameserver receives a request from a port, it will only return information fromports within the same zone as the port that generated the request. Althoughdevices are separated at the name service level, soft zones do not preventcommunication between zones. Soft zoning is typically more flexible than hardzoning because this type of zoning does not consume physical resources on theswitch. Soft zones are also considered less secure than hard zones

There are two basic topologies which are supported by NetAppin iSCSI direct-attached and switched environment.

Direct-Attached

In direct-attached, servers (or hosts) are directly attachedto the NetApp controller using a crossover cable. Direct-attach to more thanone controller in an HA Configuration is not possible.

Switched Environment

In a switched environment, servers are attached to NetAppcontrollers through Ethernet switches. This network may consist of multipleEthernet switches in any configuration.

There are two types of switched environments, dedicatedEthernet and shared Ethernet. In a dedicated Ethernet, there is no extraneousnetwork traffic. The network is totally dedicated to iSCSI and relatedmanagement traffic. Such a network is typically located in a secure datacenter. Direct-attached and dedicated Ethernet networks represent approximately90 percent of current iSCSI deployments. In a shared Ethernet, the network isshared with other traffic or a corporate Ethernet network. This typicallyintroduces firewalls, routers, and IPSEC into the Ethernet network.

A volume is the most inclusive of the logical containers. It can store the following:

Files and directories

Qtrees

LUNs

Qtrees

A qtree is a subdirectory of the root directory of a volume. Qtrees can be used to subdivide a volume in order to group LUNs.

LUNs

A LUN is a logical representation of a physical unit of storage. It is a collection of, or a part of, physical or virtual disks configured as a single disk. When you create a LUN, it is automatically striped across many physical disks. Data ONTAP manages LUNs at the block level, so it cannot interpret the file system or data in a LUN. From the host, LUNs appear as local disks, allowing you to format and manage to store data on them.

Security style – The securitystyle determines whether a volume can contain files that use UNIX security,files that use Windows file system (NTFS) file security, or both types of files 这个前面在CIFS的配置的时候讨论过

Option to designate the volumeas a SnapLock volume.把这个卷搞成一次刻录光盘一样,只能写一次

Option to designate the volume as a root volume – All new storage systems with factory-installed Data ONTAP have apre-configured root volume. The root volume is named “vol0” by default.

Space Reservations

Data ONTAP uses space reservation to guarantee that space is available for completing writes to a LUN or for overwriting data in a LUN. When you create a LUN, Data ONTAP reserves enough space in the traditional or FlexVol volume so that write operations to those LUNs do not fail because of a lack of disk space on the storage system. Other operations, such as taking a Snapshot copy or the creation of new LUNs, can occur only if there is enough available unreserved space; these operations are restricted from using reserved space.

Do not create any LUNs in the system’s root volume. Data ONTAP uses this volume to administer the storage system. The default root volume is /vol/vol0. 我看我师弟写的NETAPP安装报告,现在系统是安装在三块盘做的RAID DP上,还保留了一块盘给它做HOTSPARE,这样系统就用了4块盘,也太浪费了.

Ensure that no other files or directories exist in a volume that contains a LUN. If this is not possible and you are storing LUNs and files in the same volume, use a separate qtree to contain the LUNs.

If multiple hosts share the same volume, create a qtree on the volume to store all LUNs for the same host. This is a recommended best practice that simplifies LUN administration and tracking.

有了上面两条,看样子最好建立一个QTREE然后在里面建立LUN比较好.

Ensure that the volume option create_ucode is set to on. (vol options <volname> create_ucode on). Data ONTAP requires that the path of a volume or qtree containing a LUN is in the Unicode format. This option is Off by default when you create a volume. It is important to enable this option for volumes that will contain LUNs.

To simplify management, use naming conventions for LUNs and volumes that reflect their ownership or the way that they are used.

我在网上好像也搜索到过类似的说法,要不系统就用掉4块盘,现在盘都那么大,空间浪费也太惊人了.客户容量下的话估计都会觉的受骗了.

在这点上NETAPP应该像CX学习下,用前几块盘的部分来做就好了.这样也少了对root vol的手动迁移,免的出现些问题.

LUN SIZES

LUN Sizes

The minimum size for a LUN in a Solaris environment is 4MB. The minimum size for a LUN in a Windows environment is 31.5MB. The maximum size for a LUN should not exceed approximately 50 percent of the size of a volume, especially if Snapshot copies of the LUN are desired.

Diagnostic Scripts – Network Appliance Support must have these scripts run on your system before they are able to diagnose a problem.

Documentation – Documentation contains crucial setup information and known problems which will help with setup and troubleshooting.

sanlun utility – The sanlun utility allows you to view information about LUNs and the host HBA. It is only available on UNIX operating systems.

The AIX ODM definition package – Properly identifies LUNs as Network Appliance LUNs. The NetApp ODM package for IBM AIX&reg; is only available in the FCP Host Attach Kit for AIX.

IBM的AIX ODM是要特殊一些,别的厂家做的东西还得写个ODM的东西才能被正确识别.

The set tuneables scripts – The set tuneables script updates registry and WMI values. It is only available for Windows&reg;

SnapDrive for UNIX

SnapDrive for UNIX is a tool that simplifies data backup management so that you can recover should data accidentally be deleted or modified. SnapDrive for UNIX uses Snapshot technology to create an image of the data stored on a storage system attached to a UNIX host. You can then restore that data at a later time.

In addition, SnapDrive for UNIX lets you provision storage on the storage system. SnapDrive for UNIX provides a number of storage features that enable you to manage the entire storage hierarchy, from the host-side application-visible file down through volume manager to the storage-system-side LUNs that provide the actual repository.

Multipathing uses redundant paths between a Windows host and a LUN (virtual disk), thus eliminating the “single-point-of-failure” vulnerability that exists when a host connects to a storage system across a single, fixed physical path. SnapDrive multipathing establishes two or more physicalpaths between the host and the LUN. One of the paths is designated active and the others as passive (standby). If the active physical path fails, a passive(standby) path will take over and continue to maintain connectivity between the host and the LUN.

SnapDrive facilitates multipath redundancy by integrating a Windows MPIO-device-specific module (ntapdsm.sys) with a trio of Microsoft software drivers (mpio.sys, mpdev.sys, and mspspfltr.sys). This multipathing solution, referred to as “Windows MPIO,” is managed through the SnapDrive plug-in under the MMC or the SnapDrive for Windows sdcli.exe command-line utility.

lun create – The lun create command only creates the LUN. It does not create igroups or map LUNs to igroups.

In addition lun create does not add portsets. A portset consists of a group of FCP target ports. You bind a portset to an igroup in order to make the LUN available only on a subset of the storage system’s target porsts. Any host in an igroup can access the LUNs only by connecting to the target ports in the portset.

If an igroup is not bound to a portset, the LUNs mapped to the igroup are available on all of the storage system’s FCP target ports. By using portsets, you can selectively control which initiators can access LUNs and the ports on which they access LUNs.

lun setup – The lun setup wizard is a command line interface (CLI) wizard that walks you through the process of creating and mapping the LUN.

FilerView – FilerView performs the same function as lun setup through a GUI interface. FilerView is provided with the purchase of Data ONTAP.

SnapDrive – SnapDrive was designed specifically for LUN management. If you use SnapDrive, it is recommended that you create and manage all of your LUNs within it.

lun setup 建立LUN过程

R1> lun setup

This setup will take you through the steps needed to create LUNs

and to make them accessible by initiators. You can type ^C (Control-C)

at any time to abort the setup and no unconfirmed changes will be made

HBAnyware – HBAnyware is an HBA management GUI that is provided by Emulex.

SANsurfer – SANsurfer is an HBA and switch management GUI provided by QLogic.

有图形的，也有字符界面的。主要用在LINUX,SUN上做Bind host HBA to WWPN

HBA的配置参数都在一个相应的文件中，存储对这个文件中的参数设置是有要求的，这个需要按照厂家提供的文档去配置。

如何查找HBA的WWPN

实际上很多啦。

可以到交换机上去查。

也可以用系统的命令来查。

甚至可以到存储上去查，当然这个时候zone要是做好了的，或者是直连的。

例如可以在SYMMWIN中的FA口上用inline命令的F8来看。

NETAPP给的方法在下面

AIX and HP-UX hosts

WWPNs identify the physical ports on the HBA. You must supply the WWPNs when you create an FCP type igroup. To determine the WWPN on AIX and HP-UX hosts, use the command sanlun fcp show adapter –c. This displays the command you need to enter to create the igroup. If you do not include –c, the command will only list the HBAs and WWPNs.

Note: Sanlun is a valuable utility available in the Host Utilities kits.

Linux hosts

To find the WWPN for the HBA installed on the Linux host, complete the following steps:

Type modprobe. This command will load the driver.

Example: If your driver is called qla23000, enter /sbin/modprobe qla23000. The system creates a /proc/scsi/qla23000 directory that contains a file for each QLogic HBA port. The WWPN is the file for that port.

Look in each /proc/scsi/qla23000/HBA-port-num file and get the WWPN. The filename matches the HBA port number.

Example: To get the WWPN for port 2, you would look in the file /proc/scsi/qla23000/2.

LINUX的还比较有用。

iSCSI Nodes

说了FC的，现在来说说ISCSI的

iSCSI target nodes can be connected in two ways:

Standard Ethernet interfaces

iSCSI target HBAs

Each iSCSI node must have a node name. This node name functions much like the WWPN does for FCP. There are two possible node name formats. They are listed below.

iqn-type designator

The format of this nodename is conventionally “iqn.yyyy-mm.backward_naming_authority: unique_device_name.” This is the most popular node name format and is the default used by a NetApp storage system. Components of the logical name are the following:

Type designator, iqn, followed by a period (.)

The date when the naming authority acquired the domain name, followed by a period

The name of the naming authority, optionally followed by a colon (

A unique device name

Example: “iqn.1992-08.com.netapp:sn.50400285”

eui-type designator

The format of this nodename is “eui.nnnnnnnnnnnnnnnn.” Components of the logical name are the following:

The type designator itself, “eui”, followed by a period (.)

Sixteen hexadecimal digits

Example: “eui.123456789ABCDEF0”

Filer Commands for Initiators and Targets

Host Initiator HBAs

The command fcp show initiator displays either all initiators, or as in the example above, an initiator of interest. Initiator information that is displayed includes port name and group.

Storage System Target HBAs

The command fcp show adapter displays all adapters, or when a specific FCP port is specified, an adapter port of interest. Information generated on the adapter includes slot, description, adapter type, status, WWNN, and WWPN

simulator不能模拟FC的卡，哪位好心人贴个上面两命令的输出来给大家看看。

Access LUNs on Solaris

这个单独拿出来写，是因为SUN下面认盘相对要麻烦一些。

LUNs created on the NetApp storage system that will be accessed via FCP must be configured on the SUN Solaris? host. Complete the following steps to access your LUNs on Solaris:

Modify the /kernel/drv/sd.conf file: You must add an entry for each target ID in the sd.conf file so the host system knows which disks to probe when the system is rebooted.

Discover LUNs: There are two methods which prompt the system to probe for new devices.

Use the command /usr/sbin/devfadm.

Reboot the host with the reconfigure option (reboot — -r).

Run sanlun lun show to verify that the LUNs are visible.

Use the format utility: The format utility will format and label new LUNs.

Create a UNIX file system on the disk or use it as a raw device.

Note: If your configuration uses Volume Management software, you must configure the LUNs so they are under the control of the Volume Manager.

Host Booting from SAN or IP SAN

SAN Boot Tasks – Windows FCP

To use a LUN as a boot device, you must complete these steps:

Configure the host’s BIOS boot order: If your host has an internal disk, you must enter BIOS setup to configure the host to boot from the LUN. You must ensure that the internal disk is not bootable through the BIOS boot order. Your HBA should be the first device in the boot order.

Enable Boot BIOS on the HBA: BootBIOS enables the HBA to access the existing BIOS. It also enables you to designate an FC drive, such as a storage system LUN, as the host’s boot device. BootBIOS firmware is installed on your HBA. It is disabled by default. If you are using an Emulex HBA, use HBAnywhere or LP6DUTIL.EXE to enable BootBIOS. If you are using an QLogic HBA, use Fast!UTIL to enable BootBIOS.

Obtain the WWPN of the host initiator HBA: The WWPN is required when you create an igroup for the Boot LUN. If you are using an Emulex HBA, use HBAnywhere or BootBIOS to obtain the WWPN. If you are using Qlogic, use SANsurfer or BootBIOS to obtain the WWPN.

Cable the storage system so that only one path exists from the HBA to the boot LUN: MPIO (multipathing) drivers are not installed with the operating system and first boot. When preparing to install the operating system, there are special cabling instructions you must follow to ensure there is only one path from the HBA to the Boot LUN.

Create the Boot LUN: After you obtain the WWPN for the HBA, you must create the LUN to use as a Boot LUN, map it to an initiator group, and assign a LUN ID. You must assign a LUN ID of 0 to a LUN that will be used as a boot device. LUNs with IDs other than 0 are not supported as boot devices.

Configure the BootBIOS to use the Boot LUN: Follow steps that are specific to the BootBIOS of your HBA.

Copy the Emulex or QLogic SAN boot driver from the Web: When you boot from a LUN, you must ensure that the operating system on the LUN has the required HBA driver for booting from a LUN. You must download these drivers from the Emulex or QLogic Web site. During the Windows? (2000, 2003) installation, you must install the driver as a third-party SCSI array driver from a floppy disk.

Install Windows (2000, 2003) on the Boot LUN

Install the FCP Windows Attach Kit software drivers

第一台装好了以后的就简单了。

SAN Boot Tasks – Solaris FCP

Configure the host and storage system with supported firmware: Software and firmware on the host must be supported by the Attach Kit.

Install the host operating system on a local disk: The disk on which the operating system resides must use a file system type that matches that of host.

Download and install OpenBoot firmware on the HBA: OpenBoot firmware is available on the Emulex Web site.

Create the Boot LUN: Use standard storage system commands and procedures to create and map a LUN assigned to LUN ID 0. You must partition the LUN to match the partitions on the host boot device. To do this complete these steps:

Display information about the host boot device

Modify the bootable LUN to model the partition layout of the host boot device

Copy bootblks and boot data: The bootblk contains startup information, required by the Solaris host that you must install onto the raw bootable LUN. Installing the bootblk involves these actions:

Use the uname –a command to determine the directory in /usr/platform where the bootblk is located

Install the bootblk onto the bootable LUN

Copy the boot data to the bootable LUN involves these actions:

Create and mounting a file system on the bootable LUN. The file system must match the file system type on the host boot device.

Move boot data from the host device onto the bootable LUN

Edit the vfstab file to reference the bootable LUN

Modify OpenBoot: OpenBoot is the firmware that the Emulex software uses to startup the system. OpenBoot firmware also includes the hardware-level user interface that you use to configure the bootable LUN. Modifying OpenBoot involves these actions:

Validate the OpenBoot version

Set the topology (point-to-point or arbitrated loop) to the bootable LUN

Bind the HBA to the bootable LUN

Create an alias for the bootable LUN (The alias substitutes for the device address.)

Install the QLogic SANsurfer iSCSI HBA Manager interface on any Windows? computer: This interface will be used to update BootBIOS and firmware. It can also be used to change the HBA configuration settings to support SAN booting. A card can be updated in a computer other than the IP SAN booting host and then placed in that host.

Create a device driver diskette: Create the QLogic device driver diskette for the QLogic card and operating system combination that you are using. During the OS installation, you are prompted to install the device driver from the diskette. This enables the OS to communicate with the HBA.

Disable the primary hard drive: It is only necessary to disable the primary hard drive if the OS installed on the host. To disable the hard drive, use the system BIOS to physically remove the drive. If the OS installer detects a boot.ini file on the primary hard drive, it will not install the required boot files to boot the LUN. This means that if you disable or remove the primary hard drive at a later date, the host will not be able to boot from the LUN.

Make sure the HBA has an IP address

Update the HBA BootBIOS and firmware to theNetApp supported version levels: Before you configure the HBA to boot from theSAN, you may need to flash the Boot BIOS and firmware to the HBA in order to beat NetApp supported version levels. You can update the Boot BIOS and firmwareusing the QLogic SANsurfer iSCSI HBA Manager interface or a DOS-bootablediskette.

Obtain the initiator and target node names: TheiSCSI node name is required when you create the igroup and map the LUN to thatigroup.

Create the Boot LUN: After you obtain aninitiator node name, you create the LUN that will be used as a boot device, mapit to an igroup, and assign a LUN ID.

Configure the HBA BIOS to boot from the Boot LUN:Use the QLogic Fast!Util utility (Ctrl-Q during host boot) to configure the HBABIOS. Enable the Spinup Delay option. It is disabled by default on a factory-shippediSCSI HBA.

Configure the BIOS boot order: After youconfigure the HBA BootBIOS, you enter system BIOS setup to configure the hostto boot from the LUN and ensure that the internal disk is not bootable throughthe system BIOS boot order.

Install the operating system on the Boot LUN

Alter settings to support SAN Boot: You mustchange these settings to support SAN Boot:

QLogic configuration settings

Windows configuration settings

Multipathing

Multipathing is the term used to describe an FC or IP SAN solution that has been designed to include at least two distinct physical paths from a host to a target LUN. Multiple paths from a particular host to a particular target LUN generally reduce the risk of a single point of failure in the Fibre Channel fabric or Ethernet network and contribute to a highly available (HA) SAN solution. However, this is not a requirement. For example, if two physical paths from a host to a target LUN ran through the same switch then the solution would not be considered to be highly available or without a single point of failure. If the switch failed, then both paths would be lost.

Multipathing is implemented for two reasons.

Multipathing provides highly available (redundant) paths. It is implemented to eliminate a single point of failure in the FC fabric or Ethernet network. This includes the interfaces on the hosts and target controllers.

Some forms of multipathing increase throughput using multiple physical connections. This allows the host to simultaneously send data across more than one path to target controllers.

In active/active clustering, operating systems, applications, or services running on a cluster can access the same resources at the same time. In active/passive clustering, one node is preferred (active) and the operating system, application, or service cannot access the resources until a failure occurs and the passive node takes over for the active node.

中低端的存储多为active/passive,高端的当然是active/active.

Active/Active Storage Controller Failover

There are two types of active/active storage controller configurations: standard and mirrored.

资料中只有standard的,mirrored的没有.

A standard active/active storage system configuration contains two paths to the Fibre Channel disk shelves, a local controller, and a partner controller (except the FAS270c, which does not require two sets of disk shelves). Both controllers must be the same model and they should be running the same version of Data ONTAP.The controllers are connected to each other via a cluster interconnect (IC). Each controller continually monitors its partner, mirroring the data from its partner’s NVRAM.

以前EMC的SYMM的cache是没有镜像的,后面迫于竞争压力才加入的

There are many benefits to an active/active storage controller configuration, including:

High-availability data-clustering solution: This configuration protects against controller failure by transferring the data service from the failed storage system controller to its partner controller. In addition to controller failure, active/active storage controller failover can also protect against other hardware failures, such as network interface. Controller failover is also an effective tool for reducing planned downtime of one of the nodes.

Nondisruptive controller and disk maintenance: When you halt one controller and allow takeover, the partner controller continues to serve data for the halted controller while you replace or repair hardware in the halted controller.

Nondisruptive software upgrades: When you halt one controller and allow takeover, the other controller continues to serve data for the halted controller while you upgrade the halted controller.

Single Point of Failure

A single point of failure represents the failure of a single hardware component that can lead to the loss of data access or potential loss of data. A single point of failure does not include multiple or rolling hardware errors, such as double disk failure without RAID-DP. All hardware components have demonstrated very good reliability with low failure rates. If a hardware component, such as a storage controller or adapter, fails, then you can use an active/active storage controller configuration to provide continuous data availability and preserve data integrity for client applications and users.

Active/Active Storage Controller Configuration

In a clustered architecture, a pair of storage controllers of the same model are connected to one another through a cluster interconnect adapter. The cluster interconnect (IC) allows the storage controllers to perform these operations:

Determine whether or not the other storage controller is functioning (heartbeat)

Mirror the log data to each other’s NVRAM

Pass data across the IC between a host and the partner controller in the event of a switch or fabric failure (dependent on cfmode). Any path utilizing the IC to pass data is known as a proxy path.

In the diagram, storage controllers use FC-AL adapters to manage their disks. The cable from the FC-AL adapter in each storage controller connects to the interface module in the disk shelf. Each storage controller’s disks are on an A Loop, and the storage controller’s partner’s disks can be accessed on takeover on a B Loop. The storage controllers must have redundant network connections so that each storage controller can assume its partner’s network identity.

An active/active storage controller configuration (cluster) allows one storage controller to take over for the other if the second storage controller fails. This means that data from the failed storage controller’s disks can still be served through the functioning storage controller.

Cfmode only applies to Fibre Channelenvironments in an active/active NetApp storage controller configuration. Thecfmode determines how target ports do the following:

Log into the fabric

Handle local and partnertraffic for a cluster

Provide access to local andpartner LUNs in a cluster

In the original release of Data ONTAP 6.3,which included SAN support for Fibre Channel, cfmode standby was the implieddefault. There was not a setting for cfmode in that release, and it was notcalled cfmode standby. However, when Data ONTAP 6.5 was released, four cfmodeswere introduced. One of these modes was standby. The others were partner, mixedand dual fabric. In Data ONTAP 7.1, a new cfmode called single system image(SSI) became available. SSI is the default cfmode for new installations withData ONTAP 7.2. The availability of standby, partner, mixed and dual fabricmodes is dependant on the storage controller model, Data ONTAP version, and/orthe use of 2Gb or 4Gb FC ports.

Five cfmodes

There are five possible cfmodes on the storage controller. Only one cfmode can be set per each storage controller, and in a cluster situation the cfmode must be the same for both systems.

Standby

The standby mode is supported on all systems except the FAS270c. It supports only Windows and Solaris operating systems. In addition, this mode requires additional switch ports.

Partner

The partner mode is supported on all systems except the FAS270c and the FAS6000 series. All switches and host operating systems are supported.

Mixed

The mixed mode is supported on all systems except the FAS270c and the FAS6000 series. Mixed mode supports all host operating systems, but requires a switch that supports a public loop.

Dual Fabric

The dual fabric mode is only supported on a FAS270c. All host operating systems are supported by this mode. This mode requires a switch that supports a public loop. It requires fewer switch ports.

Single Image

The single image mode is supported on all systems, switches, and host operating systems. This mode makes all LUNs available on all target ports.

These modes can be viewed with the fcp show cfmode command.

FCP Proxy and Multi-ID Support

FCP ProxyFCP proxy allows Data ONTAP? tosupport hosts that have active/passive multipathing software. Theactive/passive connection is maintained using an adapter between storagecontrollers to proxy commands. Specifically, FCP Proxy is needed to supportHP-UX and AIX hosts that bind the physical switch port address to the targetdevice. HP-UX and AIX must have static source identifier (S_ID) and destinationidentifier (D_ID) addresses in a SAN FCP environment.

Multi-ID SupportMulti-ID support is required for controllerssuch as the FAS270c, which have only one physical FC port per controller.

Solaris , Windows , AIX HP-UX , and Linux FC Multipathing

The following multipathing solutions arecurrently supported by NetApp:

Solaris

VERITAS Dynamic Multipathing(DMP) – VERITAS DMP providesmultiple paths to a single LUN. This allows for greater throughput and highavailability.

VERITAS DMP supports active/active and active/passive modes. Inactive/active mode, multiple paths are used simultaneously. VERITAS uses a loadbalancing policy that balances the I/O across all available paths.

In active/passive mode, a primary path is owned by one controller and asecondary path by another controller. If a path that was passive becomesactive, the ownership of the LUN switches to the controller on the active path.

Windows

NTAP DSM – NetApp uses the sameDSM for FC- and iSCSI-connected LUNs. This DSM provides the active/passive failover-onlyload balancing policy on a per-LUN basis. For a single LUN, all I/Os will bedone across a single path until a failure of the active path occurs. However,the LUNs are assigned paths based on the round robin algorithm.

HP-UX PVLinks – PVLinks (PhysicalVolume Links) are HP-UX’s multipathing solution for virtual disks. Duringfailover, PVLinks uses a simple algorithm: it tries the first known path,followed by the second known path, and so on, until it has tried all paths. Ifall paths are unavailable, the LUN (Virtual Disk) goes offline. PVLinks doesnot perform any load balancing functions.

VERITAS DMP

Linux

QLogic? SANsurfer forLinux – SANsurfer is used to manage QLogic HBAs. It provides multipathingoptions for Linux.

The single image, partner, and dual fabriccfmodes always require multipathing software on the host operating system. Themixed cfmode requires multipathing software on AIX and HP-UX only. The standbycfmode is not supported on AIX or HP-UX.

iSCSI Multipathing for Windows

Multipathing Input/Output (MPIO) – The “classic” way to do multipathing is to insert a separate multipathing layer into the storage stack. This method is not specific to iSCSI or to any underlying transport, and is the standard way to achieve multipathing access to Fibre Channel and even parallel SCSI targets. There are multiple implementations of this type of multipathing on the various operating systems. The MPIO infrastructure offered by Microsoft is the standard way to do this on Windows Server technologies. With the Microsoft MPIO, each storage vendor supplies a device-specific module device specific module for its storage array.

NTAP (NetApp) DSM – NTAP DSM is a part of SnapDrive 3.2 and higher. It requires a software initiator. NTAP DSM is not currently supported in configurations that include VERITAS Storage Foundation.

Note: NTAP or NetApp DSM is for use with SnapDrive 4.1 and earlier. In future releases (subject to change), the DSM will be called Data ONTAP? DSM for Windows? MPIO. The Data ONTAP DSM for Windows will be compatible with SnapDrive 4.2 and later.

Multiple Connections per Session (MCS) – MCS creates the multiple paths starting at the iSCSI session layer of the storage stack. Both the iSCSI initiator (host) and the iSCSI target (controller) need to support multiconnection sessions in order to configure sessions with multiple connections. MCS requires a software initiator. MCS should not be confused with the Microsoft Cluster Service

iSCSI Sessions

A session is established when the host initiator logs into the iSCSI target. This session is similar to a pipe or conduit. Within the session you can have one or more connections. The session is the pipe and the connections run inside the pipe or session.

In most instances of iSCSI, each session has a single connection. When using a multipathing option like TCP/IP link aggregation and Multiple Connections per Session (MCS), the number of connections within a session can be increased. Having multiple connections within a session provides these benefits:

There is no single point of failure on the iSCSI-type SAN network

There may be increased throughput

There is session persistence in the event of a connection failure.

Sessions and Connections

The diagram provides two examples illustrating sessions and connections, and how they relate to each other.

The top example shows four sessions each with one connection each. Microsoft Multipath I/O (MPIO) would use a session and connection configuration like this. The MPIO software manages how the sessions are used to move data between the host and the target LUNs.

这些东西和NETAPP就没有直接关系了，对所有使用ISCSI的都一样。

Microsoft Multipathing Input/Output (MPIO)

There are multiple implementations of multipathing on the various operating systems. The Windows MPIO infrastructure offered by Microsoft is the standard way to do this on Windows server technologies. With the Microsoft MPIO, each storage vendor supplies a device-specific module (DSM) for its storage array. NetApp currently supports three DSMs, NTAP DSM (Data ONTAP 6.5.4 or higher), Microsoft iSCSI DSM (Data ONTAP 6.5.6 or higher), and VERITAS DSM (Data ONTAP 7.0? or higher). NetApp does not support the installation of the Microsoft iSCSi DSM on the same host as the NTAP DSM, even if the Microsoft iSCSI DSM is not being used. The same applies to the VERITAS DSM. It cannot be installed on the same host as the NTAP DSM at this time.

Note: VERITAS DSM and Microsoft iSCSI DSM are not currently compatible with SnapDrive or SnapManager?.

NTAP DSM

The 图 shows a host running the NetApp NTAP DSM with two iSCSI HBA’s, or two NICs with the Microsoft iSCSI Software Initiator, or one iSCSI HBA and one NIC with the Microsoft iSCSI Software Initiator attached to the IP SAN network. The MPIO drivers and the DSM layer manage the “paths” and shield the upper layers of the operating system from the underlying multiple paths to each LUN and the management of those paths. The two black arrows represent two sessions that have been established between the host and the target. The blue arrows represent two paths (over the two sessions) managed by the DSM that the host can use to access a LUN on the target. The DSM presents a single LUN to the upper layers of the OS. Without the DSM to manage the paths, the host would be presented with the same LUN multiple times.

Multiple Connections per Session (MCS)

Multiple Connections per Session (MCS) is an optional part of the iSCSI specification. These contain multiple connections starting at the iSCSI session layer of the storage stack. Both the iSCSI initiator (host) and the iSCSI target (storage controller) need to support MCS in order to configure sessions with multiple connections.

MCS is currently implemented on the Windows host side using the Microsoft iSCSI Software Initiator 2.X and on the target side using Data ONTAP 7.1 or higher. There is no additional MPIO layer required on the host. Refer to the NetApp iSCSI support matrix for the most up-to-date information regarding supported Data ONTAP and initiator releases. MCS does not currently support the use of iSCSI HBAs, only network interface cards. Given that the iSCSI initiator portion of the stack resides on the HBA itself, implementing MCS across iSCSI HBAs will have its challenges.

In the diagram, two connections are present in one session. With Microsoft iSCSI Software Initiator 2.0 and higher, up to four connections are allowed per session. Data ONTAP allows up to 16 connections per session.

The following points apply to MCS:

MCS requires Data ONTAP 7.1 or higher.

iSCSI must be licensed and enabled.

igroups and LUNs must be created.

A Session between the host and target must be established. This process also establishes a single connection within the session.

Additional connections can be added within the session.

In order to allow Data ONTAP to support MCS an administrator must set the options iscsi.max_connections_per_session option to specify the number of connections per session that Data ONTAP will accept. The value can range from 1 to 16 connections per session. Four is the recommended value if MCS is to be employed. The default value is use_system_default. (In Data ONTAP 7.1 and 7.2, the system default is one connection per session)

All LUNs accessed over a session are affected by that session’s load balancing policy. Active/active round robin is the only load balancing policy supported by NetApp at this time.

All interfaces on the NetApp Storage System that are involved in a particular MCS session must all belong in the same Target Portal Group

iSCSI Multipathing for UNIX? and Linux?

放这里供大家备查

Linux

IP Trunking, Portal Failover, and Device-Mapper are Linux multipathing options. The options that are supported will vary Linux distribution (Red Hat or SuSE) and version.

IP Trunking

Aggregating bandwidth across multiple physical links to a switch is referred to as teaming, trunking, port trunking, or link aggregation. This allows a machine (frequently a server) to treat multiple physical connections to switch units as a single logical link.

NIC Teaming is not supported by Microsoft with its iSCSI Software Initiator. NIC Teaming is supported on the public (no iSCSI traffic) Windows network. NIC Teaming is also currently supported by NetApp on all other operating systems on the NetApp iSCSI Support Matrix.

Portal Failover

When an existing path to the target fails on Linux, the iSCSI driver attempts to connect through the next available IP address. You may select a preferred portal in the case of failure. The portal failover feature is turned on by default and the process of failover is automatic. You may turn off portal failover by disabling the portal failover parameter in /etc/iscsi.conf.

Device-Mapper

Device-Mapper is a Linux kernel component that supports Logical Volume Management (LVM). If a path fails, Device-Mapper will reroute requests over available paths.

Device-Mapper recognizes each path as a separate device. It creates another device on top of these multiple paths and uses the new device to reroute requests to the underlying devices.

For more information on Device-Mapper, refer to Red Hat’s Multipath-usage.txt file at http://www.redhat.com/docs/manua … ipath-usagetxt.html

For more information on accessing LUNs using Linux multipathing solutions, refer to the iSCSI Red Hat Enterprise Linux Initiator Support Kit Setup Guide.

PVLinks (Physical Volume Links) are HP-UX’s LVM integrated multipathing solution for disks. During failover, PVLinks uses a simple algorithm: it tries the first known path, followed by the second known path, and so on, until it has tried all paths and will continue to try all the paths. PVLinks does not perform any load balancing functions.

In Solaris IP multipathing (IP/MP) two identical network cards are grouped together with one IP address. The cards automatically fail over from one card to the other with no loss of service. You may create multiple logical IP addresses in order to force load balancing. IPMP provides failover across switches, providing an additional layer of redundancy. IPMP works across various network adapters to ensure that the switch is not a single point of failure.

MPxIO

MPxIO is the standard Solaris multipathing solution. To use MPxIO for iSCSI, you should have at least two Ethernet interfaces on the storage system enabled for iSCSI traffic. Each iSCSI interface must be in a different iSCSI target portal group. Data ONTAP does this by default.

For more information on enabling MPxIO on Solaris, refer to the iSCSI Solaris Initiator Support Kit Setup Guide.

FlexShare

FlexShare is a Data ONTAP software feature that provides workload prioritization for a storage system. It prioritizes processing resources for key services when the system is under heavy load. FlexShare does not provide guarantees on the availability of resources or how long particular operations will take to complete. FlexShare provides a priority mechanism to give preferential treatment to higher-priority tasks.

How FlexShare Works: Basics

FlexShare allows you to assign priorities to different volumes. In addition, it provides the ability to configure certain per-volume attributes, including user versus system priority and cache policies.

WAFL Operations

A read or write request initiated from any data protocol is translated to individual read or write WAFL operations by the file system. Similarly, a system request is translated into individual WAFL operations. Data ONTAP classifies each WAFL operation as a user or system operation based on its origin. For example, a client read request is classified as a user operation; a SnapMirror? request is classified as a system operation.

Processing Buckets

FlexShare maintains different processing buckets for each volume that has a configured priority setting. FlexShare populates the processing buckets for each volume with WAFL operations as they are submitted for execution. The processing buckets are only used when the FlexShare service is on; when the FlexShare service is off, all WAFL operations are bypassed from processing buckets and sent directly to WAFL.

Data ONTAP maintains a default processing bucket. When the FlexShare service is on, all WAFL operations associated with volumes that do not have a FlexShare priority configuration are populated in the default processing bucket; all WAFL operations for a volume that have a FlexShare priority configuration are populated into a dedicated bucket.

How FlexShare Works: Example

The “FlexShare off” figure depicts the order in which tasks arrive for processing and the order in which they are processed by the storage system. The order of tasks processed is exactly the same as the order in which the tasks arrive.

The “FlexShare on” figure depicts a possible ordering of tasks when the FlexShare service is enabled. The order in which tasks arrive is different from the order in which they are processed by the storage system. FlexShare orders tasks for processing by taking into account the priority configuration. In this example, Vol1 has a higher priority configuration than the other volumes. This means that the WAFL operations from Vol1 are preferentially processed.

FlexShare Benefits

FlexShare provides storage systems with these key features:

Multiple, disparate workloads can share a single storage system. This allows for more storage consolidation.

Business-critical applications receive I/O priority

Control of system and client workloads allows you to easily adjust workloads when priorities change.

These features allow storage administrators to tune how the system should prioritize system resources in the event that the system is overloaded

SnapMirror

There are two types of SnapMirror solutions:

Asynchronous SnapMirror – Asynchronous SnapMirror is an automated file system or qtree replication for disaster recovery or data distribution. Updates of new and changed data from the source to the destination occur on a schedule defined by the storage administrator. Updates can be as frequent as per minute or as infrequent as per weekly or even monthly, depends on the need of the user.

Synchronous SnapMirror – Synchronous SnapMirror replicates writes from a source volume to a partner destination volume at the same time it is written to the source volume. Updates from source to destination are performed in real time interval.

异步是要丢部分数据的，这个无法避免。总比全丢好。

What is Snapshot?

The WAFL file system can copy itself (snapshot) at any point in time, and make the copied versions of the file system available via “special” subdirectories that appear in the current (active) file system. Each copied version of the file system is called a snapshot. Up to 255 concurrent snapshots per volume are upported by the current Data ONTAP operating system.

A storage snapshot creates a separate set of pointers to data that can be mounted as a volume or file system to another host and treated as though it were a duplicate of the original data. Creating snapshots is very quick, because it is essentially an index. Data blocks are not duplicated.

A snapshot can be scheduled to occur automatically or taken manually. Automatic schedules can be created on an hourly, nightly, or weekly basis.

When you install Data ONTAP on a storage appliance, it creates a default snapshot schedule.The default snapshot schedule automatically creates one nightly snapshot Monday through Saturday at midnight, and four hourly snapshots at 8 a.m., noon, 4 p.m., and 8 p.m. Data ONTAP retains the two most recent nightly snapshots and the six most recent hourly snapshots, and deletes the oldest nightly and hourly snapshots when new snapshots are created.

看一个老外的BLOG说，aggr的snapshot可以关掉，又多5％的空间可以用。

SnapRestore

Instantaneous recovery – any size volume (up to 16TB)

File-level SnapRestore capability – crucial for large files/LUNs

Instantly recover from data corruptions

Critical tool when integrated into change control processes (e.g., SnapRestore as a backout plan for software upgrades)

SnapRestore is an optional feature 意思应该是要用就要买许可，哈哈。NETAPP的特点，功能很多，都包含在DOT中，只要买了许可加入就可以用。和容量无关，系统大了划得来。

Recover 比较磁带和SnapRestore

Recover 比较磁带和SnapRestore

SnapRestore 的就是把指针的指向改一下，当然很快。图中写的磁带60G/H，实际现在的情况应该不止。

同步？异步？半同步?

SnapMirror can be configured into 3 different replications mode. All are available with a single license.

The first is Synchronous mirroring. In this solution the data at the DR site exactly matches the data at the primary site.

This is achieved by replicating every data write to the remote location and not acknowledging to the host the write occurred until the remote systems confirms the data was written.

This solution provides the least data loss but there is a limit of 50-100Km before latency becomes too great because the host application must wait for an acknowledgement from the remote NetApp devices.

The second is semi-synchronous SnapMirror. This configuration allows you to achieve a near zero data loss DR solution without the performance impact on the host application.

This solution also allows you to do synchronous type replication over longer distances.

How this works is that when data is written to the primary storage an acknowledgement is immediately sent back eliminating the latency impact on the host. In the background SnapMirror tries to maintain as close to synchronous communication as possible with the remote system. SnapMirror has user defined thresholds on how far out of sync are allowed for the source and remote copy data sets.

Finally asynchronous SnapMirror.

Asynchronous SnapMirror allows you to replicate data at adjustable frequencies. You can do this type of point in time replication as frequently as 1 minute or as infrequent as days.

There is no distance limitation and is frequently used to replicate long distances to protect against regional disasters. And only the blocks that have changed between each replication are sent, minimizing network usage.