July 12, 2011

Just a thought for those of you that need to use a text, binary compare tool here and there.

Today I had to waste some time and figure out what text, binary, picture, etc compare tool was the best. I Compared a handful of tools I used in the past and some I haven’t used. gVIM Diff, WinMerge, Beyond Compare, Perforce, and DiffMerge. I decided that a few of my needs was to do a diff on the fly without creating a file, show special characters, syntax highlighting, merging, line numbering, pic split view, search within both files, SVN integration, etc. I ended up going with Beyond Comapre since we have a corp license and kep WinMerge because it was a very close second and has integration to TortoiseSVN which I use as well.

November 18, 2008

These install/howto instructions are for CentOS 5.x and Dell MD3000i. You can easily modify these slightly for your specific iSCSI disk.

iSCSI/Multipath/LVM

iSCSI/Multipath Installation

Install iSCSI initiator utilities
yum install iscsi-initiator-utils

Install multipath device mapper
yum install device-mapper-multipath

Configure multipath, iscsi, and iscsid to start on startup. Check first to make sure they are not already set to start on startup. If they are set to start you will see something like below, if they are not you will not see anything output listed as “on”.
/sbin/chkconfig --list | egrep -i "iscsi|iscsid|multipathd"
iscsi 0:off 1:off 2:off 3:on 4:on 5:on 6:off
iscsid 0:off 1:off 2:off 3:on 4:on 5:on 6:off
multipathd 0:off 1:off 2:on 3:on 4:on 5:on 6:off

To configure multipathd to start on startup do the following. iSCSI(d)already have this defined in their init script.
/sbin/chkconfig multipathd on

iSCSI Configuration

Edit /etc/iscsi/iscsid.conf and make sure it is identical to this config.
**I need to verify this iscsid.conf but for now this is what I have logged as a working configs from years back.
#
# Open-iSCSI default configuration.
# Could be located at /etc/iscsi/iscsid.conf or ~/.iscsid.conf
#
# Note: To set any of these values for a specific node/session run
# the iscsiadm --mode node --op command for the value. See the README
# and man page for iscsiadm for details on the --op command.
#

#############################
# NIC/HBA and driver settings
#############################
# open-iscsi can create a session and bind it to a NIC/HBA.
# To set this up see the example iface config file.

#*****************
# Startup settings
#*****************

# To request that the iscsi initd scripts startup a session set to "automatic".
# node.startup = automatic
#
# To manually startup the session set to "manual". The default is automatic.
node.startup = automatic

# To set a CHAP username and password for initiator
# authentication by the target(s), uncomment the following lines:
#node.session.auth.username = username
#node.session.auth.password = password

# To set a CHAP username and password for target(s)
# authentication by the initiator, uncomment the following lines:
#node.session.auth.username_in = username_in
#node.session.auth.password_in = password_in

# To enable CHAP authentication for a discovery session to the target
# set discovery.sendtargets.auth.authmethod to CHAP. The default is None.
#discovery.sendtargets.auth.authmethod = CHAP

# To set a discovery session CHAP username and password for the initiator
# authentication by the target(s), uncomment the following lines:
#discovery.sendtargets.auth.username = username
#discovery.sendtargets.auth.password = password

# To set a discovery session CHAP username and password for target(s)
# authentication by the initiator, uncomment the following lines:
#discovery.sendtargets.auth.username_in = username_in
#discovery.sendtargets.auth.password_in = password_in

# ********
# Timeouts
# ********
#
# See the iSCSI REAME's Advanced Configuration section for tips
# on setting timeouts when using multipath or doing root over iSCSI.
#
# To specify the length of time to wait for session re-establishment
# before failing SCSI commands back to the application when running
# the Linux SCSI Layer error handler, edit the line.
# The value is in seconds and the default is 120 seconds.
node.session.timeo.replacement_timeout = 120

# To specify the time to wait for login to complete, edit the line.
# The value is in seconds and the default is 15 seconds.
node.conn[0].timeo.login_timeout = 15

# To specify the time to wait for logout to complete, edit the line.
# The value is in seconds and the default is 15 seconds.
node.conn[0].timeo.logout_timeout = 15

# Time interval to wait for on connection before sending a ping.
node.conn[0].timeo.noop_out_interval = 5

# To specify the time to wait for a Nop-out response before failing
# the connection, edit this line. Failing the connection will
# cause IO to be failed back to the SCSI layer. If using dm-multipath
# this will cause the IO to be failed to the multipath layer.
node.conn[0].timeo.noop_out_timeout = 5

#******
# Retry
#******

# To speficy the number of times iscsiadm should retry a login
# to the target when we first login, modify the following line.
# The default is 4. Valid values are any integer value. This only
# affects the initial login. Setting it to a high value can slow
# down the iscsi service startup. Setting it to a low value can
# cause a session to not get logged into, if there are distuptions
# during startup or if the network is not ready at that time.
node.session.initial_login_retry_max = 4

# To control how many commands the session will queue set
# node.session.cmds_max to an integer between 2 and 2048 that is also
# a power of 2. The default is 128.
node.session.cmds_max = 128

# To control the device's queue depth set node.session.queue_depth
# to a value between 1 and 128. The default is 32.
node.session.queue_depth = 32

#***************
# iSCSI settings
#***************

# To enable R2T flow control (i.e., the initiator must wait for an R2T
# command before sending any data), uncomment the following line:
#
#node.session.iscsi.InitialR2T = Yes
#
# To disable R2T flow control (i.e., the initiator has an implied
# initial R2T of "FirstBurstLength" at offset 0), uncomment the following line:
#
# The defaults is No.
node.session.iscsi.InitialR2T = No

# To specify the maximum number of unsolicited data bytes the initiator
# can send in an iSCSI PDU to a target, edit the following line.
#
# The value is the number of bytes in the range of 512 to (2^24-1) and
# the default is 262144
node.session.iscsi.FirstBurstLength = 262144

# To specify the maximum SCSI payload that the initiator will negotiate
# with the target for, edit the following line.
#
# The value is the number of bytes in the range of 512 to (2^24-1) and
# the defauls it 16776192
node.session.iscsi.MaxBurstLength = 16776192

# To specify the maximum number of data bytes the initiator can receive
# in an iSCSI PDU from a target, edit the following line.
#
# The value is the number of bytes in the range of 512 to (2^24-1) and
# the default is 131072
node.conn[0].iscsi.MaxRecvDataSegmentLength = 131072

# To specify the maximum number of data bytes the initiator can receive
# in an iSCSI PDU from a target during a discovery session, edit the
# following line.
#
# The value is the number of bytes in the range of 512 to (2^24-1) and
# the default is 32768
#
discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768

# To allow the targets to control the setting of the digest checking,
# with the initiator requesting a preference of enabling the checking, uncommen
# the following lines (Data digests are not supported and on ppc/ppc64
# both header and data digests are not supported.):
#node.conn[0].iscsi.HeaderDigest = CRC32C,None
#
# To allow the targets to control the setting of the digest checking,
# with the initiator requesting a preference of disabling the checking,
# uncomment the following lines:
#node.conn[0].iscsi.HeaderDigest = None,CRC32C
#
# To enable CRC32C digest checking for the header and/or data part of
# iSCSI PDUs, uncomment the following lines:
#node.conn[0].iscsi.HeaderDigest = CRC32C
#
# To disable digest checking for the header and/or data part of
# iSCSI PDUs, uncomment the following lines:
#node.conn[0].iscsi.HeaderDigest = None
#
# The default is to never use DataDigests and to allow the target to control
# the setting of the HeaderDigest checking with the initiator requesting
# a preference of disabling the checking.

#***************
# iSCSI Multipath settings
#***************
# Specify the use of having multiple network paths to the same target.
# Valid values are
# no - there is no multipathing. Establish at most one session to the
# target and use all portals for failover, if failover is not disabled.
# portal - establish a session to the target using each of the n/w portals to
# the target, and bind each session to a different host SCSI address.
# No portal failover is possible since each session has only one portal.
# portalgroup - establish a session to one portal from each portal group that
# reaches the target and bind each session to a different host SCSI
# address. If a portal group has more than one portal, portal failover
# can occur between that group's portals if failover is not disabled.
Multipath = portalgroup

Edit /etc/iscsi/initiatorname.iscsi and add the alias name you created (in the MD3000i for this host) to this config.
InitiatorAlias=iqn.SERVER_HOSTNAME

Verify that the iSCSI target has you as a host and has disks mapped to you.
You will want to ignore any Disk the doesn’t contain a valid partition table
You can also check /var/log/dmesg to see similar but more verbose info, but is not covered here.
fdisk -l

Now we have a configuration and modules, lets clear multipath database and re-establish the paths.

To flush the database use -F and the restart the deamon.
/sbin/multipath -F
/etc/init.d/multipathd restart
Now run multipath with the -ll option to see if you can see the 4 paths, and that the correct hardware-handler is being used. If everything is going well you should have something similar to the following.

To ensure that you don’t scan the ‘real’ devices and not the multipath devices, you need to change the filter in /etc/lvm/lvm.conf and rebuild the cache.
Find the filter line and change using your favourite editor.
####filter = [ "a/.*/" ] ###Original filter will be uncommented
filter = [ "a|/dev/sda|", "r|/dev/sd.*|", "r|/dev/dm-.*|", "a|/dev/mapper/mpath.*|", "r|/dev/.*|" ]

Now rebuild the cache.
rm /etc/lvm/cache/.cache
/sbin/vgscan

iSCSI mounting

Create a partition with fdisk
/sbin/fdisk /dev/sdb

The number of cylinders for this disk is set to 182038.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

October 14, 2008

This is how to add a index to a table. Adding this single index for syslog data added about a index (idx) file of about 12% of the size of the sys_log table data size. alter table sys_log add index (timestamp);