Configuring Device Mapper Multipath (DM-Multipath) with Linux

click to see more Author: Shan Baugher

Linux SAN Multipathing

Multipathing in linuxWhat is Multipathing?

Multipathing is a fault-tolerance technology that allows you to have more than one physical path between a host (server) and an external storage device (SAN). In the event that any component (NIC, cable, switch) in the environment fails, the physical path is switched to a path that does not involve the failed component allowing continued communications. My weapon of choice when it comes to multipathing in Linux is Device Mapper-Multipath (DM-Multipath). DM-Multipathing is a fail-over AND load-balancing solution that is highly configurable.

Tutorial
This tutorial will include

1. Oracle Enterprise Linux Server 6.6 click the following article

2. DM-Mulitipath – device-mapper-multipath-0.4.9-80.0.1.el6_6.3.x86_64 read article

To configure a system with DM-Multipath we need to install the package first. There are a couple options here… see more

go here Option 1: Install from a configured YUM repository.

# yum install device-mapper-multipath

please click for source Option 2: Download the package and manually install it…

# rpm -ivh device-mapper-multipath-0.4.9-80.0.1.el6_6.3.x86_64

Next, we need to configure DM-Multipath click here.

1. Edit the /etc/multipath.conf file and set parameters as necessary. Your storage provider probably has recommended settings and should be checked http://cobre.be/map10.

# vi /etc/multipath.conf

http://brotherkennyh.co.uk/map10 2. Load the necessary modules

# modprobe dm-multipath

3. Add dm-multipath to startup continue reading

# chkconfig multipathd on

http://schmiedeeisen-zaune.com/mapca1 4. Start multipath daemon

# service multipathd start

Now, the stage is set.
I like to take a quick inventory of the existing storage on the system prior to adding new storage to the server so the new LUN is easier to identify.

[root@OEL6-test eagle]# ll /dev/sd*
brw-rw---- 1 root disk 8,  0 Sep  7 10:11 /dev/sda
brw-rw---- 1 root disk 8,  1 Sep  7 10:12 /dev/sda1
brw-rw---- 1 root disk 8,  2 Sep  7 10:11 /dev/sda2
brw-rw---- 1 root disk 8, 16 Sep  7 10:12 /dev/sdb

[root@OEL6-test eagle]# lsblk
NAME                           MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda                              8:0    0   50G  0 disk 
├─sda1                           8:1    0  500M  0 part  /boot
└─sda2                           8:2    0 49.5G  0 part 
  ├─vg_oel6test-lv_root (dm-0) 252:0    0 43.5G  0 lvm   /
  └─vg_oel6test-lv_swap (dm-1) 252:1    0    6G  0 lvm   [SWAP]
sdb                              8:16   0    2G  0 disk 
└─3600144f09ff1616800005717cb580001 (dm-2)
                               252:2    0    2G  0 mpath
  └─3600144f09ff1616800005717cb580001p1 (dm-3)
                               252:3    0    2G  0 part  /test
sr0                             11:0    1 1024M  0 rom 

 [root@OEL6-test eagle]# cat /proc/scsi/scsi
Attached devices:
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: VBOX     Model: CD-ROM           Rev: 1.0
  Type:   CD-ROM                           ANSI  SCSI revision: 05
Host: scsi2 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: VBOX HARDDISK    Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi3 Channel: 00 Id: 00 Lun: 00
  Vendor: SUN      Model: Sun Storage 7000 Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 05

[root@OEL6-test eagle]# multipath -ll
3600144f09ff1616800005717cb580001 dm-2 SUN,Sun Storage 7000
size=2.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  `- 3:0:0:0 sdb 8:16 active ready running

check this out We are ready to add a LUN. Present a LUN to the host or have your storage administrator present a LUN to your server. I presented a 1GB LUN to the server and we are ready to proceed.

First, we need to rescan the server for the newly presented LUN(s).

[root@OEL6-test eagle]# echo "- - -" > /sys/class/scsi_host/host0/scan 
[root@OEL6-test eagle]# echo "- - -" > /sys/class/scsi_host/host1/scan 
[root@OEL6-test eagle]# echo "- - -" > /sys/class/scsi_host/host2/scan 
[root@OEL6-test eagle]# echo "- - -" > /sys/class/scsi_host/host3/scan

click to see more Alternatively, this can also be run as a for loop>

# for host in $(ls-1d /sys/class/scsi_host/*); do echo “- - -“ > ${host}/scan; done

check this out Also, in the event that fiber channel is in use, the respective commands would be>

# echo "1" > /sys/class/fc_host/host1/issue_lip
# echo "1" > /sys/class/fc_host/host2/issue_lip

And the for loop> http://segurosprivados.com.pt/mapca1

# for host in $(ls -1d /sys/class/fc_host/*); fo echo “1” > ${host}/issue_lip; done

Now, let’s take another look… http://tunekey.info/mapca1

 [root@OEL6-test eagle]# ll /dev/sd*
brw-rw---- 1 root disk 8,  0 Sep  7 10:11 /dev/sda
brw-rw---- 1 root disk 8,  1 Sep  7 10:12 /dev/sda1
brw-rw---- 1 root disk 8,  2 Sep  7 10:11 /dev/sda2
brw-rw---- 1 root disk 8, 16 Sep  7 10:18 /dev/sdb
brw-rw---- 1 root disk 8, 32 Sep  7 10:23 /dev/sdc

[root@OEL6-test eagle]# lsblk
NAME                           MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda                              8:0    0   50G  0 disk 
├─sda1                           8:1    0  500M  0 part  /boot
└─sda2                           8:2    0 49.5G  0 part 
  ├─vg_oel6test-lv_root (dm-0) 252:0    0 43.5G  0 lvm   /
  └─vg_oel6test-lv_swap (dm-1) 252:1    0    6G  0 lvm   [SWAP]
sdb                              8:16   0    2G  0 disk 
└─3600144f09ff1616800005717cb580001 (dm-2)
                               252:2    0    2G  0 mpath
  └─3600144f09ff1616800005717cb580001p1 (dm-3)
                               252:3    0    2G  0 part  /test
sdc                              8:32   0    1G  0 disk 
sr0                             11:0    1 1024M  0 rom  

[root@OEL6-test eagle]# cat /proc/scsi/scsi
Attached devices:
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: VBOX     Model: CD-ROM           Rev: 1.0
  Type:   CD-ROM                           ANSI  SCSI revision: 05
Host: scsi2 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: VBOX HARDDISK    Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi3 Channel: 00 Id: 00 Lun: 00
  Vendor: SUN      Model: Sun Storage 7000 Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi3 Channel: 00 Id: 00 Lun: 01
  Vendor: SUN      Model: Sun Storage 7000 Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 05

[root@OEL6-test eagle]# multipath -ll
3600144f09ff1616800005717cb580001 dm-2 SUN,Sun Storage 7000
size=2.0G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  `- 3:0:0:0 sdb 8:16 active ready running

Now, the LUN has been discovered by the host, but it is still not visible in multipath. So, we need to add the new LUN to the multipath configuration click at this page.

First we need to discover the wwid of the new LUNvisit web page

[root@OEL6-test eagle]# scsi_id -g -u /dev/sdc
3600144f09ff16168000057cfe9710001

Next, edit the /etc/multipath.conf file and add the wwid of the new LUN to the ‘blacklist exceptions’ and ‘multipaths’ portions of multipath.conf.
Ex: multipath.conf – you can see the new entries highlighted

[root@OEL6-test eagle]# cat /etc/multipath.conf

defaults {
  udev_dir                /dev
  polling_interval        10
  path_selector           "round-robin 0"
  path_grouping_policy    multibus
  getuid_callout          "/lib/udev/scsi_id --replace-whitespace --whitelisted --device=/dev/%n"
  path_checker            readsector0
  rr_min_io               100
  max_fds                 8192
  rr_weight               priorities
  failback                immediate
  no_path_retry           fail
  user_friendly_names     no
}
blacklist {
  devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
  devnode "^hd[a-z]"
  devnode "^dcssblk[0-9]*"
  # don't count normal SATA devices as multipaths
  device {
          vendor  "ATA"
  }
  wwid "*"
}
devices {
  device {
  }
}
blacklist_exceptions {
       wwid "3600144f09ff1616800005717cb580001"
       wwid "3600144f09ff16168000057cfe9710001"
}
multipaths {
       multipath {
  uid 0
  gid 0
       wwid "3600144f09ff1616800005717cb580001"
  mode 0600
       }
       multipath {
  uid 0
  gid 0
       wwid "3600144f09ff16168000057cfe9710001"
  mode 0600
       }
}

here Finally, reload/restart the multipath daemon>

[root@OEL6-test eagle]# multipath -r
reload: 3600144f09ff1616800005717cb580001 undef SUN,Sun Storage 7000
size=2.0G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
 `- 3:0:0:0 sdb 8:16 active ready running
reload: 3600144f09ff16168000057cfe9710001 undef SUN,Sun Storage 7000
size=1.0G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
 `- 3:0:0:1 sdc 8:32 active ready running

http://jazzmarketing.ca/map168 Now, the new LUN has been added to the multipath configuration! All that is left is to partition, format and mount the LUN!

 Want to automate this process? Contact us to learn how our team uses Shell Script to automate the DM-Multipath configuration process.

Related post: Building Test Environments with Oracle VirtualBox here

About the author read article

Shan Baugher, Systems Administrator
Shan Baugher began working with Eagle in 2012 and is responsible for initiating, architecting, and managing complex IT infrastructures for clients, as well as the offices at Eagle’s headquarters. With over 8 years of project management and systems administration experience, Shan is well-versed in managing virtual environments of all shapes and sizes. He holds an assortment of Professional Microsoft Certifications including MCSE, MCSA, MCP, and MCTS and is also Certified as a Specialist in Oracle VM 3.0 for x86 implementation.