<font color='black' size='2' face='arial'>
<div>Hi All:</div>
<div><br>
</div>
<div>We are stuck trying to get pacemaker to work with DRBD, and having tried various alternatives can't get our "drbd1" to mount and get some errors.</div>
<div><br>
</div>
<div>NOTE: we are trying to get pacemaker to work with an existing Encrypted RAID1 LVM setup - is this impossible or a "just plain bad idea"? We were thinking we'd like the potential advantages of local RAID on each box as well as the Internet RAID & failover provided by DRBD/pacemaker. We're using Debian Squeeze. Per various instructions, we've disabled the DRBD boot init (update-rc.d -f drbd remove) and set the LVM filter to filter = [ "a|drbd.*|", "r|.*|" ].</div>
<div><br>
</div>
<div>FYI - we've commented out the LVM mount "/dev/vg2/vserverLV" in our fstab, and consistently seem to need to do this to avoid a boot error.</div>
<div><br>
</div>
<div>We think DRBD works until we add in the pacemaker steps (i.e., "dev/drbd1" mounts at boot; we can move related data from server1 to server2 back and forth, though need to use the command line to accomplish this). We've seen various statements on the net that suggest it is viable to use a "mapper" <font size="2">disk </font>choice in drbd.conf. Also, if we start by configuring Pacemaker for a simple IP failover, that works (i.e., no errors, we can ping via the fail over address) but stops working when we add in the DRBD primatives and related statements. Our suspicion (other than maybe "you can't do this with existing RAID") is that we're using the wrong "disk" statement in our drbd.conf<font size="2"> or maybe in our "</font>primitive fs_vservers<font size="2">" statement</font>, though we've tried lots of alternatives and this is the same drbd.conf we use before adding in Pacemaker and it seems to work at that point.</div>
<div><br>
</div>
<div>Lastly, while various config statements refer to "vservers", we have not gotten to the point of trying to add any data to the DRBD devices other than a few text files that have disappeared since doing our "crm" work.</div>
<div><br>
</div>
<div>Any help appreciated! <font size="2">Thanks, Ted</font></div>
<div><br>
</div>
<div>CONFIGS/LOGS</div>
<div><br>
</div>
<div>A) drbd.conf</div>
<div><br>
</div>
<div>global { usage-count no; }</div>
<div>common { syncer { rate 100M; } }</div>
<div>#original</div>
<div>resource r1 {</div>
<div> protocol C;</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>startup {</div>
<div> wfc-timeout 15;</div>
<div> degr-wfc-timeout 60;</div>
<div> }</div>
<div> device /dev/drbd1 minor 1;</div>
<div> disk /dev/vg2/vserverLV;</div>
<div> meta-disk internal;</div>
<div><br>
</div>
<div># following 2 definition are equivalent</div>
<div> on server1 {</div>
<div> address 192.168.1.129:7801;</div>
<div> disk /dev/vg2/vserverLV;</div>
<div> }</div>
<div> on server2 {</div>
<div> address 192.168.1.128:7801;</div>
<div> disk /dev/vg2/vserverLV;</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>#<span class="Apple-tab-span" style="white-space:pre"> </span>disk /dev/mapper/md2_crypt;</div>
<div> }</div>
<div><br>
</div>
<div># floating 192.168.5.41:7801;</div>
<div># floating 192.168.5.42:7801;</div>
<div> net {</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>cram-hmac-alg sha1;</div>
<div> shared-secret "secret";</div>
<div> after-sb-0pri discard-younger-primary; #discard-zero-changes;</div>
<div> after-sb-1pri discard-secondary;</div>
<div> after-sb-2pri call-pri-lost-after-sb;</div>
<div> }</div>
<div>}</div>
<div><br>
</div>
<div><br>
</div>
<div>B) Pacemaker Config</div>
<div><br>
</div>
<div>crm configure show</div>
<div>node server1</div>
<div>node server2</div>
<div>primitive app_ip ocf:heartbeat:IPaddr \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>params ip="192.168.1.152" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>op monitor interval="30s"</div>
<div>primitive drbd ocf:linbit:drbd \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>params drbd_resource="r1" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>op start interval="0" timeout="240" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>op stop interval="0" timeout="100" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>op monitor interval="59s" role="Master" timeout="30s" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>op monitor interval="60s" role="Slave" timeout="30s"</div>
<div>primitive fs_vservers ocf:heartbeat:Filesystem \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>params device="/dev/drbd1" directory="/vservers" fstype="ext4" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>op start interval="0" timeout="60" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>op stop interval="0" timeout="120"</div>
<div>primitive vg2 ocf:heartbeat:LVM \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>params volgrpname="vg2" exclusive="true" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>op start interval="0" timeout="30" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>op stop interval="0" timeout="30"</div>
<div>group lvm app_ip vg2 fs_vservers</div>
<div>ms ms_drbd drbd \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>meta master-node-max="1" clone-max="2" clone-node-max="1" globally-unique="false" notify="true" target-role="Master"</div>
<div>location drbd_on_node1 ms_drbd \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>rule $id="drbd_on_node1-rule" $role="master" 100: #uname eq server1</div>
<div>colocation vserver-deps inf: ms_drbd:Master lvm</div>
<div>order app_on_drbd inf: ms_drbd:promote lvm:start</div>
<div>property $id="cib-bootstrap-options" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>dc-version="1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>cluster-infrastructure="openais" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>expected-quorum-votes="2" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>stonith-enabled="false" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>no-quorum-policy="ignore"</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>C) crm <font size="2">status </font>results (with errors)</div>
<div>Last updated: Wed Feb 27 19:05:57 2013</div>
<div>Stack: openais</div>
<div>Current DC: server1 - partition with quorum</div>
<div>Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b</div>
<div>2 Nodes configured, 2 expected votes</div>
<div>2 Resources configured.</div>
<div>============</div>
<div><br>
</div>
<div>Online: [ server2 server1 ]</div>
<div><br>
</div>
<div><br>
</div>
<div>Migration summary:</div>
<div>* Node server2: </div>
<div> drbd:1: migration-threshold=1000000 fail-count=1000000</div>
<div>* Node server1: </div>
<div> drbd:0: migration-threshold=1000000 fail-count=1000000</div>
<div><br>
</div>
<div>Failed actions:</div>
<div> drbd:1_start_0 (node=server2, call=8, rc=-2, status=Timed Out): unknown exec error</div>
<div> drbd:0_start_0 (node=server1, call=6, rc=-2, status=Timed Out): unknown exec error</div>
<div><br>
</div>
<div>D) Mount</div>
<div><br>
</div>
<div>/dev/mapper/vg1-root on / type ext4 (rw,errors=remount-ro)</div>
<div>tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)</div>
<div>proc on /proc type proc (rw,noexec,nosuid,nodev)</div>
<div>sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)</div>
<div>udev on /dev type tmpfs (rw,mode=0755)</div>
<div>tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)</div>
<div>devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)</div>
<div>/dev/md0 on /boot type ext4 (rw)</div>
<div>/dev/mapper/vg1-home on /home type ext4 (rw)</div>
<div>/dev/mapper/vg1-tmp on /tmp type ext4 (rw)</div>
<div>/dev/mapper/vg1-usr on /usr type ext4 (rw)</div>
<div>/dev/mapper/vg1-var on /var type ext4 (rw)</div>
<div>fusectl on /sys/fs/fuse/connections type fusectl (rw)</div>
<div><br>
</div>
<div><br>
</div>
<div>E) fstab</div>
<div><br>
</div>
<div># /etc/fstab: static file system information.</div>
<div>#</div>
<div># Use 'blkid' to print the universally unique identifier for a</div>
<div># device; this may be used with UUID= as a more robust way to name devices</div>
<div># that works even if disks are added and removed. See fstab(5).</div>
<div>#</div>
<div># <file system> <mount point> <type> <options> <dump> <pass></div>
<div>proc /proc proc defaults 0 0</div>
<div>/dev/mapper/vg1-root / ext4 errors=remount-ro 0 1</div>
<div># /boot was on /dev/md0 during installation</div>
<div>UUID=25829c6c-164c-4a1e-9e84-6bab180e38f4 /boot ext4 defaults 0 2</div>
<div>/dev/mapper/vg1-home /home ext4 defaults 0 2</div>
<div>/dev/mapper/vg1-tmp /tmp ext4 defaults 0 2</div>
<div>/dev/mapper/vg1-usr /usr ext4 defaults 0 2</div>
<div>/dev/mapper/vg1-var /var ext4 defaults 0 2</div>
<div>#/dev/mapper/vg2-vserverLV /vservers ext4 defaults 0 2</div>
<div>/dev/mapper/vg1-swap none swap sw 0 0</div>
<div>/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto 0 0</div>
<div>/dev/scd1 /media/cdrom1 udf,iso9660 user,noauto 0 0</div>
<div>/dev/fd0 /media/floppy0 auto rw,user,noauto 0 0</div>
<div><br>
</div>
<div><br>
</div>
<div>F) fdisk -l</div>
<div><br>
</div>
<div>Disk /dev/sda: 160.0 GB, 160041885696 bytes</div>
<div>255 heads, 63 sectors/track, 19457 cylinders</div>
<div>Units = cylinders of 16065 * 512 = 8225280 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x0007c7a2</div>
<div><br>
</div>
<div> Device Boot Start End Blocks Id System</div>
<div>/dev/sda1 * 1 61 487424 fd Linux raid autodetect</div>
<div>Partition 1 does not end on cylinder boundary.</div>
<div>/dev/sda2 61 1885 14648320 fd Linux raid autodetect</div>
<div>/dev/sda3 1885 3101 9765888 fd Linux raid autodetect</div>
<div><br>
</div>
<div>Disk /dev/sdb: 203.9 GB, 203928109056 bytes</div>
<div>255 heads, 63 sectors/track, 24792 cylinders</div>
<div>Units = cylinders of 16065 * 512 = 8225280 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x0008843c</div>
<div><br>
</div>
<div> Device Boot Start End Blocks Id System</div>
<div>/dev/sdb1 * 1 61 487424 fd Linux raid autodetect</div>
<div>Partition 1 does not end on cylinder boundary.</div>
<div>/dev/sdb2 61 1885 14648320 fd Linux raid autodetect</div>
<div>/dev/sdb3 1885 3101 9765888 fd Linux raid autodetect</div>
<div><br>
</div>
<div>Disk /dev/md0: 499 MB, 499109888 bytes</div>
<div>2 heads, 4 sectors/track, 121853 cylinders</div>
<div>Units = cylinders of 8 * 512 = 4096 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x00000000</div>
<div><br>
</div>
<div><br>
</div>
<div>Disk /dev/md1: 15.0 GB, 14998757376 bytes</div>
<div>2 heads, 4 sectors/track, 3661806 cylinders</div>
<div>Units = cylinders of 8 * 512 = 4096 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x08040000</div>
<div><br>
</div>
<div><br>
</div>
<div>Disk /dev/md2: 9999 MB, 9999147008 bytes</div>
<div>2 heads, 4 sectors/track, 2441198 cylinders</div>
<div>Units = cylinders of 8 * 512 = 4096 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x08040000</div>
<div><br>
</div>
<div><br>
</div>
<div>Disk /dev/dm-0: 15.0 GB, 14997704704 bytes</div>
<div>255 heads, 63 sectors/track, 1823 cylinders</div>
<div>Units = cylinders of 16065 * 512 = 8225280 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x00000000</div>
<div><br>
</div>
<div><br>
</div>
<div>Disk /dev/dm-1: 3997 MB, 3997171712 bytes</div>
<div>255 heads, 63 sectors/track, 485 cylinders</div>
<div>Units = cylinders of 16065 * 512 = 8225280 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x00000000</div>
<div><br>
</div>
<div><br>
</div>
<div>Disk /dev/dm-2: 1996 MB, 1996488704 bytes</div>
<div>255 heads, 63 sectors/track, 242 cylinders</div>
<div>Units = cylinders of 16065 * 512 = 8225280 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x00000000</div>
<div><br>
</div>
<div><br>
</div>
<div>Disk /dev/dm-3: 1996 MB, 1996488704 bytes</div>
<div>255 heads, 63 sectors/track, 242 cylinders</div>
<div>Units = cylinders of 16065 * 512 = 8225280 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x00000000</div>
<div><br>
</div>
<div><br>
</div>
<div>Disk /dev/dm-4: 3997 MB, 3997171712 bytes</div>
<div>255 heads, 63 sectors/track, 485 cylinders</div>
<div>Units = cylinders of 16065 * 512 = 8225280 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x00000000</div>
<div><br>
</div>
<div><br>
</div>
<div>Disk /dev/dm-5: 1996 MB, 1996488704 bytes</div>
<div>255 heads, 63 sectors/track, 242 cylinders</div>
<div>Units = cylinders of 16065 * 512 = 8225280 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x00000000</div>
<div><br>
</div>
<div><br>
</div>
<div>Disk /dev/dm-6: 499 MB, 499122176 bytes</div>
<div>255 heads, 63 sectors/track, 60 cylinders</div>
<div>Units = cylinders of 16065 * 512 = 8225280 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x00000000</div>
<div><br>
</div>
<div><br>
</div>
<div>Disk /dev/dm-7: 9998 MB, 9998094336 bytes</div>
<div>255 heads, 63 sectors/track, 1215 cylinders</div>
<div>Units = cylinders of 16065 * 512 = 8225280 bytes</div>
<div>Sector size (logical/physical): 512 bytes / 512 bytes</div>
<div>I/O size (minimum/optimal): 512 bytes / 512 bytes</div>
<div>Disk identifier: 0x00000000</div>
<div><br>
</div>
<div>G) syslog excerpt</div>
<div><br>
</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) Command '</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) drbdsetup</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) 1</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) disk</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) /dev/vg2/vserverLV</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) /dev/vg2/vserverLV</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) internal</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) --set-defaults</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) --create-device</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) ' terminated with exit code 20</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) drbdadm attach r1: exited with code 20</div>
<div>Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Called drbdadm -c /etc/drbd.conf --peer server2 up r1</div>
<div>Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Exit code 1</div>
<div>Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Command output: </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stdout) </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) Can not open device '/dev/vg2/vserverLV': No such file or directory</div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) Command 'drbdsetup 1 disk /dev/vg2/vserverLV </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) /dev/vg2/vserverLV internal --set-defaults --create-device' terminated with exit code 20#012drbdadm attach r1: exited with code 20</div>
<div>Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Called drbdadm -c /etc/drbd.conf --peer server2 up r1</div>
<div>Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Exit code 1</div>
<div>Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Command output: </div>
<div>Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stdout) </div>
<div>Feb 27 06:36:23 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) Can not open device '/dev/vg2/vserverLV': No such file or directory</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><font size="2"><br>
</font></div>
<div><font size="2"><br>
</font></div>
</font>