[Pacemaker] CLVM & Pacemaker & Corosync on Ubuntu Omeiric Server

Wed Nov 30 07:51:27 EST 2011

@Vladislav

Where and how can I set the switch for the cluster manager if it runs as a resource.

Am 30.11.2011 13:10, schrieb Vadim Bulst:
> Am 30.11.2011 12:22, schrieb Vladislav Bogdanov:
>> 30.11.2011 14:08, Vadim Bulst wrote:
>>> Hello,
>>>
>>> first of all I'd like to ask you a general question:
>>>
>>> Does somebody successfully set up a clvm cluster with pacemaker and run
>>> it in productive mode?
>> I will say yes after I finally resolve remaining dlm&fencing issues.
>>
>>> Now back to the concrete problem:
>>>
>>>   I configured two interfaces for corosync:
>>>
>>> root at bbzclnode04:~# corosync-cfgtool -s
>>> Printing ring status.
>>> Local node ID 897624256
>>> RING ID 0
>>>      id    = 192.168.128.53
>>>      status    = ring 0 active with no faults
>>> RING ID 1
>>>      id    = 192.168.129.23
>>>      status    = ring 1 active with no faults
>>>
>>> RRD set to passive
>>>
>>> I also made some changes to my cib:
>>>
>>> node bbzclnode04
>>> node bbzclnode06
>>> node bbzclnode07
>>> primitive clvm ocf:lvm2:clvmd \
>>>      params daemon_timeout="30" \
>>>      meta target-role="Started"
>> Please instruct clvmd to use corosync stack instead of openais (-I
>> corosync): otherwise it uses LCK service which is not mature and I
>> observed major problems with it.
>>
>>> primitive dlm ocf:pacemaker:controld \
>>>      meta target-role="Started"
>>> group dlm-clvm dlm clvm
>>> clone dlm-clvm-clone dlm-clvm \
>>>      meta interleave="true" ordered="true"
>>> property $id="cib-bootstrap-options" \
>>>      dc-version="1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
>>>      cluster-infrastructure="openais" \
>>>      expected-quorum-votes="3" \
>>>      no-quorum-policy="ignore" \
>>>      stonith-enabled="false" \
>>>      last-lrm-refresh="1322643084"
>>>
>>> I cleaned and restarted the resources - nothing! :
>>>
>>> crm(live)resource# cleanup dlm-clvm-clone
>>> Cleaning up dlm:0 on bbzclnode04
>>> Cleaning up dlm:0 on bbzclnode06
>>> Cleaning up dlm:0 on bbzclnode07
>>> Cleaning up clvm:0 on bbzclnode04
>>> Cleaning up clvm:0 on bbzclnode06
>>> Cleaning up clvm:0 on bbzclnode07
>>> Cleaning up dlm:1 on bbzclnode04
>>> Cleaning up dlm:1 on bbzclnode06
>>> Cleaning up dlm:1 on bbzclnode07
>>> Cleaning up clvm:1 on bbzclnode04
>>> Cleaning up clvm:1 on bbzclnode06
>>> Cleaning up clvm:1 on bbzclnode07
>>> Cleaning up dlm:2 on bbzclnode04
>>> Cleaning up dlm:2 on bbzclnode06
>>> Cleaning up dlm:2 on bbzclnode07
>>> Cleaning up clvm:2 on bbzclnode04
>>> Cleaning up clvm:2 on bbzclnode06
>>> Cleaning up clvm:2 on bbzclnode07
>>> Waiting for 19 replies from the CRMd................... OK
>>>
>>> crm_mon:
>>>
>>> ============
>>> Last updated: Wed Nov 30 10:15:09 2011
>>> Stack: openais
>>> Current DC: bbzclnode04 - partition with quorum
>>> Version: 1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
>>> 3 Nodes configured, 3 expected votes
>>> 1 Resources configured.
>>> ============
>>>
>>> Online: [ bbzclnode04 bbzclnode06 bbzclnode07 ]
>>>
>>>
>>> Failed actions:
>>>      clvm:1_start_0 (node=bbzclnode06, call=11, rc=1, status=complete):
>>> unknown error
>>>      clvm:0_start_0 (node=bbzclnode04, call=11, rc=1, status=complete):
>>> unknown error
>>>      clvm:2_start_0 (node=bbzclnode07, call=11, rc=1, status=complete):
>>> unknown error
>>>
>>>
>>> When I look in the log - there is a message which tells me that may be
>>> another clvm process is already running - but it isn't so.
>>>
>>> "clvmd could not create local socket Another clvmd is probably already
>>> running"
>>>
>>> Or is it a permission problem - writing to the filesystem? Is there a
>>> way to get rid of it?
>> You can try to run it manually under strace. It will show you what happens.
>
>
> Here we go:
>
> root at bbzclnode07:~# strace clvmd -d  -I cororsync
> execve("/usr/sbin/clvmd", ["clvmd", "-d", "-I", "cororsync"], [/* 18 vars */]) = 0
> brk(0)                                  = 0x12f7000
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f09dad000
> access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
> open("/etc/ld.so.cache", O_RDONLY)      = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=25864, ...}) = 0
> mmap(NULL, 25864, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9f09da6000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\r\0\0\0\0\0\0"...., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=14768, ...}) = 0
> mmap(NULL, 2109704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f0998b000
> mprotect(0x7f9f0998d000, 2097152, PROT_NONE) = 0
> mmap(0x7f9f09b8d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 
> 0x7f9f09b8d000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/lib/libdevmapper-event.so.1.02.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \24\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=18704, ...}) = 0
> mmap(NULL, 2113872, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f09786000
> mprotect(0x7f9f0978a000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f09989000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 
> 0x7f9f09989000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/lib/libdevmapper.so.1.02.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`L\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=143296, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f09da5000
> mmap(NULL, 2238488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f09563000
> mprotect(0x7f9f09583000, 2097152, PROT_NONE) = 0
> mmap(0x7f9f09783000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x20000) 
> = 0x7f9f09783000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/usr/lib/libcman.so.3", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \23\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=22544, ...}) = 0
> mmap(NULL, 2117800, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f0935d000
> mprotect(0x7f9f09361000, 2097152, PROT_NONE) = 0
> mmap(0x7f9f09561000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4000) = 
> 0x7f9f09561000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/usr/lib/libconfdb.so.4", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\35\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=51680, ...}) = 0
> mmap(NULL, 2148992, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f09150000
> mprotect(0x7f9f0915b000, 2097152, PROT_NONE) = 0
> mmap(0x7f9f0935b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xb000) = 
> 0x7f9f0935b000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/usr/lib/libdlm.so.3", O_RDONLY)  = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\25\0\0\0\0\0\0"...., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=18624, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f09da4000
> mmap(NULL, 2113912, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f08f4b000
> mprotect(0x7f9f08f4f000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f0914e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 
> 0x7f9f0914e000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0Pl\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=135500, ...}) = 0
> mmap(NULL, 2212920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f08d2e000
> mprotect(0x7f9f08d46000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f08f45000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) 
> = 0x7f9f08f45000
> mmap(0x7f9f08f47000, 13368, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 
> 0x7f9f08f47000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/usr/lib/libcpg.so.4", O_RDONLY)  = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\21\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=26928, ...}) = 0
> mmap(NULL, 2122104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f08b27000
> mprotect(0x7f9f08b2d000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f08d2c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5000) = 
> 0x7f9f08d2c000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/usr/lib/libSaLck.so.3", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\16\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=26952, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f09da3000
> mmap(NULL, 2122128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f08920000
> mprotect(0x7f9f08926000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f08b25000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5000) = 
> 0x7f9f08b25000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/usr/lib/libquorum.so.4", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\v\0\0\0\0\0\0"...., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=14480, ...}) = 0
> mmap(NULL, 2109656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f0871c000
> mprotect(0x7f9f0871f000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f0891e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 
> 0x7f9f0891e000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \24\2\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=1677624, ...}) = 0
> mmap(NULL, 3793768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f0837d000
> mprotect(0x7f9f08512000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f08711000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 
> 0x194000) = 0x7f9f08711000
> mmap(0x7f9f08716000, 21352, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 
> 0x7f9f08716000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`Q\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=117720, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f09da2000
> mmap(NULL, 2217512, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f0815f000
> mprotect(0x7f9f0817b000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f0837a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b000) 
> = 0x7f9f0837a000
> mmap(0x7f9f0837c000, 1576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 
> 0x7f9f0837c000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/lib/x86_64-linux-gnu/libudev.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\2401\0\0\0\0\0\0"...., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=55176, ...}) = 0
> mmap(NULL, 2150448, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f07f51000
> mprotect(0x7f9f07f5e000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f0815d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc000) = 
> 0x7f9f0815d000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/usr/lib/libcoroipcc.so.4", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\23\0\0\0\0\0\0"...., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=22872, ...}) = 0
> mmap(NULL, 2118040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f07d4b000
> mprotect(0x7f9f07d50000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f07f4f000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4000) = 
> 0x7f9f07f4f000
> close(3)                                = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f09da1000
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/lib/x86_64-linux-gnu/librt.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200!\0\0\0\0\0\0"...., 832) = 832
> fstat(3, {st_mode=S_IFREG|0644, st_size=31752, ...}) = 0
> mmap(NULL, 2129016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9f07b43000
> mprotect(0x7f9f07b4a000, 2093056, PROT_NONE) = 0
> mmap(0x7f9f07d49000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 
> 0x7f9f07d49000
> close(3)                                = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f09da0000
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f09d9e000
> arch_prctl(ARCH_SET_FS, 0x7f9f09d9e7c0) = 0
> mprotect(0x7f9f07d49000, 4096, PROT_READ) = 0
> mprotect(0x7f9f07f4f000, 4096, PROT_READ) = 0
> mprotect(0x7f9f0815d000, 4096, PROT_READ) = 0
> mprotect(0x7f9f0837a000, 4096, PROT_READ) = 0
> mprotect(0x7f9f08711000, 16384, PROT_READ) = 0
> mprotect(0x7f9f0891e000, 4096, PROT_READ) = 0
> mprotect(0x7f9f08b25000, 4096, PROT_READ) = 0
> mprotect(0x7f9f08d2c000, 4096, PROT_READ) = 0
> mprotect(0x7f9f08f45000, 4096, PROT_READ) = 0
> mprotect(0x7f9f0914e000, 4096, PROT_READ) = 0
> mprotect(0x7f9f0935b000, 4096, PROT_READ) = 0
> mprotect(0x7f9f09561000, 4096, PROT_READ) = 0
> mprotect(0x7f9f09783000, 4096, PROT_READ) = 0
> mprotect(0x7f9f09989000, 4096, PROT_READ) = 0
> mprotect(0x7f9f09b8d000, 4096, PROT_READ) = 0
> mprotect(0x67f000, 4096, PROT_READ)     = 0
> mprotect(0x7f9f09daf000, 4096, PROT_READ) = 0
> munmap(0x7f9f09da6000, 25864)           = 0
> set_tid_address(0x7f9f09d9ea90)         = 2847
> set_robust_list(0x7f9f09d9eaa0, 0x18)   = 0
> futex(0x7ffff6d7cb7c, FUTEX_WAKE_PRIVATE, 1) = 0
> futex(0x7ffff6d7cb7c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 7f9f09d9e7c0) = -1 
> EAGAIN (Resource temporarily unavailable)
> rt_sigaction(SIGRTMIN, {0x7f9f08d346c0, [], SA_RESTORER|SA_SIGINFO, 0x7f9f08d3e060}, NULL, 8) = 0
> rt_sigaction(SIGRT_1, {0x7f9f08d34750, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f9f08d3e060}, 
> NULL, 8) = 0
> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
> getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
> statfs("/selinux", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=8285134, f_bfree=7803671, 
> f_bavail=7382807, f_files=2105344, f_ffree=2016160, f_fsid={-1739579276, -1310995585}, 
> f_namelen=255, f_frsize=4096}) = 0
> brk(0)                                  = 0x12f7000
> brk(0x1318000)                          = 0x1318000
> open("/proc/filesystems", O_RDONLY)     = 3
> fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f09dac000
> read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tb"..., 1024) = 325
> read(3, "", 1024)                       = 0
> close(3)                                = 0
> munmap(0x7f9f09dac000, 4096)            = 0
> getuid()                                = 0
> geteuid()                               = 0
> socket(PF_FILE, SOCK_STREAM, 0)         = 3
> connect(3, {sa_family=AF_FILE, path="/var/run/lvm/clvmd.sock"}, 110) = -1 ENOENT (No such file or 
> directory)
> close(3)                                = 0
> open("/etc/localtime", O_RDONLY)        = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=2309, ...}) = 0
> fstat(3, {st_mode=S_IFREG|0644, st_size=2309, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9f09dac000
> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\10\0\0\0\10\0\0\0\0"..., 4096) = 2309
> lseek(3, -1467, SEEK_CUR)               = 842
> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\t\0\0\0\t\0\0\0\0"..., 4096) = 1467
> close(3)                                = 0
> munmap(0x7f9f09dac000, 4096)            = 0
> write(2, "CLVMD[9d9e7c0]: Nov 30 12:51:22 ", 32CLVMD[9d9e7c0]: Nov 30 12:51:22 ) = 32
> write(2, "CLVMD started\n", 14CLVMD started
> )         = 14
> unlink("/var/run/lvm/clvmd.sock")       = -1 ENOENT (No such file or directory)
> umask(077)                              = 022
> socket(PF_FILE, SOCK_STREAM, 0)         = 3
> fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
> fcntl(3, F_GETFL)                       = 0x2 (flags O_RDWR)
> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
> bind(3, {sa_family=AF_FILE, path="/var/run/lvm/clvmd.sock"}, 110) = -1 ENOENT (No such file or 
> directory)
> write(2, "  ", 2  )                       = 2
> write(2, "can't bind local socket: No such"..., 50can't bind local socket: No such file or 
> directory) = 50
> write(2, "\n", 1
> )                       = 1
> close(3)                                = 0
> unlink("/var/run/lvm/clvmd.sock")       = -1 ENOENT (No such file or directory)
> umask(022)                              = 077
> exit_group(2)                           = ?
>
>
>
> I created now the directory "/var/run/lvm" . It wasn't there - work for the package maintainer.
>
> IT WORKS NOW - thanks for Your help Vladislav :-))
>
>>
>>> Shell I use a different distro - our install from source?
>>>
>>>
>>> Am 24.11.2011 22:59, schrieb Andreas Kurz:
>>>> Hello,
>>>>
>>>> On 11/24/2011 10:12 PM, Vadim Bulst wrote:
>>>>> Hi Andreas,
>>>>>
>>>>> I changed my cib:
>>>>>
>>>>> node bbzclnode04
>>>>> node bbzclnode06
>>>>> node bbzclnode07
>>>>> primitive clvm ocf:lvm2:clvmd \
>>>>>          params daemon_timeout="30"
>>>>> primitive dlm ocf:pacemaker:controld
>>>>> group g_lock dlm clvm
>>>>> clone g_lock-clone g_lock \
>>>>>          meta interleave="true"
>>>>> property $id="cib-bootstrap-options" \
>>>>>          dc-version="1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
>>>>>          cluster-infrastructure="openais" \
>>>>>          expected-quorum-votes="3" \
>>>>>          no-quorum-policy="ignore" \
>>>>>          stonith-enabled="false" \
>>>>>          last-lrm-refresh="1322049979
>>>>>
>>>>> but no luck at all.
>>>> I assume you did at least a cleanup on clvm and it still does not work
>>>> ... next step would be to grep for ERROR in your cluster log and look
>>>> for other suspicious messages to find out why clvm is not that motivated
>>>> to start.
>>>>
>>>>> "And use Corosync 1.4.x with redundant rings and automatic ring recovery
>>>>> feature enabled."
>>>>>
>>>>> I got two interfaces per server - there are bonded together and bridged
>>>>> for virtualization.  Only one untagged vlan. I tried to give a tagged
>>>>> Vlan Bridge a Address but didn't worked. My network conf looks like that:
>>>> One ore two extra nics are quite affordable today to build e.g. a direct
>>>> connection between the nodes (if possible)
>>>>
>>>> Regards,
>>>> Andreas
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>>
>>> -- 
>>> Mit freundlichen Grüßen
>>>
>>> Vadim Bulst
>>> Systemadministrator BBZ
>>>
>>> Biotechnologisch-Biomedizinisches Zentrum
>>> Universität Leipzig
>>> Deutscher Platz 5, 04103 Leipzig
>>> Tel.: 0341 97 - 31 307
>>> Fax : 0341 97 - 31 309
>>>
>>>
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
>
>

-- 
Mit freundlichen Grüßen

Vadim Bulst
Systemadministrator BBZ

Biotechnologisch-Biomedizinisches Zentrum
Universität Leipzig
Deutscher Platz 5, 04103 Leipzig
Tel.: 0341 97 - 31 307
Fax : 0341 97 - 31 309