[Pacemaker] DRBD < LVM < EXT4 < NFS performance
Raoul Bhatia [IPAX]
r.bhatia at ipax.at
Sun May 20 17:20:20 UTC 2012
On 2012-05-20 12:05, Christoph Bartoschek wrote:
> Hi,
>
> we have a two node setup with drbd below LVM and an Ext4 filesystem that is
> shared vi NFS. The system shows low performance and lots of timeouts
> resulting in unnecessary failovers from pacemaker.
>
> The connection between both nodes is capable of 1 GByte/s as shown by iperf.
> The network between the clients and the nodes is capable of 110 MByte/s. The
> RAID can be filled with 450 MByte/s.
>
> Thus I would expect to have a write performance of about 100 MByte/s. But dd
> gives me only 20 MByte/s.
>
> dd if=/dev/zero of=bigfile.10G bs=8192 count=1310720
> 1310720+0 records in
> 1310720+0 records out
> 10737418240 bytes (11 GB) copied, 498.26 s, 21.5 MB/s
to give you some numbers to compare:
I've got a small XFS file system, which i'm currently testing with.
Using a single thread and NFS4 only:
my configuration:
nfsserver:
# exportfs -v
/data/export
192.168.100.0/24(rw,wdelay,no_root_squash,no_subtree_check,fsid=1000)
nfsclient mount
192.168.100.200:/data/export on /mnt type nfs
(rw,nosuid,nodev,nodiratime,relatime,vers=4,addr=192.168.100.200,clientaddr=192.168.100.107)
via network (1gbit connection for both drbd sync and nfs)
# dd if=/dev/zero of=bigfile.10G bs=6192 count=1310720
1310720+0 records in
1310720+0 records out
8115978240 bytes (8.1 GB) copied, 140.279 s, 57.9 MB/s
on the same machine so that 1gbit is for drbd only:
# dd if=/dev/zero of=bigfile.10G bs=6192 count=1310720
1310720+0 records in
1310720+0 records out
8115978240 bytes (8.1 GB) copied, 70.9297 s, 114 MB/s
Maybe this numbers and configuration helps?
Cheers,
Raoul
> While the slow dd runs there are timeouts on the server resulting in a
> restart of some resources. In the logfile I also see:
>
> [329014.592452] INFO: task nfsd:2252 blocked for more than 120 seconds.
> [329014.592820] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
> this message.
> [329014.593273] nfsd D 0000000000000007 0 2252 2
> 0x00000000
...
> Has anyone an idea what could cause such problems? I have no idea for
> further analysis.
i haven't seen such issue during my current tests.
> Is ext4 unsuitable for such a setup? Or is the linux nfs3 implementation
> broken? Are buffers too large such that one has too wait too long for a
> flush?
Maybe I'll have the time to switch form xfs to ext4 and retest
during the next couple of days. But I cannot guarantee anything.
Maybe you could try switching to XFS instead?
Cheers;
Raoul
--
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at
Technischer Leiter
IPAX - Aloy Bhatia Hava OG web. http://www.ipax.at
Barawitzkagasse 10/2/2/11 email. office at ipax.at
1190 Wien tel. +43 1 3670030
FN 277995t HG Wien fax. +43 1 3670030 15
____________________________________________________________________
More information about the Pacemaker
mailing list