[Pacemaker] DRBD Recovery Policies

[Lars Ellenberg]

On Fri, Mar 12, 2010 at 09:48:57AM -0000, Darren.Mansell at opengi.co.uk wrote:
> The odd thing is - it didn't. From my test, it failed back, re-promoted
> NodeA to be the DRBD master and failed all grouped resources back too.
> 
> Everything was working with the ~7GB of data I had put onto NodeB while
> NodeA was down, now available on NodeA...
> 
> /proc/drbd on the slave said Secondary/Primary UpToDate/Inconsistent
> while it was syncing data back - so it was able to mount the
> inconsistent data on the primary node and access the files that hadn't
> yet sync'd over?! I mounted a 4GB ISO that shouldn't have been able to
> be there yet and was able to access data inside it..
> 
> Is my understanding of DRBD limited and it's actually able to provide
> access to not fully sync'd files over the network link or something?
> 
> If so - wow.

Of course it does.
 ;)

It has to.
It would not survive a local disk failure on a Primary, otherwise.

Though obviously this has performance impact,
reading from remote is most often less performant than reading locally.

(To be technically correct,
it does not care about files,
but only about blocks.)

> I believe that this is handled by DRBD by fencing the Master/Slave 
> resource during resync using Pacemaker. See 
> http://www.drbd.org/users-guide/s-pacemaker-fencing.html. This would 
> prevent Node A to promote/start services with outdated data 
> (fence-peer), and it would be forced to wait with takeover until the 
> resync is completed (after-resync-target).

For that to work as expected you should
 * not have location preference constraints on the master role directly,
   or give them a very low score. Recommended would be to place a
   location preference, if needed, not on DRBD Master role, but on some
   depending service (Filesystem for example).
 * a recent version of the drbd ocf resource agent and crm-fence-peer scripts
   (best to simply use drbd 8.3.7 [or later, in case someone pulls this
   from the archives in the future...])

-- 
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.