Re: Should branches be objects?

From: Jeff King <peff@peff.net>
Date: Jun 20, 2014, 02:27 UTC
Message-ID: <20140620022706.GB31908@sigill.intra.peff.net>
In-Reply-To: <20140620010147.GZ8557@google.com>

On Thu, Jun 19, 2014 at 06:01:47PM -0700, Jonathan Nieder wrote:

Show 7 quoted lines

> > Speaking of which: are there any power failure corruption cases left
> > in git?  How is this tested?
> 
> What kind of power failure corruption are you talking about?  Git
> usually updates files by writing a completely new file and then
> renaming it into place, so depending on your filesystem this means it
> is very hard or very easy to lose data with a power failure. :)

We use git-core on ext4 at GitHub, and we certainly have seen our share of machines failing unexpectedly. We haven't seen any problems of this nature[1] (but note that we journal data writes; you should also be fine with ordered data writes, but data=writeback is likely disastrous).

Show 5 quoted lines

> If you're on one of those filesystems where it is very easy and you
> lose power a lot, you'll probably want to enable the
> core.fsyncobjectfiles configuration option.  It might be worth adding
> another knob like that for the other files git writes if someone is
> interested.

You probably know this already Jonathan, but to be clear:

Git always fsyncs pack writes. That knob controls fsyncing of loose object files, but nothing else. So ref writes (and writing packed-refs) could be corrupted on a filesystem that doesn't order data and metadata writes (and there is currently no way to tell git to do otherwise).

My recommendation would be to steer clear or reconfigure such systems, but it also would not be very hard to add an optional fsync in those cases.

-Peff

[1] We did have one case where after a crash packfiles would end up
    corrupted, but it turned out to be bad RAM in a battery-backed RAID
    card that was transparently caching (and losing) the writes.
    There's not much git can do when fsync lies to it, nor much the
    kernel can do when the hardware lies to it. :)