Re: broken repo after power cut

From: Richard Weinberger <richard@nod.at>
Date: Jun 22, 2015, 11:19 UTC
Message-ID: <5587EF5F.90207@nod.at>
In-Reply-To: <20150622003551.GP29480@thunk.org>

Am 22.06.2015 um 02:35 schrieb Theodore Ts'o:

Show 10 quoted lines

> On Sun, Jun 21, 2015 at 03:07:41PM +0200, Richard Weinberger wrote:
> 
>>> I was then shocked to learn that ext4 apparently has a default
>>> setting that allows it to truncate files upon power failure
>>> (something about a full journal vs a fast journal or some such)
> 
> s/ext4/all modern file systems/
> 
> POSIX makes **no guarantees** about what happens after a power failure
> unless you use fsync() --- which git does not do by default (see below).

Thanks for pointing this out.

Show 17 quoted lines

> The bottome lins is that if you care about files being written, you
> need to use fsync().  Should git use fsync() by default?  Well, if you
> are willing to accept that if your system crashes within a second or
> so of your last git operation, you might need to run "git fsck" and
> potentially recover from a busted repo, maybe speed is more important
> for you (and git is known for its speed/performance, after all. :-)
> 
> The actual state of the source tree would have been written using a
> text editor which tends to be paranoid about using fsync (at least, if
> you use a real editor like Emacs or Vi, as opposed to the toy notepad
> editors shipped with GNOME or KDE :-).  So as long as you know what
> you're doing, it's unlikely that you will actually lose any work.
> 
> Personally, I have core.fsyncobjectfiles set to yes in my .gitconfig.
> Part of this is because I have an SSD, so the speed hit really doesn't
> bother me, and needing to recover a corrupted git repository is a pain
> (although I have certainly done it in the past).

I think core.fsyncObjectFiles documentation really needs an update. What about this one?

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 43bb53c..b08fa11 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -693,10 +693,16 @@ core.whitespace::
 core.fsyncObjectFiles::
 	This boolean will enable 'fsync()' when writing object files.
 +
-This is a total waste of time and effort on a filesystem that orders
-data writes properly, but can be useful for filesystems that do not use
-journalling (traditional UNIX filesystems) or that only journal metadata
-and not file contents (OS X's HFS+, or Linux ext3 with "data=writeback").
+For performance reasons git does not call 'fsync()' after writing object
+files. This means that after a power cut your git repository can get
+corrupted as not all data hit the storage media. Especially on modern
+filesystems like ext4, xfs or btrfs this can happen very easily.
+If you have to face power cuts and care about your data it is strongly
+recommended to enable this setting.
+Please note that git's behavior used to be safe on ext3 with data=ordered,
+for any other filesystems or mount settings this is not the case as
+POSIX clearly states that you have to call 'fsync()' to make sure that
+all data is written.

 core.preloadIndex::
 	Enable parallel index preload for operations like 'git diff'

--
Thanks,
//richard