Re: Left with empty files after "git stash pop" when system hung

From: Jeff King <peff@peff.net>
Date: Sep 14, 2016, 22:58 UTC
Message-ID: <20160914225828.hlsc2ae4vfghmqv7@sigill.intra.peff.net>
In-Reply-To: <5b203a8e-faa8-9f6e-8cdd-1024194e74a3@thequod.de>

On Tue, Sep 13, 2016 at 11:39:56PM +0200, Daniel Hahler wrote:

Show 19 quoted lines

> I have used "git stash --include-untracked", checked out another branch,
> went back, and "git stash pop"ed the changes.
> Then my system crashed/hung (music that was playing was repeated in a
> loop).  I have waited for some minutes, and then turned it off.
> 
> Afterwards, the repository in question was in a state where all files
> contained in the stash were empty.
> "git status" looked good on first sight: all the untracked and modified
> files were listed there; but they were empty.
> 
>   % git fsck --lost-found
>   error: object file .git/objects/04/1e659b5dbfd3f0be351a782b54743692875aec is empty
>   error: object file .git/objects/04/1e659b5dbfd3f0be351a782b54743692875aec is empty
>   fatal: loose object 041e659b5dbfd3f0be351a782b54743692875aec (stored in .git/objects/04/1e659b5dbfd3f0be351a782b54743692875aec) is corrupt
>   % find .git/objects -size 0|wc -l
>   12
>
> [...]
> The filesystem in question is ext4, and I am using Arch Linux.

Is your filesystem mounted with data=writeback? Git should never write an empty object file; it writes the content to a temporary file, and then hardlinks it into place. If your filesystem does not order data and metadata writes (i.e., the hardlink may get journaled and picked up, even though the data did not hit the disk), then you can end up with empty files. If you set core.fsyncobjectfiles in your config file, then Git will fsync each object write (at the cost of some performance).

> I would have assumed that the "stash pop" operation would be "atomic",
> i.e. it should not remove the stash object before other objects have
> been written successfully.

Stash does not remove any objects at all; it should only be updating the stash reflog to delete the entry (which also happens via write to a tempfile and rename, though I don't think we ever fsync it, even with core.fsyncobjectfiles).

The empty object you found is probably the result of a write too close to the crash. In general I wouldn't expect "stash pop" to write, but I suspect it may in order to populate the index.

> I have removed all empty files in .git/objects and tried to find the
> previous stash with `gitk --all $( git fsck | awk '{print $3}' )` then,
> but it appears to have disappeared.

fsck won't mention the object as dangling if it's reachable from a reflog. Did you try "git stash list" (or just "git log -g refs/stash")?

-Peff