Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file

From: Nicolas Pitre <nico@cam.org>
Date: Jan 7, 2009, 15:31 UTC
Message-ID: <alpine.LFD.2.00.0901071008150.26118@xanadu.home>
In-Reply-To: <1231314099.8870.415.camel@starfruit>

On Tue, 6 Jan 2009, R. Tyler Ballance wrote:

Show 27 quoted lines

> On Tue, 2009-01-06 at 20:54 -0800, Linus Torvalds wrote:
> > 
> > On Tue, 6 Jan 2009, R. Tyler Ballance wrote:
> > > 
> > > I'll back the patch out and redeploy, it's worth mentioning that a
> > > coworker of mine just got the issue as well (on 1.6.1). He was able to
> > > `git pull` and the error went away, but I doubt that it "magically fixed
> > > itself"
> > 
> > Quite frankly, that behaviour sounds like a disk _cache_ corruption issue. 
> > The fact that some corruption "comes and goes" and sometimes magically 
> > heals itself sounds very much like some disk cache problem, and then that 
> > particular part of the cache gets replaced and then when re-populated it 
> > is magically correct.
> > 
> > We had that in one case with a Linux NFS client, where a rename across 
> > directories caused problems.
> > 
> > This was a networked filesystem on OS X, right? File caching is much more 
> > "interesting" in networked filesystems than it is in normal private 
> > on-disk ones.
> 
> Not quite, what I meant was that some users (not all) who've experienced
> this issue are using Samba to copy files over directly into the Git
> repository. I was mentioning this in case somewhere between Finder,
> Samba, ext3 and Git, some file system change events were pissing Git off
> and causing it.

As long as those files are not within the .git directory that should be fine.

> I don't think this is the case as the coworker that I
> mentioned earlier doesn't use Samba and neither do I (we both experience
> the issue today, mine disappeared by upgrading to 1.6.1, his by `git
> pull`).

Problem is that none of that "makes sense". If a real git corruption was there, it wouldn't disappear without explicit action from your part. What git v1.6.1 is able to do over earlier versions is to still function properly if some corrupted objects have a redundant copy in the same repository, but it wouldn't stop complaining about the existence of corrupted data. Doing a 'git gc' or 'git repack -a' might get rid of the corruption. And a 'git pull' might hide it if the received pack during the pull operation happens to contain another copy of the object that was corrupted before, but it wouldn't prevent 'git fsck --full' from seeing it.

The fact that you cannot reproduce the corruption issues after unarchiving a repository that was known to have problems right before it was archived is really really strange. That does not rule out git bugs of course, but at least this shows that no actual corruption on disk was initially involved.

Again, I'd suggest you perform your git usage within a script session so to capture the exact operation performed and error messages produced when/if similar problems do occur again. Otherwise we're only running after our tail.

Nicolas