Re: What broke snapshots now?

On Sun, 10 Jul 2005, David Woodhouse wrote:

Which script is this? I'm looking at your scripts, but "cg-feedmaillist.sh" is unreadable for me, so I can't see all of it.

Anyway, it's possible that this is a temporary problem: one of the issues is that since you seem to be using the "rsync:" protocol for updating things, what happens is that if the mirroring is off a bit, you may have gotten a new head, but not all the objects. Then you'd get exactly this.

I don't know how to solve that, really. It's fundamental to any mirroring that doesn't guarantee some kind of ordering, ie the dumb kind.

All the "native git" pack-file synchronization protocols should be strictly ordered, ie they will unpack and write the new objects _before_ they write the new refs that point to them..

But you can just imagine what happens if the mirror script is busy
mirroring at the same time, and has already mirrored the object
directories before we wrote all of them, but then gets to the refs/
directory by the time we have written the new refs (and all the new object
- but it didn't notice that because it had already mirrored the object
dirs). 

See what I'm saying? The thing fixes itself after a while when you try again and the next round of mirroring has gotten the objects it missed the last time, but there fundamentally isn't anything to be done about the fact that the mirroring wasn't "atomic". Only a git-aware mirror could do an atomic snapshot.

I'm actually surprised that people haven't complained more about this than they have. I know it's happened a couple of times: it bit Dave Jones' scripts once, and it's caused webgit to sometimes report "permission denied" problems (or whatever) and then showing empty logs.

The fact that my upload scripts _do_ upload in the right order, together with the fact that rsync normally syncs alphabetically (ie "objects" before "refs") means that we _normally_ don't see this, but it's occasionally unavoidable when I happen to do a new push while a previous mirror sweep is going on.

Of course, in this case it may also be just some cogito thing that doesn't like pack-files. The fact that it complains about an object that is not actually either of the "heads" of the diff makes me suspect it's really a missing object, though, which in turn makes me suspect the above mirror mis-feature.

			Linus