Re: Careful object writing..

From: Jan Harkes <jaharkes@cs.cmu.edu>
Date: May 3, 2005, 20:00 UTC
Message-ID: <20050503200034.GA16104@delft.aura.cs.cmu.edu>
In-Reply-To: <Pine.LNX.4.58.0505031204030.26698@ppc970.osdl.org>

On Tue, May 03, 2005 at 12:15:08PM -0700, Linus Torvalds wrote:

Show 10 quoted lines

> I just pushed out the commit that tries to finally actually write the sha1
> objects the right way in a shared object directory environment.
> 
> I used to be lazy, and just do "O_CREAT | O_EXCL" on the final name, but
> that obviously is not very nice when it can result in other people seeing
> objects that haven't been fully finalized yet.
> 
> So now I do it "right", and create a temporary file in the "top" object
> directory, and then when it's all done, I do a "link()" to the final place
> and unlink the original.

Annoyingly until this commit, git has been just about the perfect SCM system to run on top of Coda. Almost every other SCM can and will get conflicts on it's repository files, which are pretty much impossible to resolve (just try merging two diverging copies of an RCS archive..)

But the only conflicts we ever see with git are when two people create the same SHA1 object. And if the contents are in fact identical this conflict will be trivially resolved.

I tried to pull in the latest version of your tree, but it doesn't look like this commit has propagated to rsync.kernel.org yet. Hopefully you will accept a small patch (should be < 5 lines) that makes git work nicely when Coda complains about the cross-directory hardlink without affecting the reliability of using link/unlink on normal filesystems.

Jan

Show 34 quoted lines

> 
> I also change the permission to 0444 before it gets its final name.
> 
> Two notes:
> 
>  - because the objects all get created initially in .git/objects rather 
>    than in the subdirectory they get moved to, you can't use symlinks 
>    to other filesystems for the 256 object subdirectories. The object 
>    directory has to be one filesystem (but it doesn't have to be the same 
>    one as you actually keep your working ddirectories on, of course)
> 
>  - The upside of this is that filesystem block allocators should do the 
>    right thing. Instead of spreading the objects out (because they are in 
>    different directories), they should be created together.
> 
> Anyway, somebody should double-check the thing. It _should_ now work
> correctly over NFS etc too, and everything should be nice and atomic (and
> with any half-way decent filesystem, it also means that even if you have a
> system crash in the middle, you'll never see half-created objects).
> 
> NOTE NOTE NOTE! I have _not_ updated all the helper stuff that also write 
> objects. So things like "git-http-pull" etc will still write objects 
> directly into the object directory, and that can cause problems with 
> shared usage. Same goes for "write_sha1_from_fd()" that rpull.c uses. I 
> hope somebody will take a look at those issues..
> 
> Anyway, at least the really core operations should now really be
> "thread-safe" in a shared object directory environment.
> 
> 		Linus
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html