Re: git push sends more objects than it needs to

From: Linus Torvalds <torvalds@osdl.org>
Date: Oct 31, 2005, 19:36 UTC
Message-ID: <Pine.LNX.4.64.0510311119170.27915@g5.osdl.org>
In-Reply-To: <Pine.LNX.4.64.0510311035310.27915@g5.osdl.org>

On Mon, 31 Oct 2005, Linus Torvalds wrote:

Show 8 quoted lines

> > 
> > Now the "unpack" on kernel.org did the right thing and noticed
> > that over 9000 of the objects were already in the packfile.  But
> > I wonder if it couldn't have been smarter and not sent them?
> 
> It should have been smarter, but I suspect you got caught by the fact that 
> kernel.org by default has git-0.99.8f on it, which has the old 
> pre-multi_ack code to figure out what the common commit was.

Oh, actually, I take that back.

This is not the multi-ack code at all, I suspect.

The problem is totally different: you copied the new pack to your master.kernel.org repository, but you never updated any branches there.

So you had the objects, but git had no way of knowing. The pack generation doesn't look at what _objects_ you have, it looks at what _refs_ you have. And your refs were all to the old state.

So git actually did everything right (you can never rely on objects: you may have a partial object list due to some earlier incomplete pull/push).

This is actually easy enough to fix up in one of several ways:

 a) The "don't do that then" approach:

    Don't go behind git's back and add objects on your own, and expect git 
    to realize what you did. ;^p

 b) The "live with it" approach:

    You copied the pack by hand, and that will keep git-unpack-objects 
    from duplicating the objects, but you'll still waste time and network 
    when trying pushing the objects (just once, though).

    Ie this is what happened this time: nothing really lost, and the end 
    result is fine. Now you know why it happened, and you're fine.

 c) The "I'm smarter than git" approach:

    When you copy my objects, copy my reference to the top-of-tree too 
    (and rename it). NOTE! Now you need to be really careful, and you need 
    to make sure you copy _all_ the objects, because if you screw this up, 
    your repo will be missing objects that you claim are there, and it 
    will be all your fault.

    I really don't advocate this approach at all. It's certainly doable, 
    but it's also the only approach where you can really screw up.

 d) Just let git do it for you.

    Copy the pack-files, or add my object directory as an "alternates" for 
    your object directory, do the "git prune-objects", and then _locally_ 
    on master.kernel.org just do something like

	git fetch ..linus-directory.. master:linus

    which will still create the unnecessary pack-file and unpack it into 
    nothingness (since you have the objects in the pack-file you copied by 
    hand), but at least it won't eat any network bandwidth, and it will do 
    the right thing if it turns out that I've pushed something after doing 
    the pack-file, and fetch those individual objects in _addition_ to the 
    pack-file you snarfed by hand. It will also obviously update a ref in
    your tree (the "linus" branch), so now when you send stuff later, it 
    will know all about the objects you already have.

 e) Re-create the tree entirely

    Blow away your tree on master entirely, just re-create it locally with 
    "git clone -l -s" from my tree (which will do all the "alternates" 
    object files for you), and then populate the result with a simple push
    from your home tree.

 f) any number of variations on a theme. IOW, there are endless ways to do 
    this.

Hmm?

		Linus