threads / discuss / 10802

Dose git-fetch need --reference option like git-clone?

Subject: Dose git-fetch need --reference option like git-clone?

## tl;dr

5 messages between Nov 11, 2007 and Nov 11, 2007.

replies: 4people: 3as markdown or json

Yin Ping· Nov 11, 2007, 08:09 UTC · lore

I want to track remote repsotory (say remoteA) on my local repository (say localB), so i do the following in directory localB $ git remote add remoteA git://remoteAUrl $ git fetch remoteA This will fetch all objects from git://remoteAUrl if localB and remoteA don't have common objects.

If I already have a cloned remoteA on local machine (say /path/to/remoteACloned), I want to do following to reduce the net traffic as git-clone: git fetch --reference /path/to/remoteACloned remotedA

Is this reasonable? Or is there already a resolution for this case?
-- 
Ping Yin
Junio C Hamano· Nov 11, 2007, 08:36 UTC · re: Yin Ping · lore

Re: Dose git-fetch need --reference option like git-clone?

"Yin Ping" <pkufranky@gmail.com> writes:
Show 6 quoted lines
> If I already have a cloned remoteA on local machine (say
> /path/to/remoteACloned), I want to do following to reduce the net
> traffic as git-clone:
> git fetch --reference /path/to/remoteACloned remotedA
>
> Is this reasonable? Or is there already a resolution for this case?

Resolution, meaning "No, that's not a good thing and we refuse to support such an option"?

No, there is no such thing.

I think what you are talking about is a reasonable thing to want. It would have been easier to hack in back when git-fetch was a script, but now you would need to work a bit harder in the C code. On the other hand, however, I suspect the resulting code would be cleaner without having to actually create and delete temporary refs in refs/reference-tmp/ namespace but fake them only in-core, with a proper transport API enhancements.

But if you only want a quick-and-dirty workaround, you can manually do refs/reference-tmp and objects/info/alternates dance that is done by git-clone before running a git-fetch from such "nearby" remote.

I strongly suspect that an approach not to use temporary refs on the filesystem would be needed for robustness if we were to do this for real inside git-fetch as a real solution, so that we would not leave them behind when interrupted. This is not such a big deal for git-clone which knows it always starts from a void and cleaning up the mess is not a big issue, but matters for the use of git-fetch under discussion, which _will_ work inside an already initialized repository.

Yin Ping· Nov 11, 2007, 11:38 UTC · re: Junio C Hamano · lore

Re: Dose git-fetch need --reference option like git-clone?

On Nov 11, 2007 4:36 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 15 quoted lines
>
> No, there is no such thing.
>
> I think what you are talking about is a reasonable thing to
> want.  It would have been easier to hack in back when git-fetch
> was a script, but now you would need to work a bit harder in the
> C code.  On the other hand, however, I suspect the resulting
> code would be cleaner without having to actually create and
> delete temporary refs in refs/reference-tmp/ namespace but fake
> them only in-core, with a proper transport API enhancements.
>
> But if you only want a quick-and-dirty workaround, you can
> manually do refs/reference-tmp and objects/info/alternates dance
> that is done by git-clone before running a git-fetch from such
> "nearby" remote.

Now my workaround is $ git remote add remoteA path/to/remoteACloned $ git fetch remoteA Then update .git/config to let remote.remoteA.url=git://remoteAUrl $ git fetch

-- 
Ping Yin
Mike Hommey· Nov 11, 2007, 08:38 UTC · re: Yin Ping · lore

Re: Dose git-fetch need --reference option like git-clone?

On Sun, Nov 11, 2007 at 04:09:55PM +0800, Yin Ping wrote:
Show 13 quoted lines
> I want to track remote repsotory (say remoteA) on my local repository
> (say localB), so i do the following in directory localB
> $ git remote add remoteA git://remoteAUrl
> $ git fetch remoteA
> This will fetch all objects from git://remoteAUrl if localB and
> remoteA don't have common objects.
> 
> If I already have a cloned remoteA on local machine (say
> /path/to/remoteACloned), I want to do following to reduce the net
> traffic as git-clone:
> git fetch --reference /path/to/remoteACloned remotedA
> 
> Is this reasonable? Or is there already a resolution for this case?

It would probably be reasonable to have this on git-remote. Anyways, you can easily do it yourself by editing .git/objects/info/alternates and adding /path/to/remoteACloned in it. You can happily git fetch after that.

Mike
Junio C Hamano· Nov 11, 2007, 09:19 UTC · re: Mike Hommey · lore

Re: Dose git-fetch need --reference option like git-clone?

Mike Hommey <mh@glandium.org> writes:
> It would probably be reasonable to have this on git-remote. Anyways, you
> can easily do it yourself by editing .git/objects/info/alternates and
> adding /path/to/remoteACloned in it. You can happily git fetch after
> that.
Not really.
There are two parts in the --reference dance git-clone plays:
 - After the reference clone is done, obviously because you are
   avoiding to actually transfer the data, you need to set up
   alternates to permanently borrow from the reference
   repository.
 - Because the fetch/clone procedure considers an object that is
   locally accessible (either it locally exists, or exists in an
   alternate object store) "complete" (meaning, there is no need
   to fetch the objects that are reachable from it) ONLY when it
   is reachable from one of the refs in the repository, the
   above alone would not prevent fetch/clone from actually
   transferring the objects.  For this reason, git-clone copies
   the refs that exist in the reference repository as temporary
   refs.  This ensures that the objects that the clone borrows
   from the reference repository are considered "complete"
   during the clone process.

The original design of git differenciated the object store (that is, .git/objects/) and a repository (.git). A repository has an object store associated with it, but the design allowed an object store to be shared among multiple repositories by symlinking. This is why alternates point at objects/ directory of the reference repository, not one level above.

Today, no git tool creates such a "shared object store" layout, so if the original design were "an object store belongs only to one repository, and the ../refs directory relative to it always is the only set of refs that defines the completeness of the objects within it. No sharing of object store across repositories using symlink is allowed", we could redefine the completeness rule to also follow the alternates and doing so would eliminate the need for the latter "temporary ref" trick git-clone plays. But until we officially declare that a set of old repositories that uses the "shared object store" layout are not supported (and give a reasonably migration path), we unfortunately cannot do so.

← back to recent threads