Re: efficient cloning

From: Shawn Pearce <spearce@spearce.org>
Date: Mar 19, 2006, 22:31 UTC
Message-ID: <20060319223111.GA20464@spearce.org>
In-Reply-To: <m3r74ykue7.fsf@lugabout.cloos.reno.nv.us>

James Cloos <cloos@jhcloos.com> wrote:

Show 32 quoted lines

> Is there a way to accomplish the effect of this script w/o having to
> download any unnecessary objects?
> 
> ==================================================
> #!/bin/bash
> 
> lt="/gits/linux-2.6/.git"
> 
> if [ $# -ne 2 ]; then
>     echo >&2 "Usage: $0 <repo> <target-dir>"
>     exit 1
> fi
> 
> git-clone $1 $2
> mkdir -p $2/objects/info
> {
>  test -f "$lt/objects/info/alternates" &&
>  cat "$lt/objects/info/alternates";
>  echo "$lt/objects"
> } >"$2/objects/info/alternates"
> 
> cd $2
> git-repack -a -d -s
> git-prune-packed
> ==================================================
> 
> I tried to modify git-clone to add an alternates file before calling
> fetch, but that file just gets deleted.
> 
> I presume I need to clone -s -l the local alternate, re-parent it to
> the new URL and grab anything missing, but how can I assure that it
> results in exactly the same repo as this script?

Exactly right. There was some discussion about this perhaps just two weeks back and it become clear that the easiest way to clone through a thin straw is to use `git clone -s -l' from a locally available repository which is ``close''[*1*] to the remote you are going to actually trying to clone from, edit .git/remotes/origin to have the correct URL: and Pull: lines, then `git-pull origin' to bring down whatever you don't have yet. This won't miss any objects so it will result in the same repository as a clone would have[*2*].

Footnotes:

  [*1*] Here ``close'' means probably related to the same project.
  Meaning if you are cloning the Linux kernel at least start with
  another kernel repository and not say the GIT repository.  :-)
  The more your original repository has in common with the remote you
  are trying to pull from the less that will need to be downloaded.

  [*2*] This isn't entirely true.  During a normal clone everything
  is pulled down into a single pack. Using this strategy the missing
  objects that are downloaded will be loose; a git-repack after
  the pull might be a good idea to pull them into a pack.

-- 
Shawn.