threads / discuss / 3948

git-clone --reference problem?

Subject: git-clone --reference problem?

## tl;dr

5 messages between Apr 23, 2006 and Apr 24, 2006.

replies: 4people: 3as markdown or json

David Woodhouse· Apr 23, 2006, 16:34 UTC · lore

Should I expect cloning with alternates using '--reference' to be transitive?

Using '--reference clone1 --reference linux-2.6' for the second clone works OK, but surely it ought to work with just '--reference clone1'?

It doesn't work -- it can't find the objects which are in the original linux-2.6 tree....

 (linux-2.6 contains a current copy of linus' tree)

shinybook /shiny/git $ git-clone --reference linux-2.6 git://git.infradead.org/mtd-2.6.git clone1
Checking files out...
 100% (19552/19552) done
shinybook /shiny/git $ git-clone --reference clone1 git://git.infradead.org/mtd-2.6.git clone2
error: refs/reference-tmp/refs/tags/v2.6.13-rc7 does not point to a valid commit object!
error: refs/reference-tmp/refs/tags/v2.6.13-rc2 does not point to a valid commit object!
error: refs/reference-tmp/refs/tags/v2.6.14 does not point to a valid commit object!
   < ..... >
error: refs/reference-tmp/refs/tags/v2.6.14-rc1 does not point to a valid commit object!
error: refs/reference-tmp/refs/tags/v2.6.15-rc2 does not point to a valid commit object!
error: refs/reference-tmp/refs/tags/v2.6.13-rc6 does not point to a valid commit object!
error: refs/reference-tmp/refs/tags/v2.6.13-rc4 does not point to a valid commit object!
error: Could not read 7e6684741b15e98dd52bd0dbcb08a4eb69857c23
error: Could not read 7e6684741b15e98dd52bd0dbcb08a4eb69857c23
error: Could not read 7260448207915a170bb812f8639a90a30329adce
error: Could not read 7260448207915a170bb812f8639a90a30329adce
error: Could not read acf8d9bd83be879328c558400d94ee61fc229672
error: Could not read acf8d9bd83be879328c558400d94ee61fc229672
error: Could not read c7270e76718f635bed33afe6c4751a270b2d031b
error: Could not read c7270e76718f635bed33afe6c4751a270b2d031b
error: Could not read 0a8763981774041f3fee0a71e016dcaa096fa3f8
error: Could not read 0a8763981774041f3fee0a71e016dcaa096fa3f8
error: Could not read 4a47136ddde07488a5741e6be267b8f5bddc407b
error: Could not read 4a47136ddde07488a5741e6be267b8f5bddc407b
error: Could not read 6cad1d2664ec091ba5a6f3e3e552cf550ec8c2e0
error: Could not read 6cad1d2664ec091ba5a6f3e3e552cf550ec8c2e0
   < ..... >
error: Could not read a0d4c65cd5642e7f7bbd2f806a69d20b2e43edc4
error: Could not read 611dd7a56906343db81c9fe340be0eb78c4ec260
error: Could not read 611dd7a56906343db81c9fe340be0eb78c4ec260
error: Could not read 227b4665ae0105348868d7a0a577209c8d16c6e3
error: Could not read 227b4665ae0105348868d7a0a577209c8d16c6e3
error: Could not read 9f19a4bfe0d0f0d203113d34d7455ceb82ff8341
error: Could not read 9f19a4bfe0d0f0d203113d34d7455ceb82ff8341
Checking files out...
error: git-checkout-index: unable to read sha1 file of .gitignore (27fd37621255799602d74e94d670ff7a1658d40a)
error: git-checkout-index: unable to read sha1 file of COPYING (ca442d313d86dc67e0a2e5d584b465bd382cbf5c)
error: git-checkout-index: unable to read sha1 file of CREDITS (787564bc248b1d6125fc42f3932966b60aa2f7f4)
error: git-checkout-index: unable to read sha1 file of Kbuild (2d4f95e4b89f7f81da6cb94b07e8449b3689ba37)
error: git-checkout-index: unable to read sha1 file of Makefile (fc8e08c419f09e81252f3aa41cb7f47524c0af60)
error: git-checkout-index: unable to read sha1 file of README (05e055530bbb687599dd732d6753c77ffa281ae5)
   < .....>

-- 
dwmw2

Junio C Hamano· Apr 23, 2006, 17:50 UTC · re: David Woodhouse · lore

Re: git-clone --reference problem?

David Woodhouse <dwmw2@infradead.org> writes:

> Should I expect cloning with alternates using '--reference' to be
> transitive?

Not with the current code and design, but a patch that cleanly does that can be considered for inclusion. Adjusting relative paths obtained from other repositories correctly, and avoiding circular alternates by mistake would be tricky, though.

Linus Torvalds· Apr 23, 2006, 18:34 UTC · re: David Woodhouse · lore

Re: git-clone --reference problem?

On Sun, 23 Apr 2006, David Woodhouse wrote:

>
> Should I expect cloning with alternates using '--reference' to be
> transitive?

Well, certainly not now.

> Using '--reference clone1 --reference linux-2.6' for the second clone
> works OK, but surely it ought to work with just '--reference clone1'?

Actually, I don't think using "--reference clone1 --reference linux-2.6" works OK at all - in the sense that it doesn't do what you _think_ it does.

It doesn't use two reference repos at all: it uses just one, which is the last one (ie linux-2.6).

Maybe something like this could work.

Untested, of course. It probably isn't even syntactically correct shell. Whatever. You get the idea.

(That while-loop could probably be simplified to just a

  cat ""$reference/objects/info/alternates" >> "$GIT_DIR/objects/info/alternates"

because all alternates _should_ already be absolute paths, but hey, whatever. I also forget whether we decided that non-absolute paths were relative to the $reference directory, or to the $reference/objects/ directory. The while-loop should be fixed to match whatever that decision was (or just not accept anything but absolute paths: make it use a

	grep '^/' ..

instead of "cat"?)

		Linus

---

diff --git a/git-clone.sh b/git-clone.sh
index 0805168..df4c135 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -221,6 +221,13 @@ then
 		fi
 		reference=$(cd "$reference" && pwd)
 		echo "$reference/objects" >"$GIT_DIR/objects/info/alternates"
+		if test -s "$reference/objects/info/alternates"
+		then
+			while read alt
+			do
+				echo $(cd "$reference" && cd "$alt" && pwd) >> "$GIT_DIR/objects/info/alternates"
+			done < "$reference/objects/info/alternates"
+		fi
 		(cd "$reference" && tar cf - refs) |
 		(cd "$GIT_DIR/refs" &&
 		 mkdir reference-tmp &&

Linus Torvalds· Apr 23, 2006, 18:40 UTC · re: Linus Torvalds · lore

Re: git-clone --reference problem?

On Sun, 23 Apr 2006, Linus Torvalds wrote:

>
> I also forget whether we decided that non-absolute paths were 
> relative to the $reference directory, or to the $reference/objects/ 
> directory.

Just checked. It should be relative to $reference/objects, and we should strip out lines that start with '#', since we allow comments.

Blah. That whole shell-thing is cerainly convenient for prototyping, but it would be wonderful if we had some enterprising young soul that rewrote these things in C and made them built-ins. It may be wasteful to use C for this, but it would make portability easier, and we have all those nice routines for validating and building alternate db lists already that do it right.

Oh, well.

		Linus

Junio C Hamano· Apr 24, 2006, 08:12 UTC · re: Linus Torvalds · lore

Re: git-clone --reference problem?

Linus Torvalds <torvalds@osdl.org> writes:

Show 11 quoted lines

> Maybe something like this could work.
>
> Untested, of course. It probably isn't even syntactically correct shell. 
> Whatever. You get the idea.
>
> (That while-loop could probably be simplified to just a 
>
>   cat ""$reference/objects/info/alternates" >> "$GIT_DIR/objects/info/alternates"
>
> because all alternates _should_ already be absolute paths, but hey, 
> whatever.

I was hoping that the solution would be to do the recursive alternate resolution on the runtime side (IOW, prepare_alt_odb() in sha1_file.c). Then the repository you borrow from _could_ change its own alternates after you set up your repository to borrow from it.

Currently, if the repository you borrowed from suddenly starts borrowing from somewhere else and pruned itself by running "repack -a -d -l", the borrower would be in trouble, even with your patch.