Re: [PATCH] Teach git submodule update to use distributed repositories

On Thu, Jul 17, 2008 at 04:07:11PM +0100, Nigel Magnay wrote:

Oh! So, only after replying to most of your mail, I have realized what are you talking about all the time - _just_ this particular failure mode:

	"Someone pushed out a repository repointing submodules to
	invalid commits, and instead of waiting for the person to fix
	this breakage, we want to do a one-off fetch of all submodules
	from a different repository."

There's nothing else you're trying to solve by this, right?

Now, I think that this is a completely wrong problem to solve. Your gitweb is going to be broken, everyone has to jump through hoops because of this, and that all just because of a single mistake. It shouldn't have _happenned_ in the first place.

So the proper solution for this should be to make an update hook that will simply not _let_ you push out a tree that's broken like this. Something like this (completely untested):

die() { echo "$@"; exit 1; }
git rev-list ^$2 $3 | while read commit; do
	git show $commit:.gitmodules >/tmp/gm$$
	git config -f /tmp/gm$$ --get-regexp 'submodule\..*\.path' |
		cut -d ' ' -f 1 |
		sed 's/^.*\.//; s/\..*$//;' |
		while read submodule; do
			path=$(git config -f /tmp/gm$$ "submodule.$submodule.path")
			url=$(git config -f /tmp/gm$$ "submodule.$submodule.url")
			entry=$(git ls-tree $commit "$path")
			[ -n "$entry" ] || die "submodule $submodule points at a non-existing path"
			[ "$(echo "$entry" | cut -d ' ' -f 1)" = "160000" ] || die "submodule $submodule does not point to a gitlink entry"
			
			subcommit="$(echo "$entry" | cut -d ' ' -f 2)"
			urlhash="$(echo "$url" | sha1sum | cut -d ' ' -f 1)"
			# We keep local copies of submodule repositories
			# for commit existence checking
			echo "Please wait, updating $url cache..."
			if [ -d /tmp/ucache/$urlhash ]; then
			        (cd /tmp/ucache/$urlhash && git fetch)
			else
			        git clone --bare "$url" /tmp/ucache/$urlhash
			fi
			[ "$(git --git-dir=/tmp/ucache/$urlhash cat-file -t "$subcommit" 2>/dev/null)" = "commit" ] || die "submodule $submodule does not point at an existing commit"
		done
	done

Comments? If it seems good, it might be worth including in contrib/hooks/. Maybe even in the default update hook, controlled by a config option.

All the troubles here stem from the fact that normally, Git will not let you push any invalid state to the server. This is not completely true in this case, but we should prevent this behaviour instead of inventing hacks to work it around.

This is ultra-evil. I think assuming things like this is extremely dirty and not reasonable for a universal code, _unless_ we explicitly decide that this is a new convention you want to introduce as a recommendation. But you should've been very clear about this upfront.

_If_ you still insist on the one-off fetches for some reason, I think it's reasonable to provide your own simple script for your users that will autogenerate these URLs appropriately for your particular setup. I don't think there is any real need for a more generic solution.

Are you following the thread about submodule support for git mv, git rm?