Re: git submodule: 3 modules same git repo?

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Aug 14, 2018, 23:10 UTC
Message-ID: <20180814231049.GH142615@aiede.svl.corp.google.com>
In-Reply-To: <95e4f9df528a40bf3f3e648318904500343abf9a.camel@infinera.com>

Hi,

Joakim Tjernlund wrote:

Show 5 quoted lines

> I am trying to create 3 submodules from the same git repo, each pointing to a different branch.
> Since the repo is somewhat large, I don't want the 3 submodules to clone the same repo 3
> times, I want one clone and then have the 3 submodules to point to different commits.
>
> Is this possible? If not, could it be added?

It's an interesting case. You're not the only one that has wanted something like this: for example, "repo" (a similar tool with some differences) included the change [1] (repo: Support multiple branches for the same project, 2014-01-30) for this kind of application.

In practice, it's a bit messy. To allow switching to a branch where the submodule is not present, we store the submodule .git directory in the superproject's .git/modules/<submodule name> directory. This is an ordinary .git directory, with files like

	HEAD
	config
	packed-refs
	[etc]

The "git worktree" tool allows having multiple worktrees for a single Git repository. Each worktree has its own HEAD. So the layout would look something like

	.git/modules/<submodule name>/
		config
		packed-refs
		[etc]
		worktrees/
			<branch1>/
				HEAD
			<branch2>/
				HEAD

By piggy-backing on the "git worktree" feature, this should work great, but it will take some work to teach Git to set it up.

A side note: as Stefan (cc-ed) mentioned, there is another, related interaction between submodules and "git worktree". Namely: I might want to have multiple worktrees for a single superproject Git repository. In that case, the layout might look something like

	.git/
		HEAD
		config
		packed-refs
		[etc]
		modules/
			<submodule name>/
				HEAD
				config
				packed-refs
				...
		worktrees/
			<branch1>/
				HEAD
			<branch2>/
				HEAD

The patch that Stefan sent heads in this direction, but it has a problem: if the submodule is checked out in both worktrees, then they cannot have the same HEAD. So to support that goal well would also require supporting the goal you've described as a side effect: each submodule would need to have multiple worktrees, at least one per worktree of the superproject.

Sorry, that got a bit contorted after a while. If you'd be interested in pursuing this, I'd be happy to review any thoughts you have (for example if you write a brief design doc).

My experience from interacting with the feature [1] is that it is easy to make mistakes when trying to support this kind of case. (repo had some bugs due to, for example, multiple checkouts of a repository trying to run "git prune" at the same time.) The "git worktree" feature should be a good foundation to build on that avoids some of the problems encountered there.

Thanks, Jonathan

[1] https://gerrit-review.googlesource.com/c/git-repo/+/50715/