Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup

From: Stefan Beller <sbeller@google.com>
Date: Jul 27, 2016, 16:53 UTC
Message-ID: <CAGZ79kZtfpmZRyOfF6NMhCqjNBgfH1bX+xWUCEdedQGbCbL71Q@mail.gmail.com>
In-Reply-To: <5798C744.5080308@gmail.com>

Jakub wrote:

> I think the problem with `--reference` is that it does not
> setup backreferences to prevent gc removing borrowed objects;
> which is a hard problem to solve, except for limited cases...
> like git-worktree.

Right. And instead of solving the reference problem, I'd rather solve the worktree problem as I think it yields more?

Show 20 quoted lines

>
>> So I think the current workflow for submodules
>> may need some redesign anyway as the submodule
>> commands were designed with a strict "one working
>> tree only" assumption.
>>
>> Submodule URLs  are stored in 3 places:
>>  A) In the .gitmodules file of the superproject
>>  B) In the option submodule.<name>.URL in the superproject
>>  C) In the remote.origin.URL in the submodule
>>
>> A) is a recommendation from the superproject to make life
>> of downstream easier to find and setup the whole thing.
>> You can ignore that if you want, though generally a caring
>> upstream provides good URLs here.
>
> Also, this URL might have change if the repository moves
> to other server; even when checking out ancient version
> we usually want to use current URL, not the one in currently
> checked-out .gitmodules file.

Right.

Show 8 quoted lines

>
>> C) is where we actually fetch from (and hope it has all
>> the sha1s that are recorded as gitlinks in the superproject)
>
> Is it? Or is it only the case if you do `git fetch` or
> equivalent from within inside of submodule? You can fetch
> updates using `git submodule ...` from supermodule, isn't it?
> But I might be wrong here.

If you call `submodule update` in the superproject it actually just does a `(cd $submodule && git fetch)`.

And in the submodule we have a .git file pointing to the superprojects ".git/modules/<name>/" which is a full blown git dir, i.e. it has its own config, HEAD etc.

>
> Also: if .git file is gitfile link, do submodule even has
> it's own configuration file?

Yes they do.

Show 11 quoted lines

>
>>
>> B) seems like a hack to enable the workflow as below:
>
> It has overloaded meaning, being used both for current URL
> of submodule as seen in supermodule, AND that submodule
> is checked out / needs to be checked out in the worktree
> of a supermodule.  There might be the case when you check
> out (in given worktree) a version of a supermodule that
> do not include submodule at all, but you want to know that
> when going back, this submodule is to be checked out (or not).

I am currently working on solving that with a patch series, that allows 2 settings. The URL will be used only to overwrite the URL from the .gitmodules file and another setting will be used to determine if we want to checkout the submodule.

Show 5 quoted lines

>
> The second information needs to be per-worktree. How to
> solve it, be it per-worktree configuration (not shared),
> or a special configuration variable, or worktree having
> unshared copy of configuration -- this what is discussed.

Show 7 quoted lines

>
>> Current workflow for handling submodule URLs:
>>  1) Clone the superproject
>>  2) Run git submodule init on desired submodules
>
> Or 1-2) clone the superproject recursively, with all its
> submodules.

Only if the URLs are setup properly.

>
>>  3) Inspect .git/config to see if any submodule URL needs adaption
>
> Which is usually not needed.

Yeah, I should have added the assertion that the .gitmodules may be out of date or such for this workflow to make sense. Usually just go with recursive clone.

Show 11 quoted lines

>>
>> This long lived stuff probably doesn't make sense for the a single
>> repository, but in combination with submodules (which is another way
>> to approach the "sparse/narrow" desire of a large project), I think
>> that makes sense, because the "continuous integration" shares a lot
>> of submodules with my "regular everyday hacking" or the "I need to
>> test my colleague work now" worktree.
>
> One thing that git-worktree would be very useful, if it could work
> with submodules: you could use separate worktrees to easily test
> if the supermodule works with and without its submodules present.

Oh! Yeah that makes sense!

Show 10 quoted lines

>
> [...]
>> If you switch a branch (or to any sha1), the submodule currently stays
>> "as-is" and may be updated using "submodule update", which goes through
>> the list of existing (checked out) submodules and checks them out to the
>> sha1 pointed to by the superprojects gitlink.
>
> Which might be simply a problem that submodule UI is not mature enough.
> I would like to see automatic switch of submodule contents, if
> configured so.

Me too. Once upon a time Jens pushed for that with a series found at: https://github.com/jlehmann/git-submod-enhancements/tree/git-checkout-recurse-submodules