Re: [PATCH 2/2] submodule: munge paths to submodule git directories

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Aug 16, 2018, 02:34 UTC
Message-ID: <20180816023446.GA127655@aiede.svl.corp.google.com>
In-Reply-To: <CAGZ79kYfoK9hfXM2-VMAZLPpqBOFQYKtyYuYJb8twzz6Oz5ymQ@mail.gmail.com>

Hi again,

Stefan Beller wrote:

> On Tue, Aug 14, 2018 at 2:12 PM Jonathan Nieder <jrnieder@gmail.com> wrote:

Show 5 quoted lines

>> What if we forbid directory separator characters in the gitdirname?
>
> Fine with me, but ideally we'd want to allow sharding the
> submodules. When you have 1000 submodules
> we'd want them not all inside the toplevel "modules/" ?

That's a good reason to permit slashes in the gitdirname.

If I understood the rest of your reply correctly, your worry was about dangerous gitdirname values in .gitmodules. I never had any wish to read them from there anyway, so this worry hopefully goes away.

[...]

Show 6 quoted lines

>> In this proposal, it would only be read from config, not from
>> .gitmodules.
>
> Ah good point. That makes sense.
>
> Stepping back a bit regarding the config:

[...]

Show 12 quoted lines

> Now that we have the submodule.active or even
> submodule.<name>.active flags, we do not need (b) any more.
> So the URL turns into a useless piece of cruft that just is unneeded
> and might confuse the user.
>
> So maybe I'd want to propose a patch that removes
> submodule.<name>.url from the config once it is cloned.
> (I just read up on "submodule sync" again, but that might not
> even need special care for this new world)
>
> And with all that said, I think if we can avoid having the submodules
> gitdir in the config, the config would look much cleaner, too.

Yes, I understand and agree with this.

I should further spell out my motivation with this gitdirname suggestion. The issue that some people have mentioned in this thread is that urlencoding might not be perfect --- it's pretty close to perfect, but it's likely we'll come up with some unanticipated needs later (like sharding) that it doesn't solve. Solving those all right now would not necessarily be wise, since the thing about unanticipated needs is that you never know in advance what they will be. ;-)

So it would be nice, for future-proofing, if we can change the naming scheme later.

As a bonus, that would also make interoperability with other implementations easier. For example, suppose we mess up in JGit and urlencode a different set of characters than Git does. Then a mixed Git + JGit installation would have this subtle bug of the submodule .git directory not being reused when I switch to and from and branch not containing that submodule, in some circumstances. That sounds difficult to support.

Whereas if we have a gitdirname configuration variable, then JGit and libgit2 and go-git do not have to match the naming scheme Git chooses. They can try, but if one gets it subtly wrong then that is okay because the submodule's directory name is right there and easy to look up.

All at the cost of recording a little configuration somewhere. If we want to decrease the configuration, we can avoid recording it there in the easy cases (e.g. when name == gitdirname). That's "just" an optimization.

And then we have the ability later to handle all the edge cases we haven't handled yet today:

- sharding when the number of submodules is too large
- case-insensitive filesystems
- path name length limits
- different sets of filesystem-special characters

Sane?

Thanks, Jonathan