threads / discuss / 48728

Adding nested repository with slash adds files instead of gitlink

Subject: Adding nested repository with slash adds files instead of gitlink

## tl;dr

22 messages between Jun 18, 2018 and Aug 14, 2024.

replies: 21people: 8as markdown or json

Heiko Voigt· Jun 18, 2018, 11:19 UTC · lore
Hi,

I just discovered that when you have a slash at the end of a nested repository, the files contained in the repository get added instead of the gitlink.

I found this when I was adding a submodule and wanted to commit a small change before that. You get the slash by using tab autocompletion.

Here is a recipe to reproduce:

mkdir test cd test; git init touch a; git add a; git commit -m a mkdir ../test.git; (cd ../test.git; git init --bare) git remote add origin ../test.git git push origin master git submodule add ../test.git submodule git reset git add submodule/

Now instead of just submodule gitlink there is an entry for submodule/a in the index.

I just thought I put this out there. Will have a look if I find the time to cook up a proper testcase and investigate.

Cheers Heiko
Duy Nguyen· Jun 18, 2018, 15:06 UTC · re: Heiko Voigt · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Mon, Jun 18, 2018 at 1:23 PM Heiko Voigt <hvoigt@hvoigt.net> wrote:
Show 27 quoted lines
>
> Hi,
>
> I just discovered that when you have a slash at the end of a nested
> repository, the files contained in the repository get added instead of
> the gitlink.
>
> I found this when I was adding a submodule and wanted to commit a small
> change before that. You get the slash by using tab autocompletion.
>
> Here is a recipe to reproduce:
>
> mkdir test
> cd test; git init
> touch a; git add a; git commit -m a
> mkdir ../test.git; (cd ../test.git; git init --bare)
> git remote add origin ../test.git
> git push origin master
> git submodule add ../test.git submodule
> git reset
> git add submodule/
>
> Now instead of just submodule gitlink there is an entry for submodule/a
> in the index.
>
> I just thought I put this out there. Will have a look if I find the time
> to cook up a proper testcase and investigate.

This sounds like the submodule specific code in pathspec.c, which has been replaced with something else in bw/pathspec-sans-the-index. If you have time, try a version without those changes (e.g. v2.13 or before) to see if it's a possible culprit.

> Cheers Heiko
-- 
Duy
Brandon Williams· Jun 18, 2018, 18:12 UTC · re: Duy Nguyen · lore

Re: Adding nested repository with slash adds files instead of gitlink

On 06/18, Duy Nguyen wrote:
Show 33 quoted lines
> On Mon, Jun 18, 2018 at 1:23 PM Heiko Voigt <hvoigt@hvoigt.net> wrote:
> >
> > Hi,
> >
> > I just discovered that when you have a slash at the end of a nested
> > repository, the files contained in the repository get added instead of
> > the gitlink.
> >
> > I found this when I was adding a submodule and wanted to commit a small
> > change before that. You get the slash by using tab autocompletion.
> >
> > Here is a recipe to reproduce:
> >
> > mkdir test
> > cd test; git init
> > touch a; git add a; git commit -m a
> > mkdir ../test.git; (cd ../test.git; git init --bare)
> > git remote add origin ../test.git
> > git push origin master
> > git submodule add ../test.git submodule
> > git reset
> > git add submodule/
> >
> > Now instead of just submodule gitlink there is an entry for submodule/a
> > in the index.
> >
> > I just thought I put this out there. Will have a look if I find the time
> > to cook up a proper testcase and investigate.
> 
> This sounds like the submodule specific code in pathspec.c, which has
> been replaced with something else in bw/pathspec-sans-the-index. If
> you have time, try a version without those changes (e.g. v2.13 or
> before) to see if it's a possible culprit.

I just tested this with v2.13 and saw the same issue. I don't actually think this ever worked in the way you want it to Heiko. Maybe git add needs to be taught to be more intelligent when trying to add a submodule which doesn't exist in the index.

-- 
Brandon Williams
Heiko Voigt· Jun 19, 2018, 10:36 UTC · re: Brandon Williams · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Mon, Jun 18, 2018 at 11:12:15AM -0700, Brandon Williams wrote:
Show 10 quoted lines
> On 06/18, Duy Nguyen wrote:
> > This sounds like the submodule specific code in pathspec.c, which has
> > been replaced with something else in bw/pathspec-sans-the-index. If
> > you have time, try a version without those changes (e.g. v2.13 or
> > before) to see if it's a possible culprit.
> 
> I just tested this with v2.13 and saw the same issue.  I don't actually
> think this ever worked in the way you want it to Heiko.  Maybe git add
> needs to be taught to be more intelligent when trying to add a submodule
> which doesn't exist in the index.

That was also my guess, since my feeling is that this is a quite rare use case. Adding submodules alone is not a daily thing, let alone selecting different changes after 'git submodule add'.

I also think git could be more intelligent here.
Cheers Heiko
Duy Nguyen· Jun 19, 2018, 15:16 UTC · re: Heiko Voigt · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Tue, Jun 19, 2018 at 12:36 PM Heiko Voigt <hvoigt@hvoigt.net> wrote:
Show 18 quoted lines
>
> On Mon, Jun 18, 2018 at 11:12:15AM -0700, Brandon Williams wrote:
> > On 06/18, Duy Nguyen wrote:
> > > This sounds like the submodule specific code in pathspec.c, which has
> > > been replaced with something else in bw/pathspec-sans-the-index. If
> > > you have time, try a version without those changes (e.g. v2.13 or
> > > before) to see if it's a possible culprit.
> >
> > I just tested this with v2.13 and saw the same issue.  I don't actually
> > think this ever worked in the way you want it to Heiko.  Maybe git add
> > needs to be taught to be more intelligent when trying to add a submodule
> > which doesn't exist in the index.
>
> That was also my guess, since my feeling is that this is a quite rare
> use case. Adding submodules alone is not a daily thing, let alone
> selecting different changes after 'git submodule add'.
>
> I also think git could be more intelligent here.

Ah.. the "submodule not registered in index" case. I think I remember this (because I remember complaining about it once or two times). Definitely agreed that git-add should do the right thing here.

Brandon already moved the submodule check outside pathspec code (wonderful!) so adding more checks based on worktree state should not be a big work. I think the only concern here is catching submodule locations so we don't check submodule at the same location multiple times.

No actually, we could do better. Let me see if I can come up with a patch or something...

-- 
Duy
Junio C Hamano· Jun 19, 2018, 15:56 UTC · re: Duy Nguyen · lore

Re: Adding nested repository with slash adds files instead of gitlink

Duy Nguyen <pclouds@gmail.com> writes:
Show 23 quoted lines
> On Tue, Jun 19, 2018 at 12:36 PM Heiko Voigt <hvoigt@hvoigt.net> wrote:
>>
>> On Mon, Jun 18, 2018 at 11:12:15AM -0700, Brandon Williams wrote:
>> > On 06/18, Duy Nguyen wrote:
>> > > This sounds like the submodule specific code in pathspec.c, which has
>> > > been replaced with something else in bw/pathspec-sans-the-index. If
>> > > you have time, try a version without those changes (e.g. v2.13 or
>> > > before) to see if it's a possible culprit.
>> >
>> > I just tested this with v2.13 and saw the same issue.  I don't actually
>> > think this ever worked in the way you want it to Heiko.  Maybe git add
>> > needs to be taught to be more intelligent when trying to add a submodule
>> > which doesn't exist in the index.
>>
>> That was also my guess, since my feeling is that this is a quite rare
>> use case. Adding submodules alone is not a daily thing, let alone
>> selecting different changes after 'git submodule add'.
>>
>> I also think git could be more intelligent here.
>
> Ah.. the "submodule not registered in index" case. I think I remember
> this (because I remember complaining about it once or two times).
> Definitely agreed that git-add should do the right thing here.

I am not sure if this even needs to be implemented as "look for the submodule in the index". Even before submodule was added, we knew that "git add foo/bar" should reject the request if we find foo is a symbolic link, and we should do the same when foo/ is a directory that is the top of a working tree under control of another repository, no?

Hmm, what happens when we do this?
	git init
	ln -s /tmp foo
	>foo/bar
	git add foo/

I think we should say either "let's add foo symlink" or "foo/. (directory) is beyond symlink" (the latter is preferrable, but the former is acceptable as long as foo is pointing at a directory; but foo could be a dangling symlink whose pointee's type may not be discernable by "git add").

Shouldn't we be reacting pretty much the same when we see this?
	git init
	git init foo
	>foo/bar
	git add foo/

That is, either drop '/' and add 'foo' as a submodule, or say "foo/. (directory) belongs to another repository, cannot add here" (again, the latter is preferrable for consistency with the symlink behaviour above).

Duy Nguyen· Jun 19, 2018, 16:11 UTC · re: Junio C Hamano · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Tue, Jun 19, 2018 at 5:56 PM Junio C Hamano <gitster@pobox.com> wrote:
Show 33 quoted lines
>
> Duy Nguyen <pclouds@gmail.com> writes:
>
> > On Tue, Jun 19, 2018 at 12:36 PM Heiko Voigt <hvoigt@hvoigt.net> wrote:
> >>
> >> On Mon, Jun 18, 2018 at 11:12:15AM -0700, Brandon Williams wrote:
> >> > On 06/18, Duy Nguyen wrote:
> >> > > This sounds like the submodule specific code in pathspec.c, which has
> >> > > been replaced with something else in bw/pathspec-sans-the-index. If
> >> > > you have time, try a version without those changes (e.g. v2.13 or
> >> > > before) to see if it's a possible culprit.
> >> >
> >> > I just tested this with v2.13 and saw the same issue.  I don't actually
> >> > think this ever worked in the way you want it to Heiko.  Maybe git add
> >> > needs to be taught to be more intelligent when trying to add a submodule
> >> > which doesn't exist in the index.
> >>
> >> That was also my guess, since my feeling is that this is a quite rare
> >> use case. Adding submodules alone is not a daily thing, let alone
> >> selecting different changes after 'git submodule add'.
> >>
> >> I also think git could be more intelligent here.
> >
> > Ah.. the "submodule not registered in index" case. I think I remember
> > this (because I remember complaining about it once or two times).
> > Definitely agreed that git-add should do the right thing here.
>
> I am not sure if this even needs to be implemented as "look for the
> submodule in the index".  Even before submodule was added, we knew
> that "git add foo/bar" should reject the request if we find foo is a
> symbolic link, and we should do the same when foo/ is a directory
> that is the top of a working tree under control of another
> repository, no?

Exactly. I started with the intention to do something related to the index only to slowly realize that it was not the right place. We traverse directories and stop looking inside a symlink, we can do the same if we realize it's a submodule.

-- 
Duy
Duy Nguyen· Jun 19, 2018, 16:09 UTC · re: Duy Nguyen · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Tue, Jun 19, 2018 at 05:16:17PM +0200, Duy Nguyen wrote:
> No actually, we could do better. Let me see if I can come up with a
> patch or something...

OK. What we currently do is, when we search for potential untracked paths for adding to the index, we unconditionally ignore anything inside ".git". For example, if "foo" is a submodule, "git add ." will visit "foo/.git" then ignore its content completely.

We could do something very similar: when we visit "foo", if "foo/.git" exists, we ignore it as well. In other words, we extend from "ignore anything inside a git repository" to "ignore anything inside any other git worktree".

The following patch basically does that. If you specify "git add foo/bar". It will still visit "foo" first, realize that it's a submodule and drop it. At the end, it will not report foo/bar as an untracked (i.e. add-able) entry, so you can't add it.

I didn't test it extensively to see if it breaks anything though. And I might need to check how it affects untracked cache...

-- 8< --
diff --git a/dir.c b/dir.c
index fe9bf58e4c..8a1a5d8dd5 100644
--- a/dir.c
+++ b/dir.c
@@ -1672,6 +1672,17 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 	if (dtype != DT_DIR && has_path_in_index)
 		return path_none;
 
+	if (dtype == DT_DIR) {
+		int path_len = path->len;
+		int is_submodule;
+
+		strbuf_addstr(path, "/.git");
+		is_submodule = is_directory(path->buf);
+		strbuf_setlen(path, path_len);
+		if (is_submodule)
+			return path_none;
+	}
+
 	/*
 	 * When we are looking at a directory P in the working tree,
 	 * there are three cases:
-- 8< --
Duy Nguyen· Jun 19, 2018, 16:20 UTC · re: Duy Nguyen · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Tue, Jun 19, 2018 at 6:09 PM Duy Nguyen <pclouds@gmail.com> wrote:
Show 18 quoted lines
> On Tue, Jun 19, 2018 at 05:16:17PM +0200, Duy Nguyen wrote:
> > No actually, we could do better. Let me see if I can come up with a
> > patch or something...
>
> OK. What we currently do is, when we search for potential untracked
> paths for adding to the index, we unconditionally ignore anything
> inside ".git". For example, if "foo" is a submodule, "git add ." will
> visit "foo/.git" then ignore its content completely.
>
> We could do something very similar: when we visit "foo", if "foo/.git"
> exists, we ignore it as well. In other words, we extend from "ignore
> anything inside a git repository" to "ignore anything inside any other
> git worktree".
>
> The following patch basically does that. If you specify "git add
> foo/bar". It will still visit "foo" first, realize that it's a
> submodule and drop it. At the end, it will not report foo/bar as an
> untracked (i.e. add-able) entry, so you can't add it.

Another note (which I added, then thought otherwise and dropped). I believe this approach also solves the problem that die_path_inside_submodule() tries to work around.

When you feed a path inside a submodule, read_directory() code does not realize it and walk through like it's part of the current worktree (wrong!). But if read_directory() does the right thing from the beginning, you don't need this trick. We don't even need this trick if a submodule is not real on worktree (no ".git" directory there) but registered in the index as a git link because the d/f check should catch that and complain loudly anyway when you add a new entry.

-- 
Duy
Kevin Daudt· Jun 18, 2018, 15:55 UTC · re: Heiko Voigt · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Mon, Jun 18, 2018 at 01:19:19PM +0200, Heiko Voigt wrote:
Now with cc to the mailing list.
Show 28 quoted lines
> Hi,
> 
> I just discovered that when you have a slash at the end of a nested
> repository, the files contained in the repository get added instead of
> the gitlink.
> 
> I found this when I was adding a submodule and wanted to commit a small
> change before that. You get the slash by using tab autocompletion.
> 
> Here is a recipe to reproduce:
> 
> mkdir test
> cd test; git init
> touch a; git add a; git commit -m a
> mkdir ../test.git; (cd ../test.git; git init --bare)
> git remote add origin ../test.git
> git push origin master
> git submodule add ../test.git submodule
> git reset
> git add submodule/
> 
> Now instead of just submodule gitlink there is an entry for submodule/a
> in the index.
> 
> I just thought I put this out there. Will have a look if I find the time
> to cook up a proper testcase and investigate.
> 
> Cheers Heiko

This has been the case as far as I can remember, and is basically lore in the #git irc channel).

This can also be reproduced by just cloning a repo inside another repo and running `git add path/`.

Heiko Voigt· Jun 19, 2018, 10:27 UTC · re: Kevin Daudt · lore

Re: Adding nested repository with slash adds files instead of gitlink

Hi,
On Mon, Jun 18, 2018 at 05:55:44PM +0200, Kevin Daudt wrote:
> On Mon, Jun 18, 2018 at 01:19:19PM +0200, Heiko Voigt wrote:
> > I just discovered that when you have a slash at the end of a nested
> > repository, the files contained in the repository get added instead of
> > the gitlink.
[...]
Show 11 quoted lines
> > 
> > I just thought I put this out there. Will have a look if I find the time
> > to cook up a proper testcase and investigate.
> > 
> > Cheers Heiko
> 
> This has been the case as far as I can remember, and is basically lore
> in the #git irc channel).
> 
> This can also be reproduced by just cloning a repo inside another repo
> and running `git add path/`.

Interesting and nobody complained to the mailinglist? IMO, there is no reason 'git add path' and 'git add path/' should behave differently or is there?

So it seems it is a very seldom operation (in my daily work it is at least) and people just accepted it as a quirk of git.

If someone wants to look into changing this feel free, otherwise I will have a look, but I am not sure when yet.

Cheers Heiko
Rafael Ascensão· Jun 19, 2018, 22:29 UTC · re: Heiko Voigt · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Tue, Jun 19, 2018 at 11:28 AM Heiko Voigt <hvoigt@hvoigt.net> wrote:
>
> Interesting and nobody complained to the mailinglist?
>
For reference this was sometimes called "Fake Submodules" online.
Kevin Daudt· Jun 20, 2018, 04:39 UTC · re: Rafael Ascensão · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Tue, Jun 19, 2018 at 11:29:02PM +0100, Rafael Ascensão wrote:
Show 6 quoted lines
> On Tue, Jun 19, 2018 at 11:28 AM Heiko Voigt <hvoigt@hvoigt.net> wrote:
> >
> > Interesting and nobody complained to the mailinglist?
> >
> 
> For reference this was sometimes called "Fake Submodules" online.

Do you refer to cloning a repository in another repository, and running `git add path` (no trailing /) so that a gitlink is created without a .gitmodules file? Because that's not what Heiko is referring to.

What this is about that when doing `git add path/` (with trailing /), git will add the files of the cloned repository to the parent repo, and basically ignore that it's another repository.

And I guess no one complained because it looked like a feature rather than a bug.

Kevin
Rafael Ascensão· Jun 20, 2018, 11:52 UTC · re: Kevin Daudt · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Wed, Jun 20, 2018 at 5:39 AM Kevin Daudt <me@ikke.info> wrote:
>
> What this is about that when doing `git add path/` (with trailing /),
>

This is what I was referring to. If you search for 'Fake Submodules', you'll see that some people were/are intentionally using this instead of subtrees or submodules. Unfortunately the original article [1] seems to be dead, but searching url in the mailing list archives leads to some additional discussion on the subject [2,3].

[1]:http://debuggable.com/posts/git-fake-submodules:4b563ee4-f3cc-4061-967e-0e48cbdd56cb [2]:https://public-inbox.org/git/xmqqy47o6q71.fsf@gitster.mtv.corp.google.com/ [3]:https://public-inbox.org/git/CAGZ79kZofg3jS+g0weTdco+PGo_p-_Hd-NScZ=q2UfB7tF2GPA@mail.gmail.com/

Duy Nguyen· Jun 20, 2018, 14:57 UTC · re: Rafael Ascensão · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Wed, Jun 20, 2018 at 1:55 PM Rafael Ascensão <rafa.almas@gmail.com> wrote:
Show 11 quoted lines
>
> On Wed, Jun 20, 2018 at 5:39 AM Kevin Daudt <me@ikke.info> wrote:
> >
> > What this is about that when doing `git add path/` (with trailing /),
> >
>
> This is what I was referring to. If you search for 'Fake Submodules',
> you'll see that some people were/are intentionally using this instead of
> subtrees or submodules. Unfortunately the original article [1] seems to
> be dead, but searching url in the mailing list archives leads to some
> additional discussion on the subject [2,3].

Abusing a long standing bug does not make it a feature. I'm not opposed to having a new option to keep that behavior, but it should not be the default. If you use it that way, you're on your own.

> [1]:http://debuggable.com/posts/git-fake-submodules:4b563ee4-f3cc-4061-967e-0e48cbdd56cb
> [2]:https://public-inbox.org/git/xmqqy47o6q71.fsf@gitster.mtv.corp.google.com/
> [3]:https://public-inbox.org/git/CAGZ79kZofg3jS+g0weTdco+PGo_p-_Hd-NScZ=q2UfB7tF2GPA@mail.gmail.com/
-- 
Duy
Rafael Ascensão· Jun 20, 2018, 16:21 UTC · re: Duy Nguyen · lore

Re: Adding nested repository with slash adds files instead of gitlink

On Tue, Jun 19, 2018 at 11:28 AM Heiko Voigt <hvoigt@hvoigt.net> wrote:
>
> Interesting and nobody complained to the mailinglist?
>
On Wed, Jun 20, 2018 at 3:57 PM Duy Nguyen <pclouds@gmail.com> wrote:
>
> Abusing a long standing bug does not make it a feature.
>

To make things clear, I wasn't defending if this should be considered a bug or a feature. Was just clarifying that this isn't a newly discovered thing and was reported/discussed in the past.

-- Cheers, Rafael

Phil Sainty· Aug 8, 2024, 11:20 UTC · re: Duy Nguyen · lore

Re: Adding nested repository with slash adds files instead of gitlink

Show 13 quoted lines
> On Wed, Jun 20, 2018 at 1:55 PM Rafael Ascensão <rafa.almas@gmail.com> wrote:
> > On Wed, Jun 20, 2018 at 5:39 AM Kevin Daudt <me@ikke.info> wrote:
> > > What this is about that when doing `git add path/` (with trailing /),
> >
> > This is what I was referring to. If you search for 'Fake Submodules',
> > you'll see that some people were/are intentionally using this instead of
> > subtrees or submodules. Unfortunately the original article [1] seems to
> > be dead, but searching url in the mailing list archives leads to some
> > additional discussion on the subject [2,3].
>
> Abusing a long standing bug does not make it a feature. I'm not
> opposed to having a new option to keep that behavior, but it should
> not be the default. If you use it that way, you're on your own.
Was such an option ever worked on?

(I.e. a way to git-add some sub-directory 'foo' which contains another repository, and have git-add act as if foo/.git didn't exist -- simply adding the (other) contents of foo to the containing repo's index.)

I haven't spotted anything in the git-add man page for v2.34, nor in the release notes for subsequent git versions.

The old behaviour was a genuinely useful ability which I've used a great in the past, and if there's any convenient way to achieve the same thing nowadays, I've failed to find it. (One can temporarily move the nested .git directory elsewhere, run git-add, and then move the .git directory back again; but that's frustratingly cumbersome by comparison).

If I haven't missed some existing solution, could a new git-add option for restoring this ability be implemented?

-Phil
> > [1]:http://debuggable.com/posts/git-fake-submodules:4b563ee4-f3cc-4061-967e-0e48cbdd56cb
> > [2]:https://public-inbox.org/git/xmqqy47o6q71.fsf@gitster.mtv.corp.google.com/
> > [3]:https://public-inbox.org/git/CAGZ79kZofg3jS+g0weTdco+PGo_p-_Hd-NScZ=q2UfB7tF2GPA@mail.gmail.com/
Junio C Hamano· Aug 8, 2024, 16:35 UTC · re: Phil Sainty · lore

Re: Adding nested repository with slash adds files instead of gitlink

Phil Sainty <phil@catalyst.net.nz> writes:
Show 15 quoted lines
>> On Wed, Jun 20, 2018 at 1:55 PM Rafael Ascensão <rafa.almas@gmail.com> wrote:
>> > On Wed, Jun 20, 2018 at 5:39 AM Kevin Daudt <me@ikke.info> wrote:
>> > > What this is about that when doing `git add path/` (with trailing /),
>> >
>> > This is what I was referring to. If you search for 'Fake Submodules',
>> > you'll see that some people were/are intentionally using this instead of
>> > subtrees or submodules. Unfortunately the original article [1] seems to
>> > be dead, but searching url in the mailing list archives leads to some
>> > additional discussion on the subject [2,3].
>>
>> Abusing a long standing bug does not make it a feature. I'm not
>> opposed to having a new option to keep that behavior, but it should
>> not be the default. If you use it that way, you're on your own.
>
> Was such an option ever worked on?
No.

I do not recall hearing anybody who have been active in the development community saying anything good about such an option. For the past 6 or so years, nobody who actively works on git thought it was an interesting and/or useful thing to work on.

I cannot quite say that they thought that it is actively a bad idea to offer such an option, though.

Johannes Schindelin· Aug 13, 2024, 12:48 UTC · re: Junio C Hamano · lore

Re: Adding nested repository with slash adds files instead of gitlink

Hi,
On Thu, 8 Aug 2024, Junio C Hamano wrote:
Show 27 quoted lines
> Phil Sainty <phil@catalyst.net.nz> writes:
>
> >> On Wed, Jun 20, 2018 at 1:55 PM Rafael Ascensão <rafa.almas@gmail.com> wrote:
> >> > On Wed, Jun 20, 2018 at 5:39 AM Kevin Daudt <me@ikke.info> wrote:
> >> > > What this is about that when doing `git add path/` (with trailing /),
> >> >
> >> > This is what I was referring to. If you search for 'Fake Submodules',
> >> > you'll see that some people were/are intentionally using this instead of
> >> > subtrees or submodules. Unfortunately the original article [1] seems to
> >> > be dead, but searching url in the mailing list archives leads to some
> >> > additional discussion on the subject [2,3].
> >>
> >> Abusing a long standing bug does not make it a feature. I'm not
> >> opposed to having a new option to keep that behavior, but it should
> >> not be the default. If you use it that way, you're on your own.
> >
> > Was such an option ever worked on?
>
> No.
>
> I do not recall hearing anybody who have been active in the
> development community saying anything good about such an option.
> For the past 6 or so years, nobody who actively works on git thought
> it was an interesting and/or useful thing to work on.
>
> I cannot quite say that they thought that it is actively a bad idea
> to offer such an option, though.

I have encountered the wish quite frequently to be able to make changes in a subdirectory and have them reflected as a commit that is both part of that subdirectory's revision history as well as part of the containing project's.

The benefits for monorepos, and for reproducible builds, are probably obvious.

Sadly, I cannot think of an elegant technical design, and this is not for lack of trying.

There are lots of projects I worked on that would have benefitted from being able to track a subdirectory both independently as well as within the context of a containing project, i.e. offering to view (and fetch/push) the changesets in both contexts as equal first citizens.

Even the Git and libxdiff projects, as a concrete example, could potentially have benefitted from such a feature: Ideally, it should be possible to push commits made in the `xdiff/` directory not only to the git/git repository but also separately from the rest of Git, say, to xdiff/libxdiff. This way, the `libxdiff` project would still be able today to thrive as an independent project.

The only way to implement a feature like this that I _can_ think of is to generate duplicate commit objects, though, with a reference to the "inner" commit stored in the "outer" commit object. And that strategy breaks down really quickly, no matter whether I think about deep dependency trees or about integrating commit histories that have been made in the "inner" project separately and now need to be merged into the "outer" project, and there are many more processes for which this strategy strikes me as inadequate.

So yes, I think that the idea per se has a lot of merit in the real world out there. It's the implementation details that are an obstacle.

Ciao, Johannes

Junio C Hamano· Aug 13, 2024, 17:18 UTC · re: Johannes Schindelin · lore

Re: Adding nested repository with slash adds files instead of gitlink

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 20 quoted lines
>> >> Abusing a long standing bug does not make it a feature. I'm not
>> >> opposed to having a new option to keep that behavior, but it should
>> >> not be the default. If you use it that way, you're on your own.
>> >
>> > Was such an option ever worked on?
>>
>> No.
>>
>> I do not recall hearing anybody who have been active in the
>> development community saying anything good about such an option.
>> For the past 6 or so years, nobody who actively works on git thought
>> it was an interesting and/or useful thing to work on.
>>
>> I cannot quite say that they thought that it is actively a bad idea
>> to offer such an option, though.
>
> I have encountered the wish quite frequently to be able to make changes in
> a subdirectory and have them reflected as a commit that is both part of
> that subdirectory's revision history as well as part of the containing
> project's.

Let me make sure I understand the above. You create a commit to contain the change in the submodule and at the same time create a new commit to bind the updated submodule commit to superproject tree. If that is what you are talking about, of course, that would be a nice thing to have. Sort of "commit --recursive". It is still debatable what to do with the log messages, though (you want to leave in the submodule an explanation that is understandable in the scope of the submodule, while you want to record in the superproject a bigger picture reasoning).

But I did not get the impression that it is what the original poster wants. My reading of the original thread (this is a resurrection of an antient thread dating back to 2018) was that you have a submodule at path S, you muck with a file in S/file, and you want to commit in the context of the superproject, having the superproject track S/file in its history (not just S gitlink).

And that is quite different from what you said.
Phil Sainty· Aug 13, 2024, 23:13 UTC · re: Junio C Hamano · lore

Re: Adding nested repository with slash adds files instead of gitlink

Junio C Hamano <gitster@pobox.com> writes:
> Let me make sure I understand the above.  You create a commit to
> contain the change in the submodule and at the same time create a
> new commit to bind the updated submodule commit to superproject tree.

I can imagine this ability also being useful, but it would be an independent feature from the one initially requested here...

Show 6 quoted lines
> But I did not get the impression that it is what the original poster
> wants.  My reading of the original thread (this is a resurrection of
> an antient thread dating back to 2018) was that you have a submodule
> at path S, you muck with a file in S/file, and you want to commit in
> the context of the superproject, having the superproject track S/file
> in its history (not just S gitlink).
That's correct.

My common usage was that I would add the entire contents of S, along with some associated configuration outside of S, and then make a commit of all of that in the superproject.

The two repos (superproject and submodule S) are then tracking the files in S independently; so if I was to pull new changes to the submodule from its own upstream, git commands run from the S directory would not show any changes vs the state of the submodule repo, whereas commands run from the superproject would see new changes.

Cloning the superproject repo would produce its version of S, and without the S/.git directory.

-Phil
Johannes Schindelin· Aug 14, 2024, 12:00 UTC · re: Phil Sainty · lore

Re: Adding nested repository with slash adds files instead of gitlink

Hi Phil & Junio,
On Wed, 14 Aug 2024, Phil Sainty wrote:
Show 7 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> > Let me make sure I understand the above.  You create a commit to
> > contain the change in the submodule and at the same time create a
> > new commit to bind the updated submodule commit to superproject tree.
>
> I can imagine this ability also being useful, but it would be an
> independent feature from the one initially requested here...
FWIW I did not intend to suggest anything to do with submodules.

There are too many issues with submodules and I don't think they will ever be resolved, as they simply don't integrate into the original, elegant design of having all Git objects in the same repository.

What I had in mind (and which I described only very loosely) was to create a super-project commit that references the modified subdirectory as part of the same project, much like `xdiff/` files are tracked inside git/git.

Yes, I papered over the need to create at least one new tree object that is derived from the super-project's root tree object and references the root tree object of the "inner" commit. I did not want to go into too many details about the technical implementation, and chose to use the `xdiff/` example as a way to transport my idea, but apparently I was wrong to believe that it would do a good enough job.

In essence, what I would love to have is more along the lines of how Subversion allows you to check out just a subdirectory of the project, and restrict the revision history to that directory in that checkout.

This is different from submodules, and also from subtrees, as it would allow to treat `xdiff/` as a stand-alone repository, but also allow to treat it as a tracked subdirectory of the super-project at the same time, both equally-well integrated into the standard Git processes like `git log -G`, `git bisect`, `git rebase`, etc.

Show 12 quoted lines
> > But I did not get the impression that it is what the original poster
> > wants.  My reading of the original thread (this is a resurrection of
> > an antient thread dating back to 2018) was that you have a submodule
> > at path S, you muck with a file in S/file, and you want to commit in
> > the context of the superproject, having the superproject track S/file
> > in its history (not just S gitlink).
>
> That's correct.
>
> My common usage was that I would add the entire contents of S, along
> with some associated configuration outside of S, and then make a commit
> of all of that in the superproject.

This could be achieved in a very hacky way by moving the `.git` directory of S out of the way, committing, then moving it back, right?

Show 5 quoted lines
> The two repos (superproject and submodule S) are then tracking the files
> in S independently; so if I was to pull new changes to the submodule
> from its own upstream, git commands run from the S directory would not
> show any changes vs the state of the submodule repo, whereas commands
> run from the superproject would see new changes.

I did indeed have a more integrated thing in mind, where the commits of the super-project that touch S and the commits of S were more connected.

Ciao, Johannes

> Cloning the superproject repo would produce its version of S, and
> without the S/.git directory.

← back to recent threads