RFC patchclone: introduce clone.submoduleGitDir to relocate $GITDIR

This configuration variable comes into effect when 'git clone' is invoked inside an existing git repository's worktree. When set, instead of cloning the given repository as-is, it relocates the gitdir of the repository to the path specified by this variable. This setting is especially useful when working with submodules.

Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
---
 Okay, so this is part of my evil plan to make 'git add' DTRT wrt
 submodules, and deprecate 'git submodule add' (I have some code
 written down, but this is a prerequisite: I don't like the
 .git/modules nonsense).

 Unfortunately, this patch is in pathetic shape and is an RFC for
 three reasons:

 1. I've used setup_git_directory_gently() at the start of
    builtin/clone.c to check if I'm inside a git directory.  This
    breaks a lot of existing tests (I'm yet to understand these
    failures fully).

 2. setup_git_directory_gently() has the side-effect of changing the
    current directory and calling set_git_work_tree(), both of which
    must be done away with if we want the rest of clone.c to work.
    I've hacked around the issue in a very dirty manner.  What is the
    solution to this?

  3. I don't know how to test the case "clone.submoduleGitDir has no
     effect outside a git repository", because our entire test
     environment is a git repository.  Even if I remove the .git
     directory, we're still inside the soure tree's git repository.
     What do I do about this?  Even if we decide that this patch is
     fundamentally unworkable, we should try to fix this issue so that
     we can verify that a plain 'git clone' works outside a git
     repository.

  Thanks for reading.  And I'm truly sorry for making you read through
  such ugly code.

 Documentation/config.txt | 11 +++++++++++
 builtin/clone.c          | 33 ++++++++++++++++++++++++++++++++-
 environment.c            | 11 -----------
 t/t5702-clone-options.sh | 41 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 84 insertions(+), 12 deletions(-)

Ramkumar Ramachandra <artagnon@gmail.com> writes:

Relocate to where in the superproject's gitdir? Presumably you can do this more than once in a given superproject, so there needs to be a key per such a clone, no? I am guessing that you would follow the usual "when adding a submodule without name, use its path as the initial name" convention, but then I would suggest it to be spelled out (and if you are doing it differently, that choice needs to be spelled out and defended).

If the envisioned use of this is to use it as a building block of something else that is user-facing (e.g. the user says "git add", and before the command finishes, somewhere we internally run "git clone"), then would it be possible that you are better off running that clone with --separate-git-dir and let it make the gitfile for you?

Any new configuration variable brings its own problem by forcing existing users to countermand it explicitly from the command line. If the --separate-git-dir would not work for your application, you need a new feature and you can achieve the same by adding a new command line option (say, --submodule-git-dir), that would be more preferrable.

Junio C Hamano <gitster@pobox.com> writes:

As you may have already guessed, in principle I am all for teaching "git add" not just to add a submodule itself (which we already do) but also to record information about the submodule, without having to delegate it to "git submodule". "git submodule add" was meant as an interim measure until we figure out what kind of metainformation is necessary, and doing things in "git add" has always been a longer term goal.

There are two ways to "add" a submodule to a superproject. You may bring an existing project with "git clone" inside the working tree of a superproject (which I am guessing is the use case that inspired this patch), but it will leave the git dir of the submodule embedded in its working tree.

You could continue "git clone" and then teach "git add" (or "git submodule add") to relocate the embedded git directory from the submodule working tree, you could "git clone" with separate-git-dir from the beginning, or you could extend "git add", perhaps

    git add --url=git://up.stre.am/repository [--name=name] sub/mod/ule

and do that "git clone --separate-git-dir" internally (which will mean that the end user will not run "git clone").

Another way ti "add" a submodule is to run "git init" to originate a new project inside the working tree of a superproject. The resulting submodule working tree will have the embedded git dir, and again "git add" (or "git submodule add") could notice and relocate it, but if the extended "git add" wants to help that use case as well, I think it is the matter of running "git init --separate-git-dir", just like "add by cloning from elsewhere" can do the same with the flag to "git clone".

I specifically did not go down this route, because I think it is gross. Where does moving a GITDIR fit into what git add's normal job (index manipulation) is? Tools should do one specific thing, and do it well: not a mixed bag of unrelated things. git clone, on the other hand, was always intended to have a way to point to a location for GITDIR and the worktree: isn't this feature very close to --separate-git-dir already? It is, therefore, git clone's job to relocate the GITDIR. My future plan is to deny git add'ing anything but a worktree-with-a-gitfile.

Ramkumar Ramachandra <artagnon@gmail.com> writes:

I see you are trying to repeat the UNIX mantra, but I do not think it is working.

When we discuss "git add", the "one unit of work" is at much higher level than that of "git update-index". "git add dir/" has to do a lot more than "git add file", and "git add symlink" has to do quite a different thing from "git add file", but to the end user, all of them are about doing everything necessary to add what the user named to the index. "git add submodule/" that does whatever necessary to add the submodule to the index is still doing one thing well inside the same framework, and that may include moving the $GIT_DIR and turning it into a gitfile.

Not that I am saying I prefer "add --url=xxx". Quite the opposite. I very much prefer the "clone and then add, but clone drops the repository at the right place from the beginning" approach than "add that knows about URL only for submodules", which is an ugly kludge.

If the user creates here/.git without gitlink with whatever means, it is "git add here"'s job, if it wants to make it a submodule and if it wants to make it possible to later check out another branch that does not have the submodule, to stash away the repository and turn it into gitfile, if it is part of what is needed to add a submodule.

Of course, we could start from teaching "submodule add" to do so, and then internally redirect "git add subm" to "git submodule add", but that is a minor implementation detail that does not affect the end user experience.

Junio C Hamano wrote:

You're looking at it from an end-user point of view, while I'm looking at it from the implementation point of view. Here's a coarse simplification of what git add does:

1. Lock the index file, and grab a FILE handle to read/ write the file.

2. Update active_cache.  Depending on the pathspec, we might be adding
one entry or multiple entries, with different modes to the index.
Nowhere did I say that it should add exactly one entry to active_cache
with a predefined mode.  Sure, tools that operate a lower layer of
abstraction like update-index can be more picky about this.

3. Write tree and blob objects to the database corresponding to the
worktree entries.  Files and symbolic links get blob objects, while
directories get tree objects.

4. Write active_cache to FILE handler we grabbed in step 1, and
release the lock.

What it does not do:

1. Move random files/ directory around in the worktree.

2. Mangle existing files in the worktree. (Although I know that the
.gitmodules-mangling is coming soon, I'm not exactly elated with it
[1])

3. Write commit or tag objects to the database.

4. Update random refs.

5. Make coffee for the user to applaud him on the successful add.

In my opinion, with some minor exceptions, all git tools follow these
principles.  Briefly, branch is a refs/heads/* helper, checkout is a
HEAD + worktree helper, fetch is a receive-pack + refs/remotes/*
helper, and reset is a bit of a swiss army knife that operates on HEAD
+ index + worktree.

In general, I like git because commands don't create unnatural or heavy abstractions on top of these concepts. With some minor exceptions, all the commands are easy to understand and consistent.

[1]: This is what led to my OBJ_LINK proposal.

I don't know why you brought up the alternative in the first place. We both agree that it is git clone's job, although your reason is more superficial and mine's tied to the implementation.

I disagree. I think we should get a first-class tool to attach/ detach worktrees from a GITDIR. It can incoporate the logic from contrib/workdir/git-new-workdir to optionally create a worktree with an independent index, HEAD, and logs/HEAD.

Yuck. Don't you care about the implementation, as long as it fixes the end-user's problem?

Junio C Hamano wrote:

I probably wasn't clear enough in the commit message, but this is what happens when I set clone.submoduleGitDir to ~/bare: a git clone gh:artagnon/clayoven inside the superproject's worktree will make ~/bare/clayoven.git and ./clayoven corresponding to the GITDIR and the worktree of the newly cloned repository. If there are conflicts, it will complain as usual saying that the destination path %s already exists, in which case the user has to choose a name for the GITDIR (not yet implemented) and/or the worktree path (as the final command-line argument to git clone).

Junio C Hamano wrote:

I'm getting a little tired of your first instinct to oppose every new addition to git. (Ofcourse I understand your attitude as the maintainer, but still)

It doesn't make sense as a command-line option, because it is "magic" that kicks in only when git clone is executed inside an existing git worktree. The point is that the user doesn't have to remember anything special: a normal git clone already does the right thing outside a git worktree; my proposal is to make it do the right thing inside a git worktree as well. Although I'm not against allowing a user to create a "full clone" inside a git repository by overriding clone.submoduleGitDir via a command-line option, I really cannot see why this would be anything but rare. Why would a user *want* a full clone inside a git worktree?

Also, naming it --submodule-git-dir can cause a lot of confusion: --separate-git-dir names a specific directory to put the GITDIR in, while --submodule-git-dir names a directory inside which to create other named directories to put GITDIRs in. Ofcourse clone.submoduleGitDir is a bad name too: any suggestions?

On Mon, Apr 15, 2013 at 6:19 PM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

If a user is inside .git, I believe setup_git_directory() will also find correct gitdir. In that case, we do not want magic (i.e. only do your magic when you are inside worktree). Still I'd rather see no magic (i.e. command line option) first. Let people try it out for a while. If people like it and find it inconvenient, magic can come later. I suspect you might want more magic in other places. Maybe if you hold it back until you see full picture, you'll only need a few new config keys (instead of one per separate magic).

Just do what scripts do: spawn a process to run rev-parse so that it does not mess up the main process. You might be able to introduce "dry-run" mode for setup_git_directory(), but that won't be easy. -- Duy

Duy Nguyen wrote:

Good suggestion.  I'll make it a command-line option for now.

Okay, thanks.

Ramkumar Ramachandra <artagnon@gmail.com> writes:

It was purly about "do not add anything that makes no sense." and not about "oppose all new addition."

When you add a submodule with the current system, bypassing "git submodule add", you can either

    (1) "git clone $URL here" and then "git add here"; or
    (2) "git init here" and then "git add here".

Because you didn't say what you are aiming for in the grander picture, I thought you were "making the UI simpler" by making it unnecessary for the users to say "git clone" himself as a separate step before doing "git add". In such a world, "add" would internally run "clone". If that were the case (I now know it is not), then the configuration _is_ unnecessary, and it is perfectly valid to question why you thought it is needed.

If your plan is instead to keep "git clone" followed by "git add" as the pattern for use case (1), teaching "clone" to automatically use the --separate-git-dir mechanism to point at the right place inside the $GIT_DIR of the superproject does make sense to help the use case.

But if that is the direction you are aiming for, would it be possible that the same configuration variable can and should cover the use case (2) as well? After all, between "git init here" and "git add here", the user may say (cd here && git pull $URL) and the expected end result would be the same as (1), no?

I do not recall the details of the codepaths involved offhand, but when you "git clone $URL [here]", after running "mkdir here", it would create a $GIT_DIR for the "here" repository in "here/.git" (and with --separate-git-dir, it would create it elsewhere and drop gitfile at "here/.git"). When you "git init here", after running "mkdir here", the same thing happens.

How common are these two implementations?

If "clone" just calls init_db(), I would imagine that it might be trivial to cover both cases by telling init_db() to pay attention to the configuration, without doing much in the "clone" itself.

Junio C Hamano wrote:

In the original email, I wrote:

I'm not sure how you inferred "making the UI simpler" from that, or the tests.

No, I would _never_ propose something that ugly. Neither my code, tests, nor my commit message indicates that I was going in that direction, so I don't know where you got the idea from.

Good point.  Yes, I would definitely want that.

Right.  I'll start hacking.

In general I think it is a mistake to overload "git clone" with the notion of adding a submodule. If I want to *add* something to a repository, I'll use some kind of "add" command. To me "git clone" is not the kind of verb I would expect to add something to some distant-parent .git directory.

Instead of mucking around with"git clone" I would much rather see "git add"
autodetect URLs and do the submodule thing:
	git add ssh://host/blammo.git
would clone blammo.git into ./blammo/ and set it up as a submodule inside
$PWD's git repo.  (This may benefit from "git clone" learning some kind of
--separate-git-dir option, but that's irrelevant to me.)

On 13-04-15 04:19 AM, Ramkumar Ramachandra wrote:

Please try to be careful with your assumptions.

I could have
	~/.git/
to maintain revisions of various personal files, config .dotfiles, scripts in
~/bin/ and so on.

I could also have various projects' repos under ~/Code, where I do my "real"
work:
	~/Code/git/.git/
	~/Code/DayJob/.git/
	~/Code/project-foo/.git/

Now, are these Code/* repos inside ~/.git/'s worktree or not? I'd really prefer them not to be. I would be especially upset to have some "magic" that automatically adds new clones inside ~/Code/ to ~/.git/.

		M.

Marc Branchaud <mbranchaud@xiplink.com> writes:

I agree with that principle, but my understanding is that this effort is not about teaching "git clone" to create a submodule.

Both "git clone" and "git init" already know how to use a directory that is outside the working tree of the newly created repository to store its $GIT_DIR and point at it with .git in the working tree using the gitfile mechanism (their --separate-git-dir option). My understanding is that this "config" is about making that option easier to use when you _know_ any new repository you create with "git clone" or "git init" inside your (toplevel super)project's working tree will become its submodule, as it is more convenient to have their $GIT_DIR inside the .git/modules/$name of the superproject.

After that "clone" or "init" creates a repository, you still have to "add" if you want to make it a submodule to the toplevel.

I do not think the addition Ram is envisioning in the patch will prevent you from teaching "add" to do that. An implemention of such an addition indeed would most likely use the same --separate-git-dir mechanism anyway.

Junio C Hamano wrote:

Right. But I'm still worried about .git/modules/$name. Can you explain why it's a better idea than having a dedicated ~/bare? In the case when I have it in ~/bare, I can do many more interesting things: for instance, if I cloned a repository that is actually another project's submodule for instance, I don't have to re-clone it when I clone that superproject. What's more? I can remove submodules and attach a worktree to my ~/bare/repo.git and use it as a separate repository easily. I can move submodules between projects. In comparison, .git/modules/$name just seems like a mess.

Well, I'm against the change in principle because add operates on worktree paths, not URLs. I don't want to change that arbitrarily.

On Mon, Apr 15, 2013 at 11:30:40PM +0530, Ramkumar Ramachandra wrote:

I do not have too much deep knowledge of submodules, nor have I been following this thread very closely, but I have not seen how ~/bare would handle per-submodule information?

That is, let us imagine I do:

  git clone $PROJECT one && cd one && git submodule update foo
  git clone $PROJECT two && cd two && git submodule update foo

The current scheme would put the cloned modules into one/.git/modules/foo and two/.git/modules/foo, respectively. Let us imagine instead that the first one writes to ~/modules/$URL (assuming some sane mapping of the URL into the filesystem), and the second one says "A-ha, I already have ~/modules/$URL, so I can skip cloning it".

But that is not the end of the story. If I do:

  cd one/foo &&
  hack hack hack &&
  git commit -m foo &&
  cd .. &&
  git commit -m 'updated submodule'

you would not want to see a dirty, updated submodule in project "two". You did not touch "two/foo" nor advance its HEAD at all.

So there is some information that is per-clone (the objects, the remote tips), but there is some information that is per-submodule (where our local branches are, the index, the worktree). I can see why it is advantageous to share the per-clone information between similar clones (because it avoids disk space and network transfer). But I do not think you can escape having some form of per-submodule repo, even if it is a thin git-new-workdir-ish repo that points back to a parent repo for the clone.

Is there some part of your proposal that I am missing? It seems like you would still need one/.git/modules/foo for this "thin" repo.

And once we separate out those concerns, I also do not see why sharing per-clone information needs to be related to submodules at all. If I do:

  git clone $URL one &&
  git clone $URL two

those can potentially be shared in the same way as two submodule repos that happen to point to the same $URL. It would make sense to me to improve such a shared-object setup independently, and then build the shared-submodule storage on top of that.

And by the way, I am actually not sure that such a shared-object setup is a good idea, but only that _if_ you are going to do it with submodules, you might as well do it for all repos. In theory, it is not that hard to have a big per-user object-only repository (either for all repos, or for related ones). But we can do that already with "git clone -s", and people do not generally bother, because the maintenance is very tricky (especially dealing with reachability and pruning).

I am open to the argument that solving it in a specific case (submodules) lets us make assumptions that simplify the problem from the general case, but I do not offhand see how it would be any easier in this case.

Jeff King <peff@peff.net> writes:

Nicely put.

Making it easier to manage such a shared object store by limiting use cases is somewhat an intriguing idea, but those I can think of offhand all have to involve a use case without any rewound history, so being a submodules repository would not help.

Jeff King wrote:

I want the flexibility to do the following:

1. Do a "simple clone", where the clone contains the GITDIR embedded
in the worktree.  This is the most common case, and there is no reason
to complicate it.  I can optionally attach additional workdirs to this
clone.  I can also optionally relocate the GITDIR at a later date, if
I feel the need to do so.

2. Attach a worktree to any object store without having to write a
gitfile and set core.worktree by hand.  The limitation is that you
can't have two submodules from two different superprojects sharing the
same object store (since both of them are worktrees).  However, for
the purpose of working on the submodule repository as an independent
repository (this is a very common case for me), I can attach a new
"workdir" to the GITDIR very easily.

3. Attach multiple submodules to the same object store.  This will
require maintaining a separate index, HEAD and logs/HEAD (aka.
workdir) for each additional submodule (the first one doesn't need it)
in .git/modules of the superproject.

You're talking about #3, while I'm still working on #2. And why do you want to use a hammer again if I don't want to share the same object store with multiple submodules? This .git/modules/<name> is completely optional, and is only required for the _second_ submodule onwards that I'm attaching to the same object store.

No, no. I'm against dumping objects from all repositories into one giant object store. That's a sledgehammer solution, while I'm looking for control and flexibility. Moreover, it has lots of downsides, as you already pointed out.

So my proposal is to build a new first-class tool to make manipulations in #1, #2 and #3 easily possible. The first step is to formalize the names "bare worktree" (which refers to a worktree with a gitfile), "worktree" (which refers to a worktree with a GITDIR embedded in it), and "workdir" (which refers to a worktree with a "thin" GITDIR).

The reason I want to build it for submodules first is because the non-submodule case (#2) is simply a reduced case of the submodule case (#3):

- When I attempt to attach a new worktree to an existing GITDIR with a
worktree attached, I will create a workdir instead.  This simply
involves creating a thin .git directory in the worktree in the
non-submodule case.  In the submodule case, it is more complicated: I
have to locate the superproject's .git directory, and put it there.

On 13-04-16 04:13 AM, Ramkumar Ramachandra wrote:

Doesn't contrib/workdir/git-new-workdir do this?

		M.

On 13-04-15 02:00 PM, Ramkumar Ramachandra wrote:

I don't understand that statement.

If "git add" is all about specifying what lives under paths in the worktree, what's wrong with letting "git add" go beyond specifying just files?

Syntax aside for the moment, I think a command like
	git add git-repo-reference foo
is perfectly natural:  It specifies what is inside worktree path foo.

		M.

Marc Branchaud wrote:

I never said "just files". Files, directories, symlinks and submodules are all "things in the worktree", and all fine. Remote URLs, on the other hand, have nothing to do with the worktree.

On 13-04-16 04:17 AM, Ramkumar Ramachandra wrote:

But they have everything to do with submodules. You need a URL to identify a submodule. If you want a submodule in your worktree, at some point you have to specify the submodule's URL.

I really feel like I'm missing something here. You seem to be saying that it's wrong to let "git add" interpret a URL as a submodule. Instead you seem to want to have some other mechanism create the files, directories and symlinks that make up a submodule, so that "git add" can then operate with the purity you desire. That's what I don't understand.

As a submodule user, I want to "git add" a submodule. I don't see why it's necessary to have more than one command to do that. But if you're saying that it's fine for "git add" to work this way, then I don't see the point of the proposed change to "git clone".

		M.

On 13-04-15 01:50 PM, Junio C Hamano wrote:

To me it makes more sense to move the .git directory when the user invokes
"git submodule add" instead of creating it in an unusual place when the
sub-repo is cloned.  After all, git can't *know* that it'll be a submodule
until it's submodule-added to the super-repo.  Sure, the user might have set
clone.submoduleGitDir somewhere, but users make mistakes, and this setting
makes it harder to clean up a mistake:
	git clone foo.git
	# Doh!  I mean to clone foof.git!
	rm -rf foo
	# Gah, now there's cruft in my clone.submoduleGitDir...

All that said, the basic idea of being able to configure where "git clone" stores .git directories might be reasonable. Something like clone.gitDirHome. It seems like something only a git hacker would ever care about, but that's no reason not to have such a config option. OTOH, I still don't see a reason for it, because I don't buy the submodule-at-clone-time argument.

		M.

Marc Branchaud <marcnarc@xiplink.com> writes:

It does not relieve "git add" (or "git submodulea add") from the responsibility of moving .git directory. It only reduces the need to do so.

When the user says "add" and the repository has .git directory in it, "add" (or "submodule add") is still responsible for relocating it.

On 13-04-15 02:50 PM, Junio C Hamano wrote:

So it looks like the proposed change to git-clone provides no benefit to the submodule-adding machinery, which still needs to know when and how to relocate .git directories.

Ram, assuming Junio's explanations match your intentions, if the whole motivation for this change is "to make 'git add' DTRT wrt submodules, and deprecate 'git submodule add'" then I don't think it's bringing you any closer to that goal.

		M.

Marc Branchaud <marcnarc@xiplink.com> writes:

That is a huge assumption, given that I have a proven track record of guessing Ram's intention wrong ;-).

Junio C Hamano wrote:

Since you're so stubborn about it, I suppose 'git add' could call a function in my "new first-class program to attach detach worktrees/workdirs and relocate GITDIRs" as a last resort (if the user somehow managed to put a GITDIR in the submodule worktree despite our well-designed tools). But last resort is not what we should be discussing now: we're discussing what the design should ideally be. And ideally, I think we both agree that it's best if init/clone did the relocation.

On 13-04-16 04:21 AM, Ramkumar Ramachandra wrote:

If that's the question, then put me on the "disagree" side. I just don't see why that approach is "best", especially if the intention is "to make 'git add' DTRT wrt submodules, and deprecate 'git submodule add'".

		M.

Marc Branchaud wrote:

Heh. And I want git add *coffee* to make me coffee. What's your gripe with git submodule add?

Please don't version your entire ~, effectively shooting yourself in the face?

Use a dotfiles repo and write a simple Makefile to symlink ~/.etc to ~/dotfiles/.etc. If you're looking for a good example, see https://github.com/artagnon/dotfiles.

Ramkumar Ramachandra wrote:

Interesting. As the discussion downthread from this illustrated, I am not convinced this is better than a subcommand of "git submodule" for that particular purpose, yet.

Is the goal to be able to, under some certain configuration, make "git clone" + "git add" behave like "git submodule add"?

[...]

As Jeff mentioned, a given repository can be a subproject of multiple different containing projects, that use different versions of it. It doesn't make sense for different directories on the filesystem to share an index anyway.

Do you want the subprojects to be symlinks to the One True Version of each project? (I can see that working ok in some workflows.) Or do you want subprojects to be lightweight workdirs like git-new-workdir creates, with .git/objects pointing to the project's One True Object Store?

That is the part of this design that seems least well fleshed out to me at the moment.

I quite like .git/modules/<subproject name> (for some reasons that I've mentioned in other threads) and don't consider it nonsense, which makes me assume I don't understand the goal of this patch, either. Please don't take that personally.

Hope that helps, Jonathan

Jonathan Nieder wrote:

There's nothing to take personally, Jonathan. We're designing software, and the rationale for choosing a design is never "Jonathan personally likes this particular design, so therefore we'll go with it", but rather "Ram's design is objectively superior, and therefore we'll go with it". I'll proceed with bashing .git/modules, while your job is to defend it:

1. The path to the object store of a submodule depends upon how deeply
it is nested in other submodules, and hence how many /modules/
components to add to the path to the project's name.  Presumably, this
is to avoid conflicts: but it's an overkill for such a simple job.  In
the 98% case, I never have two submodules with the same name in my
superproject; for the 2% case, I can live with the inconvenience of
naming a directory by hand, rather than putting up with this ugliness.

2. This ugliness complicates implementation of add/ rm/ mv, because
each of them will have to know about this contrived path solution.

3. The paths in the gitfiles in various submodules is horribly ugly
with tons of ../ components.  This is especially the case in deeply
nested submodules.  We can't use an absolute path, because the
superproject directory can be moved anywhere in the filesystem.

4. To relocate the object store and reuse it elsewhere is almost
impossible.  What if I want to remove the submodule, but work on it
independently from the superproject?  Re-clone?

My solution fixes all these problems, and we need .git/modules/<name>.git (no path-to-submodule nonsense) only as a last resort: #3 (ref: my email to Peff).

Ramkumar Ramachandra <artagnon@gmail.com> writes:

Have you noticed that there are distinction between submodule path and submodule name already in the current system, and name is derived from path if you do not give it when adding a submodule merely as a convenience?

If some existing code uses .git/modules/<path>.git in "git submodule", that is a bug that needs to be fixed.

Ramkumar Ramachandra wrote:

Why is that? Can't they look at the gitfile or call some helper (that happens to be part of the same binary)?

On Sun, Apr 14, 2013 at 5:23 AM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

What if I clone a repo then realize it was a mistake and remove it? With current clone, a "rm -rf" would do. With this, I'll need to figure out which subdir in the top .git contains the repo I want to remove. I'm not sure how "git submodule" handles this case though (i.e. total submodule ignorant speaking..) -- Duy

Duy Nguyen wrote:

Currently, submodules relocate the GITDIR of submodules to .git/modules. So, my proposed patch doesn't make the situation any worse. In fact, it improves the situation because you're guaranteed that all your GITDIRs will be in ~/bare (or whatever your core.submoduleGitDir is), as opposed to a complex path in .git/modules of your containing superproject.

On Wed, Apr 17, 2013 at 8:53 PM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

No, submodule code does not change "git clone". If I'm not mistaken, submodule will not kick in until you type "git submodule something". If I turn clone.submoduleGitDir on, how can I undo my mistake in a user friendly way? -- Duy

Duy Nguyen wrote:

So, if you currently want to add a submodule, you have to 'git submodule add', which runs clone internally apart from other things. How do you undo this mistake?

What I'm essentially proposing is to give the job of cloning back to clone, and the job of adding back to add, instead of creating an unnatural abstraction over them using 'git submodule add'. The point being: why would you ever _want_ to clone inside a worktree unless you intend to add a submodule? In other words, you intent for running a 'git clone' inside a worktree is exactly the same as your intent for running a 'git submodule add' inside a worktree. Ofcourse, if you have a fringe case where that was _not_ your intent, we'll provide a command-line switch to turn off the relocation for that clone.

On Wed, Apr 17, 2013 at 9:13 PM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

Well, it has "submodule" in the command line. My first reaction would be looking for "git submodule rm" or something.

No, the point is people make mistakes. What do we do in that case? Or will you introduce yet another "gc" command for clean up ~/bare? -- Duy

Duy Nguyen wrote:

No, 'git submodule rm' cannot remove the corresponding GITDIR. What if there are other branches that refer to the submodule? What if you want to remove it from this branch and add it to another branch?

So, people don't make mistakes when they use 'git submodule add', but do make mistakes when using 'git clone'? How has the problem _changed_ with my patch? It's reasonable to point it out as an existing problem, and ask for it to be fixed independent of this discussion, but that is not what you are doing.

git cannot read your mind to determine if you made a mistake, if that's what you're asking. No, a gc equivalent won't work either (and there's nothing in the current submodule world), because it is impossible to determine if a workdir is attached to that GITDIR somewhere on your filesystem.

You'll have to do _something_ to say that you don't want that GITDIR anymore. It's reasonable to request tooling to help with this task, but your request is entirely different.

On Thu, Apr 18, 2013 at 1:02 AM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

It's the magic in git-clone that changes its behavior that I want to address. I know you agree to go with a command line option. But I think in the end there will be a switch hidden somewhere in config to make things smooth, unless you make this mode the default (*). With normal mode, "rm -rf repo" is enough, with the new submodule mode, it leaves some garbage behind that the user may not be aware about. Maybe this is something that should be addressed anyway even for .gitmodules mode like Junio said. But I wonder if there are any other traps that come with the config switch.

(*) I don't think you can make the new mode the default though. There are repos in repos in the field that are not managed by "git submodule". Switching the default will disrupt those setups. Some deprecation cycles might help, I don't know. -- Duy

Duy Nguyen <pclouds@gmail.com> writes:

I do not know if it will be a "gc", but we would need a way for the user to say "I no longer need the repository for this submodule kept locally here (I may have to re-clone when I check out a version that needs the submodule)" to free up the .git/modules/<name> directories in the superproject. We might want to allow "submodule deinit" to also ask for it, but "deinit" will not be the only occasion the user might want it.

It is already a problem that needs to be addressed in the current setup.