`.git` symlink makes `git submodule add` fail

Hi,

	$ uname -a
	Linux witiko 4.3.0-1-amd64 #1 SMP Debian 4.3.5-1 (2016-02-06) x86_64 GNU/Linux
	$ git --version
	git version 2.7.0

I tend to make `.git` a symlink, when I need to maintain several Git repositories inside a single directory. This was always transparent to Git, but problems arose, when working with submodules:

	$ mkdir -p .repos/repoA
	$ ln -s .repos/repoA .git
	$ git init
	Initialized empty Git repository in /tmp/tmp.q3hkme6nm4/.repos/repoA/
	$ git submodule add https://github.com/witiko/git-parallel
	Cloning into 'git-parallel'...
	remote: Counting objects: 212, done.
	remote: Compressing objects: 100% (16/16), done.
	remote: Total 212 (delta 6), reused 1 (delta 1), pack-reused 195
	Receiving objects: 100% (212/212), 64.20 KiB | 0 bytes/s, done.
	Resolving deltas: 100% (120/120), done.
	Checking connectivity... done.
	fatal: Could not chdir to '../../../git-parallel': No such file or directory
	Unable to checkout submodule 'git-parallel'

Is this a bug, or is the ability to symlink `.git` just a happy coincidence?

Best Regards, Vít

Vít Novotný <witiko@gmail.com> writes:

It has never been supported.

Junio C Hamano <gitster@pobox.com> writes:

Oops, hit "send" too early.

We have support for a "gitdir:" facility that would work even on a filesystem that cannot do symlinks (see gitrepository-layout(5)), and both the higher-level submodule Porcelain and the more recent "worktree" experimental code do use it.

On Fri, Feb 12, 2016 at 10:19:38AM -0800, Junio C Hamano wrote:

And the way to convince git to make the link for you is with clone's "--separate-git-dir" option.

-Peff

On Fri, Feb 12, 2016 at 01:27:33PM -0500, Jeff King wrote:

What's curious is that this doesn't really work either, so the issue doesn't seem to be the lack of symlink support, but rather the lack of willingness on the part of Git to resolve a path recursively.

The following works flawlessly:

  $ mkdir repos
  $ git init
  $ mv .git repos/repoA
  $ printf 'gitdir: repos/repoA\n' >.git
  $ git submodule add https://github.com/witiko/git-parallel

There is, however, a minor pain, which makes the repository unportable:

  $ cat git-parallel/.git
  gitdir: /tmp/foobar/repos/repoA/modules/git-parallel

When I change the gitdir symlink to a relative path, everything works fine as long as I don't make Git go over two gitdir symlinks:

  $ sed -i 's#^/tmp/foobar#..#' git-parallel/.git
  $ git status
  [works]
  $ sed -i 's#repos/repoA#.git#' git-parallel/.git
  $ git status
  fatal: Not a git repository: git-parallel/../.git/modules/git-parallel

Clearly, Git resolves the first gitdir symlink:

  git-parallel/.git => git-parallel/../.git/modules/git-parallel

but refuses to resolve recursively:

  git-parallel/../.git/modules/git-parallel =>
  git-parallel/../repos/repoA/modules/git-parallel

On Sat, Feb 13, 2016 at 05:20:09AM +0100, Vít Novotný wrote:

Right. The fundamental problem is that if git needs to construct a related path, symlinks mean it cannot do so textually. I'm sure there are spots where we use real_path() to get a canonical path for a particular $GIT_DIR, but from your results, it's clear we don't do so consistently.

So you can call that "git doesn't support symlinks", or you can call it "git supports symlinks, but needs to do so more consistently". :) I am not sure whether the existing uses of real_path() are there intentionally to support the kind of path-munging that submodules do or not. But either way, canonicalizing the paths before creating relative ones is probably going to be the solution.

I say all of that without having really looked at the code. You asked earlier if there would be any objects to a patch to implement better symlink support. That is hard to answer without somebody digging in to see what the fix would look like, and whether there would be any tradeoffs.

-Peff

On Fri, Feb 12, 2016 at 10:19:38AM -0800, Junio C Hamano wrote:

That sounds ideal, thank you for letting me know. Still, I believe that well-behaved applications should be indifferent to symlinks, unless the distinction is significant. Would there be any objections, if I submitted a patch / series of patches that would add proper symlink support?