Re: git submodules implementation question

From: Uma Srinivasan <usrinivasan@twitter.com>
Date: Sep 1, 2016, 19:56 UTC
Message-ID: <CAN5XQft1YFEdxcB8Q_qFG4iYmSaSX_JXDCESzeEyYqPO0BUbaw@mail.gmail.com>
In-Reply-To: <xmqqa8frwhpr.fsf@gitster.mtv.corp.google.com>

>>> The final version needs to be accompanied with tests to show the
>>> effect of this change for callers.  A test would set up a top-level
>>> and submodule, deliberately break submodule/.git/ repository and
>>> show what breaks and how without this change.

Agreed!

The repo where the original problem surfaced is huge and in fact "git status" appears to go into an infinite loop forking a zillion processes on the system. If the dev system is small, it brings it to a standstill. However, the good news is that I could build myself a smaller reproducer by doing the following:

1) mkdir sm_test
2) cd sm_test
3) git clone git://git.mysociety.org/commonlib commonlib
4) git init
5) git submodule add ./commonlib/
6) cd commonlib/.git
7) rm -f all files

After this "git status" will fork several thousand processes but it will ultimately fail as it runs into the path length max limit. I still don't know why this doesn't happen with my original problem repo. Perhaps I have to let it run overnight or perhaps it runs into other system limitations before then....

Anyway, with the fix "git status" fails quickly both in my reproducer (and original repo) with the following message..... fatal: Not a git repository: '.git' fatal: 'git status --porcelain' failed in submodule commonlib

Hope this helps.

Uma

On Thu, Sep 1, 2016 at 12:19 PM, Junio C Hamano <gitster@pobox.com> wrote:

Show 24 quoted lines

> Stefan Beller <sbeller@google.com> writes:
>
>>> The final version needs to be accompanied with tests to show the
>>> effect of this change for callers.  A test would set up a top-level
>>> and submodule, deliberately break submodule/.git/ repository and
>>> show what breaks and how without this change.
>>
>> Tests are really good at providing this context as well, or to communicate
>> the actual underlying problem, which is not quite clear to me.
>> That is why I refrained from jumping into the discussion as I think the
>> first few emails were dropped from the mailing list and I am missing context.
>
> I do not know where you started reading, but the gist of it is that
> submodule.c spawns subprocess to run in the submodule's context by
> assuming that chdir'ing into the <path> of the submodule and running
> it (i.e. cp.dir set to <path> to drive start_command(&cp)) is
> sufficient.  When <path>/.git (either it is a directory itself or it
> points at a directory in .git/module/<name> in the superproject) is
> a corrupt repository, running "git -C <path> command" would try to
> auto-detect the repository, because it thinks <path>/.git is not a
> repository and it thinks it is not at the top-level of the working
> tree, and instead finds the repository of the top-level, which is
> almost never what we want.
>