threads / discuss / 31943

Unexpected behaviour after removing submodule

Subject: Unexpected behaviour after removing submodule

## tl;dr

3 messages between Oct 25, 2012 and Oct 26, 2012.

replies: 2people: 2as markdown or json

Nicolas Morey-Chaisemartin· Oct 25, 2012, 15:06 UTC · lore

Hi,

At work, we use a lot of submodules (several levels of submodules actually). As we also work with development branches, we use scripts to resync the whole checked-out tree (mainly in automated integration)

We recently run across an issue where a branch (dev) contained a submodule while it had been removed in master and the files were imported in a subdirectory with the same name (probably using git-subtree).

Basically:

On dev:
* top/refs(submodule)/file1
On master:
* top/refs(dir)/file1

Outside the fact that it is quite hard to move from one branch to the other while having a perfectly clean tree checked out underneath, we manage to end up into a weird (invalid) state that was neither clearly described nor "easy" to cleanup (using standard git clean/checkout/reset commands).

While I cannot explain how we got in this state, here is a small test-case that produce the same results: ------ mkdir folder1 cd folder1 git init echo "Ooops" > file git add file git commit -m "Add file" cd .. mkdir folder2 cd folder2 git init mkdir folder1 echo "Ooops" > folder1/file git add folder1/file git commit -m "Add file again" git checkout -b branch cp -R ../folder1/.git ./folder1 ------ The 'cp' just seems pointless but with the submodule described as above we manage to end up in a similar state. In this state, when being in folder2, git status reports nothing. Dev branch is checked out and everything looks great.

However if you change dir to folder2/folder1, while still being inside folder2, git thinks (because of the .git dir) that you are actually on master branch of folder1 repository. Which mean that if you happen to commit from a subdirectory, you may easily end-up committing in another repository than the one expected.

The issue is, there is no way from folder2 to see that something "wrong" is going on inside your tree! As we manage to reach this state using only standard git commands (I'll try to reproduce it) with submodules, and this being part of an automated flow, it is quite worrying. We may actually be committing in the wrong repo and pushing the wrong things around.

Is there or should there be a way to look for such issues? And is this an expected behaviour?

Thanks in advance

Nicolas

Jens Lehmann· Oct 25, 2012, 22:47 UTC · re: Nicolas Morey-Chaisemartin · lore

Re: Unexpected behaviour after removing submodule

Am 25.10.2012 17:06, schrieb Nicolas Morey-Chaisemartin:

Show 14 quoted lines

> At work, we use a lot of submodules (several levels of submodules actually).
> As we also work with development branches, we use scripts to resync the whole checked-out tree (mainly in automated integration)
> 
> We recently run across an issue where a branch (dev) contained a submodule while it had been removed in master and the files were imported in a subdirectory with the same name (probably using git-subtree).
> 
> Basically:
> 
> On dev:
> * top/refs(submodule)/file1
> On master:
> * top/refs(dir)/file1
> 
> Outside the fact that it is quite hard to move from one branch to the other while having a perfectly clean tree checked out underneath, we manage to end up into a weird (invalid) state
> that was neither clearly described nor "easy" to cleanup (using standard git clean/checkout/reset commands).

<snipped example without submodules>

Show 5 quoted lines

> The issue is, there is no way from folder2 to see that something "wrong" is going on inside your tree!
> As we manage to reach this state using only standard git commands (I'll try to reproduce it) with submodules, and this being part of an automated flow, it is quite worrying.
> We may actually be committing in the wrong repo and pushing the wrong things around.
> 
> Is there or should there be a way to look for such issues? And is this an expected behaviour?

Unfortunately this is how things work at the moment. If you remove a submodule its work tree will currently stay around nonetheless. And when you replace it with a directory containing files tracked by git, things start to get really weird when you do checkouts crossing that conversion.

But the solution to that problem is coming closer. The first step was to move the .git directory out of the submodule's work tree, so that we are able to remove it without loosing any history. The next step will be to enable git to remove and re-add submodules on checkout. The "remove a submodule" part already works in my - still experimental - recursive_submodule_checkout branch at my github repo. The thing that is still missing - apart from tests ;) - is that checkout has to learn to look into the to-be-checked-out .gitmodules to be able to populate a re-appearing submodule. I plan to add that for git fetch first (so it can fetch submodules the user showed interest in but which aren't currently checked out) and after that I'll reuse it for checkout.

But that's still some work to do and will take some time ...

Nicolas Morey-Chaisemartin· Oct 26, 2012, 10:05 UTC · re: Jens Lehmann · lore

Re: Unexpected behaviour after removing submodule

On 26/10/12 00:47, Jens Lehmann wrote:

Show 46 quoted lines

> Am 25.10.2012 17:06, schrieb Nicolas Morey-Chaisemartin:
>> At work, we use a lot of submodules (several levels of submodules actually).
>> As we also work with development branches, we use scripts to resync the whole checked-out tree (mainly in automated integration)
>>
>> We recently run across an issue where a branch (dev) contained a submodule while it had been removed in master and the files were imported in a subdirectory with the same name (probably using git-subtree).
>>
>> Basically:
>>
>> On dev:
>> * top/refs(submodule)/file1
>> On master:
>> * top/refs(dir)/file1
>>
>> Outside the fact that it is quite hard to move from one branch to the other while having a perfectly clean tree checked out underneath, we manage to end up into a weird (invalid) state
>> that was neither clearly described nor "easy" to cleanup (using standard git clean/checkout/reset commands).
> <snipped example without submodules>
>
>> The issue is, there is no way from folder2 to see that something "wrong" is going on inside your tree!
>> As we manage to reach this state using only standard git commands (I'll try to reproduce it) with submodules, and this being part of an automated flow, it is quite worrying.
>> We may actually be committing in the wrong repo and pushing the wrong things around.
>>
>> Is there or should there be a way to look for such issues? And is this an expected behaviour?
> Unfortunately this is how things work at the moment. If you remove a
> submodule its work tree will currently stay around nonetheless. And
> when you replace it with a directory containing files tracked by git,
> things start to get really weird when you do checkouts crossing that
> conversion.
>
> But the solution to that problem is coming closer. The first step was
> to move the .git directory out of the submodule's work tree, so that
> we are able to remove it without loosing any history. The next step
> will be to enable git to remove and re-add submodules on checkout. The
> "remove a submodule" part already works in my - still experimental -
> recursive_submodule_checkout branch at my github repo. The thing that
> is still missing - apart from tests ;) - is that checkout has to learn
> to look into the to-be-checked-out .gitmodules to be able to populate
> a re-appearing submodule. I plan to add that for git fetch first (so
> it can fetch submodules the user showed interest in but which aren't
> currently checked out) and after that I'll reuse it for checkout.
>
> But that's still some work to do and will take some time ...
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

That's what I figured. I just wished there was at least a way to see that there's a .git directory that does not really belong. To check that now we'd have to do a git-ls tree and look for .git that shouldn't be here but with the amount of file we have, a shell script doing that would take ages !

We'll wait for a more submodule aware version of checkout then !

Thanks

Nicolas