{"thread":{"id":"31943","subject":"Unexpected behaviour after removing submodule","startedAt":"2012-10-25T15:06:40Z","lastAt":"2012-10-26T10:05:24Z","messageCount":3,"participants":["Nicolas Morey-Chaisemartin","Jens Lehmann"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"201904","messageId":"50895580.9080408@morey-chaisemartin.com","threadId":"31943","inReplyTo":null,"subject":"Unexpected behaviour after removing submodule","fromName":"Nicolas Morey-Chaisemartin","fromEmail":"devel-git@morey-chaisemartin.com","sentAt":"2012-10-25T15:06:40Z","receivedAt":"2012-10-25T15:06:40Z","isPatch":false,"sender":{"key":"devel-git@morey-chaisemartin.com","avatar":"https://avatars.githubusercontent.com/u/108326?v=4"},"body":"Hi,\n\nAt work, we use a lot of submodules (several levels of submodules actually).\nAs we also work with development branches, we use scripts to resync the whole checked-out tree (mainly in automated integration)\n\nWe recently run across an issue where a branch (dev) contained a submodule while it had been removed in master and the files were imported in a subdirectory with the same name (probably using git-subtree).\n\nBasically:\n\nOn dev:\n* top/refs(submodule)/file1\nOn master:\n* top/refs(dir)/file1\n\nOutside the fact that it is quite hard to move from one branch to the other while having a perfectly clean tree checked out underneath, we manage to end up into a weird (invalid) state\nthat was neither clearly described nor \"easy\" to cleanup (using standard git clean/checkout/reset commands).\n\nWhile I cannot explain how we got in this state, here is a small test-case that produce the same results:\n------\nmkdir folder1\ncd folder1\ngit init\necho \"Ooops\" > file\ngit add file\ngit commit -m \"Add file\"\ncd ..\nmkdir folder2\ncd folder2\ngit init\nmkdir folder1\necho \"Ooops\" > folder1/file\ngit add folder1/file\ngit commit -m \"Add file again\"\ngit checkout -b branch\ncp -R ../folder1/.git ./folder1\n------\nThe 'cp' just seems pointless but with the submodule described as above we manage to end up in a similar state.\nIn this state, when being in folder2, git status reports nothing. Dev branch is checked out and everything looks great.\n\nHowever if you change dir to folder2/folder1, while still being inside folder2, git thinks (because of the .git dir) that you are actually on master branch of folder1 repository.\nWhich mean that if you happen to commit from a subdirectory, you may easily end-up committing in another repository than the one expected.\n\nThe issue is, there is no way from folder2 to see that something \"wrong\" is going on inside your tree!\nAs we manage to reach this state using only standard git commands (I'll try to reproduce it) with submodules, and this being part of an automated flow, it is quite worrying.\nWe may actually be committing in the wrong repo and pushing the wrong things around.\n\nIs there or should there be a way to look for such issues? And is this an expected behaviour?\n\nThanks in advance\n\nNicolas\n"},{"id":"201925","messageId":"5089C192.2050903@web.de","threadId":"31943","inReplyTo":"50895580.9080408@morey-chaisemartin.com","subject":"Re: Unexpected behaviour after removing submodule","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-10-25T22:47:46Z","receivedAt":"2012-10-25T22:47:46Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 25.10.2012 17:06, schrieb Nicolas Morey-Chaisemartin:\n> At work, we use a lot of submodules (several levels of submodules actually).\n> As we also work with development branches, we use scripts to resync the whole checked-out tree (mainly in automated integration)\n> \n> We recently run across an issue where a branch (dev) contained a submodule while it had been removed in master and the files were imported in a subdirectory with the same name (probably using git-subtree).\n> \n> Basically:\n> \n> On dev:\n> * top/refs(submodule)/file1\n> On master:\n> * top/refs(dir)/file1\n> \n> Outside the fact that it is quite hard to move from one branch to the other while having a perfectly clean tree checked out underneath, we manage to end up into a weird (invalid) state\n> that was neither clearly described nor \"easy\" to cleanup (using standard git clean/checkout/reset commands).\n\n<snipped example without submodules>\n\n> The issue is, there is no way from folder2 to see that something \"wrong\" is going on inside your tree!\n> As we manage to reach this state using only standard git commands (I'll try to reproduce it) with submodules, and this being part of an automated flow, it is quite worrying.\n> We may actually be committing in the wrong repo and pushing the wrong things around.\n> \n> Is there or should there be a way to look for such issues? And is this an expected behaviour?\n\nUnfortunately this is how things work at the moment. If you remove a\nsubmodule its work tree will currently stay around nonetheless. And\nwhen you replace it with a directory containing files tracked by git,\nthings start to get really weird when you do checkouts crossing that\nconversion.\n\nBut the solution to that problem is coming closer. The first step was\nto move the .git directory out of the submodule's work tree, so that\nwe are able to remove it without loosing any history. The next step\nwill be to enable git to remove and re-add submodules on checkout. The\n\"remove a submodule\" part already works in my - still experimental -\nrecursive_submodule_checkout branch at my github repo. The thing that\nis still missing - apart from tests ;) - is that checkout has to learn\nto look into the to-be-checked-out .gitmodules to be able to populate\na re-appearing submodule. I plan to add that for git fetch first (so\nit can fetch submodules the user showed interest in but which aren't\ncurrently checked out) and after that I'll reuse it for checkout.\n\nBut that's still some work to do and will take some time ...\n"},{"id":"201935","messageId":"508A6064.4060706@morey-chaisemartin.com","threadId":"31943","inReplyTo":"5089C192.2050903@web.de","subject":"Re: Unexpected behaviour after removing submodule","fromName":"Nicolas Morey-Chaisemartin","fromEmail":"devel-git@morey-chaisemartin.com","sentAt":"2012-10-26T10:05:24Z","receivedAt":"2012-10-26T10:05:24Z","isPatch":false,"sender":{"key":"devel-git@morey-chaisemartin.com","avatar":"https://avatars.githubusercontent.com/u/108326?v=4"},"body":"On 26/10/12 00:47, Jens Lehmann wrote:\n> Am 25.10.2012 17:06, schrieb Nicolas Morey-Chaisemartin:\n>> At work, we use a lot of submodules (several levels of submodules actually).\n>> As we also work with development branches, we use scripts to resync the whole checked-out tree (mainly in automated integration)\n>>\n>> We recently run across an issue where a branch (dev) contained a submodule while it had been removed in master and the files were imported in a subdirectory with the same name (probably using git-subtree).\n>>\n>> Basically:\n>>\n>> On dev:\n>> * top/refs(submodule)/file1\n>> On master:\n>> * top/refs(dir)/file1\n>>\n>> Outside the fact that it is quite hard to move from one branch to the other while having a perfectly clean tree checked out underneath, we manage to end up into a weird (invalid) state\n>> that was neither clearly described nor \"easy\" to cleanup (using standard git clean/checkout/reset commands).\n> <snipped example without submodules>\n>\n>> The issue is, there is no way from folder2 to see that something \"wrong\" is going on inside your tree!\n>> As we manage to reach this state using only standard git commands (I'll try to reproduce it) with submodules, and this being part of an automated flow, it is quite worrying.\n>> We may actually be committing in the wrong repo and pushing the wrong things around.\n>>\n>> Is there or should there be a way to look for such issues? And is this an expected behaviour?\n> Unfortunately this is how things work at the moment. If you remove a\n> submodule its work tree will currently stay around nonetheless. And\n> when you replace it with a directory containing files tracked by git,\n> things start to get really weird when you do checkouts crossing that\n> conversion.\n>\n> But the solution to that problem is coming closer. The first step was\n> to move the .git directory out of the submodule's work tree, so that\n> we are able to remove it without loosing any history. The next step\n> will be to enable git to remove and re-add submodules on checkout. The\n> \"remove a submodule\" part already works in my - still experimental -\n> recursive_submodule_checkout branch at my github repo. The thing that\n> is still missing - apart from tests ;) - is that checkout has to learn\n> to look into the to-be-checked-out .gitmodules to be able to populate\n> a re-appearing submodule. I plan to add that for git fetch first (so\n> it can fetch submodules the user showed interest in but which aren't\n> currently checked out) and after that I'll reuse it for checkout.\n>\n> But that's still some work to do and will take some time ...\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n>\n\nThat's what I figured. I just wished there was at least a way to see that there's a .git directory that does not really belong.\nTo check that now we'd have to do a git-ls tree and look for .git that shouldn't be here but with the amount of file we have, a shell script doing that would take ages !\n\nWe'll wait for a more submodule aware version of checkout then !\n\nThanks\n\nNicolas\n"}]}