{"thread":{"id":"44323","subject":"Integrating submodules with no side effects","startedAt":"2016-10-18T19:35:52Z","lastAt":"2016-10-25T15:05:32Z","messageCount":8,"participants":["Robert Dailey","Stefan Beller"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"304356","messageId":"CAHd499AN2VHL66c6JWxHqS-1bQ6y4PrGjZJiR_ad6HJsCGpeDQ@mail.gmail.com","threadId":"44323","inReplyTo":null,"subject":"Integrating submodules with no side effects","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2016-10-18T19:35:47Z","receivedAt":"2016-10-18T19:35:52Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"Hello git experts,\n\nI have in the past attempted to integrate submodules into my primary\nrepository using the same directory name. However, this has always\ncaused headache when going to and from branches that take you between\nwhen this integration occurred and when it didn't. It's a bit hard to\nexplain. Basically, if I have a submodule \"foo\", and I delete that\nsubmodule and physically add its files under the same directory \"foo\",\nwhen I do a pull to get this change from another clone, it fails\nsaying:\n\nerror: The following untracked working tree files would be overwritten\nby checkout:\n foo/somefile.txt\nPlease move or remove them before you switch branches.\nAborting\ncould not detach HEAD\n\n\nObviously, git can't delete the submodule because the files have also\nbeen added directly. I don't think it is built to handle this\nscenario. Here is the series of commands I ran to \"integrate\" the\nsubmodule (replace the submodule with a directory containing the exact\ncontents of the submodule itself):\n\n#!/usr/bin/env bash\nmv \"$1\" \"${1}_\"\ngit submodule deinit \"$1\"\ngit rm \"$1\"\nmv \"${1}_\" \"$1\"\ngit add \"$1/**\"\n\nThe above script is named git-integrate-submodule, I run it like so:\n\n$ git integrate-submodule foo\n\nThen I do:\n\n$ git commit -m 'Integrated foo submodule'\n\nIs there any way to make this work nicely? The only solution I've\nfound is to obviously rename the directory before adding the physical\nfiles, for example name it foo1. Because they're different, they never\n\"clash\".\n"},{"id":"304364","messageId":"CAGZ79kaFtzQDZrSJhJZ59xvBzn+6+UVDO65Ac+T6aFxMQaM_xQ@mail.gmail.com","threadId":"44323","inReplyTo":"CAHd499AN2VHL66c6JWxHqS-1bQ6y4PrGjZJiR_ad6HJsCGpeDQ@mail.gmail.com","subject":"Re: Integrating submodules with no side effects","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-10-18T21:17:37Z","receivedAt":"2016-10-18T21:17:49Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Oct 18, 2016 at 12:35 PM, Robert Dailey\n wrote:\n> Hello git experts,\n>\n> I have in the past attempted to integrate submodules into my primary\n> repository using the same directory name. However, this has always\n> caused headache when going to and from branches that take you between\n> when this integration occurred and when it didn't. It's a bit hard to\n> explain. Basically, if I have a submodule \"foo\", and I delete that\n> submodule and physically add its files under the same directory \"foo\",\n> when I do a pull to get this change from another clone, it fails\n> saying:\n>\n> error: The following untracked working tree files would be overwritten\n> by checkout:\n> foo/somefile.txt\n> Please move or remove them before you switch branches.\n> Aborting\n> could not detach HEAD\n>\n>\n> Obviously, git can't delete the submodule because the files have also\n> been added directly. I don't think it is built to handle this\n> scenario. Here is the series of commands I ran to \"integrate\" the\n> submodule (replace the submodule with a directory containing the exact\n> contents of the submodule itself):\n>\n> #!/usr/bin/env bash\n> mv \"$1\" \"${1}_\"\n> git submodule deinit \"$1\"\n\nThis removes the submodule entries from .git/config\n(and it would remove the contents of that submodule, but they are moved)\n\n> git rm \"$1\"\n\nRemoving the git link here.\n\nSo we still have the entries in the .gitmodules file there.\nMaybe add:\n\n name=$(git submodule-helper name $1)\n git config -f .gitmodules --unset submodule.$name.*\n git add .gitmodules\n\n? (Could be optional)\n\n> mv \"${1}_\" \"$1\"\n> git add \"$1/**\"\n\nMoving back into place and adding all files in there.\n\n>\n> The above script is named git-integrate-submodule, I run it like so:\n>\n> $ git integrate-submodule foo\n>\n> Then I do:\n>\n> $ git commit -m 'Integrated foo submodule'\n>\n> Is there any way to make this work nicely?\n\nI think you can just remove the gitlink from the index and not from the working\ntree (\"git rm --cached $1\")\n\n> The only solution I've\n> found is to obviously rename the directory before adding the physical\n> files, for example name it foo1. Because they're different, they never\n> \"clash\".\n\nAlso look at the difference between plumbing and porcelain commands[1],\nas plumbing is more stable than the porcelain, so it will be easier to maintain\nthis script.\n\nI think this would be an actually reasonable feature, which Git itself\ncould support via \"git submodule [de]integrate\", but then we'd also want\nto see the reverse, i.e. take a sub directory and make it a submodule.\n\n[1] e.g. https://www.kernel.org/pub/software/scm/git/docs/\n\nThanks,\nStefan\n"},{"id":"304389","messageId":"CAHd499CN3+cHVwjOEirwXVu3DsJwPrmJwEgSJL2CHD5CvoYxxg@mail.gmail.com","threadId":"44323","inReplyTo":"CAGZ79kaFtzQDZrSJhJZ59xvBzn+6+UVDO65Ac+T6aFxMQaM_xQ@mail.gmail.com","subject":"Re: Integrating submodules with no side effects","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2016-10-19T13:27:41Z","receivedAt":"2016-10-19T14:34:31Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Tue, Oct 18, 2016 at 4:17 PM, Stefan Beller wrote:\n> On Tue, Oct 18, 2016 at 12:35 PM, Robert Dailey\n> wrote:\n>> Hello git experts,\n>>\n>> I have in the past attempted to integrate submodules into my primary\n>> repository using the same directory name. However, this has always\n>> caused headache when going to and from branches that take you between\n>> when this integration occurred and when it didn't. It's a bit hard to\n>> explain. Basically, if I have a submodule \"foo\", and I delete that\n>> submodule and physically add its files under the same directory \"foo\",\n>> when I do a pull to get this change from another clone, it fails\n>> saying:\n>>\n>> error: The following untracked working tree files would be overwritten\n>> by checkout:\n>> foo/somefile.txt\n>> Please move or remove them before you switch branches.\n>> Aborting\n>> could not detach HEAD\n>>\n>>\n>> Obviously, git can't delete the submodule because the files have also\n>> been added directly. I don't think it is built to handle this\n>> scenario. Here is the series of commands I ran to \"integrate\" the\n>> submodule (replace the submodule with a directory containing the exact\n>> contents of the submodule itself):\n>>\n>> #!/usr/bin/env bash\n>> mv \"$1\" \"${1}_\"\n>> git submodule deinit \"$1\"\n>\n> This removes the submodule entries from .git/config\n> (and it would remove the contents of that submodule, but they are moved)\n>\n>> git rm \"$1\"\n>\n> Removing the git link here.\n>\n> So we still have the entries in the .gitmodules file there.\n> Maybe add:\n>\n> name=$(git submodule-helper name $1)\n> git config -f .gitmodules --unset submodule.$name.*\n> git add .gitmodules\n>\n> ? (Could be optional)\n\nActually I verified that it seems `git rm` is specialized for\nsubmodules somewhere, because when I run that command on a submodule\nthe relevant entries in the .gitmodules file are removed. I did not\nhave to do this as a separate step.\n\n>> mv \"${1}_\" \"$1\"\n>> git add \"$1/**\"\n>\n> Moving back into place and adding all files in there.\n>\n>>\n>> The above script is named git-integrate-submodule, I run it like so:\n>>\n>> $ git integrate-submodule foo\n>>\n>> Then I do:\n>>\n>> $ git commit -m 'Integrated foo submodule'\n>>\n>> Is there any way to make this work nicely?\n>\n> I think you can just remove the gitlink from the index and not from the working\n> tree (\"git rm --cached $1\")\n\nWhat is the goal of doing it this way? What does this simplify?\n\n>> The only solution I've\n>> found is to obviously rename the directory before adding the physical\n>> files, for example name it foo1. Because they're different, they never\n>> \"clash\".\n>\n> Also look at the difference between plumbing and porcelain commands[1],\n> as plumbing is more stable than the porcelain, so it will be easier to maintain\n> this script.\n\nWhich plumbing commands did you have in mind?\n\n> I think this would be an actually reasonable feature, which Git itself\n> could support via \"git submodule [de]integrate\", but then we'd also want\n> to see the reverse, i.e. take a sub directory and make it a submodule.\n\nIntegrating this as a feature might be fine, I think when you bring up\nthe question of retaining history makes things much harder.\nFortunately for me that is not a requirement in this case, so I'm able\nto do things with much less effort.\n\nHowever the primary purpose of my post was to find out how to\nintegrate the submodule without the error on next pull by other\ncollaborators of my repository. It's a real pain to recover your\nworking copy when going inbetween commits where the submodule\nintegration happened inbetween them. I did quote the exact error\nmessage I got in my original post.\n\nDo you have any info on how I can prevent that error? Ideally I want\nthe integration to go smoothly and transparently, not just for the\nperson doing the actual transition (me) but for everyone else that\ngets those changes from upstream. They should not even notice that it\nhappened (i.e. no failed commands, awkward behavior, or manual steps).\n"},{"id":"304400","messageId":"CAGZ79kYKdHHcfRxCOJwNuqnOGMjBBeMy3Av5U7_ssuOp-to61w@mail.gmail.com","threadId":"44323","inReplyTo":"CAHd499CN3+cHVwjOEirwXVu3DsJwPrmJwEgSJL2CHD5CvoYxxg@mail.gmail.com","subject":"Re: Integrating submodules with no side effects","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-10-19T16:23:26Z","receivedAt":"2016-10-19T16:23:33Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Oct 19, 2016 at 6:27 AM, Robert Dailey wrote:\n> On Tue, Oct 18, 2016 at 4:17 PM, Stefan Beller wrote:\n>> On Tue, Oct 18, 2016 at 12:35 PM, Robert Dailey\n>> wrote:\n>>> Hello git experts,\n>>>\n>>> I have in the past attempted to integrate submodules into my primary\n>>> repository using the same directory name. However, this has always\n>>> caused headache when going to and from branches that take you between\n>>> when this integration occurred and when it didn't. It's a bit hard to\n>>> explain. Basically, if I have a submodule \"foo\", and I delete that\n>>> submodule and physically add its files under the same directory \"foo\",\n>>> when I do a pull to get this change from another clone, it fails\n>>> saying:\n>>>\n>>> error: The following untracked working tree files would be overwritten\n>>> by checkout:\n>>> foo/somefile.txt\n>>> Please move or remove them before you switch branches.\n>>> Aborting\n>>> could not detach HEAD\n>>>\n>>>\n>>> Obviously, git can't delete the submodule because the files have also\n>>> been added directly. I don't think it is built to handle this\n>>> scenario. Here is the series of commands I ran to \"integrate\" the\n>>> submodule (replace the submodule with a directory containing the exact\n>>> contents of the submodule itself):\n>>>\n>>> #!/usr/bin/env bash\n>>> mv \"$1\" \"${1}_\"\n>>> git submodule deinit \"$1\"\n>>\n>> This removes the submodule entries from .git/config\n>> (and it would remove the contents of that submodule, but they are moved)\n>>\n>>> git rm \"$1\"\n>>\n>> Removing the git link here.\n>>\n>> So we still have the entries in the .gitmodules file there.\n>> Maybe add:\n>>\n>> name=$(git submodule-helper name $1)\n>> git config -f .gitmodules --unset submodule.$name.*\n>> git add .gitmodules\n>>\n>> ? (Could be optional)\n>\n> Actually I verified that it seems `git rm` is specialized for\n> submodules somewhere, because when I run that command on a submodule\n> the relevant entries in the .gitmodules file are removed. I did not\n> have to do this as a separate step.\n>\n>>> mv \"${1}_\" \"$1\"\n>>> git add \"$1/**\"\n>>\n>> Moving back into place and adding all files in there.\n>>\n>>>\n>>> The above script is named git-integrate-submodule, I run it like so:\n>>>\n>>> $ git integrate-submodule foo\n>>>\n>>> Then I do:\n>>>\n>>> $ git commit -m 'Integrated foo submodule'\n>>>\n>>> Is there any way to make this work nicely?\n>>\n>> I think you can just remove the gitlink from the index and not from the working\n>> tree (\"git rm --cached $1\")\n>\n> What is the goal of doing it this way? What does this simplify?\n\nYou don't have to mv it back and forth with an underscore I would imagine?\n\n>\n>>> The only solution I've\n>>> found is to obviously rename the directory before adding the physical\n>>> files, for example name it foo1. Because they're different, they never\n>>> \"clash\".\n>>\n>> Also look at the difference between plumbing and porcelain commands[1],\n>> as plumbing is more stable than the porcelain, so it will be easier to maintain\n>> this script.\n>\n> Which plumbing commands did you have in mind?\n\nNone specifically. I write scripts using porcelain all the time for\npersonal use.\nBut if you were planning to publish this seriously then I'd recommend looking at\nplumbing commands.\n\n>\n>> I think this would be an actually reasonable feature, which Git itself\n>> could support via \"git submodule [de]integrate\", but then we'd also want\n>> to see the reverse, i.e. take a sub directory and make it a submodule.\n>\n> Integrating this as a feature might be fine, I think when you bring up\n> the question of retaining history makes things much harder.\n> Fortunately for me that is not a requirement in this case, so I'm able\n> to do things with much less effort.\n\nThat reminds me of subtree merging, which could be used for this case.\n(see 'git subtree')\n\n>\n> However the primary purpose of my post was to find out how to\n> integrate the submodule without the error on next pull by other\n> collaborators of my repository. It's a real pain to recover your\n> working copy when going inbetween commits where the submodule\n> integration happened inbetween them. I did quote the exact error\n> message I got in my original post.\n\nYou could try this patch series:\nhttps://github.com/jlehmann/git-submod-enhancements/tree/git-checkout-recurse-submodules\n(rebased to a newer version; no functional changes:)\nhttps://github.com/stefanbeller/git/tree/submodule-co\n(I'll rebase that later to origin/master)\n\n>\n> Do you have any info on how I can prevent that error? Ideally I want\n> the integration to go smoothly and transparently, not just for the\n> person doing the actual transition (me) but for everyone else that\n> gets those changes from upstream. They should not even notice that it\n> happened (i.e. no failed commands, awkward behavior, or manual steps).\n\nIt depends on how long you want to postpone the transition, but I plan to\nupstream the series referenced above in the near future,\nwhich would enable your situation to Just Work (tm). ;)\n"},{"id":"304408","messageId":"CAHd499D_fehBHsNdH9PLQj+H_WAJqbbMhDwv_nCT3fZcX60ENA@mail.gmail.com","threadId":"44323","inReplyTo":"CAGZ79kYKdHHcfRxCOJwNuqnOGMjBBeMy3Av5U7_ssuOp-to61w@mail.gmail.com","subject":"Re: Integrating submodules with no side effects","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2016-10-19T19:19:29Z","receivedAt":"2016-10-19T19:19:34Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Wed, Oct 19, 2016 at 11:23 AM, Stefan Beller wrote:\n> You could try this patch series:\n> https://github.com/jlehmann/git-submod-enhancements/tree/git-checkout-recurse-submodules\n> (rebased to a newer version; no functional changes:)\n> https://github.com/stefanbeller/git/tree/submodule-co\n> (I'll rebase that later to origin/master)\n>\n>>\n>> Do you have any info on how I can prevent that error? Ideally I want\n>> the integration to go smoothly and transparently, not just for the\n>> person doing the actual transition (me) but for everyone else that\n>> gets those changes from upstream. They should not even notice that it\n>> happened (i.e. no failed commands, awkward behavior, or manual steps).\n>\n> It depends on how long you want to postpone the transition, but I plan to\n> upstream the series referenced above in the near future,\n> which would enable your situation to Just Work (tm). ;)\n\nAt first glance, what you've linked to essentially looks like\nautomated `git submodule update` for every `git checkout`. Am I\nmisunderstanding?\n\nIf I'm correct, this is not the same as what I'm talking about. The\nproblem appears to be more internal: When a submodule is removed, the\nphysical files that were there are not removed by Git. It leaves them\nthere in the working copy as untracked files. The next step Git takes\n(again, just from outside observation) is to add those very same files\nto the working copy, since they were added to a commit. However, at\nthis point Git fails because it's trying to create (write) files to\nthe working copy when an exact file of that name already exists there.\nGit will not overwrite untracked files, so at this point it fails.\n\nWhat needs to happen, somehow, is Git sees that the files were\nactually part of a submodule (which was removed) and remove the\nphysical files as well, assuming that they were not modified in the\nsubmodule itself. This will ensure that the next step (creating the\nfiles) will succeed since the files no longer block it.\n"},{"id":"304410","messageId":"CAGZ79kZWaShG9iG565bD3aVYFcb53+opbpnJH-v3G6hrQL2tzA@mail.gmail.com","threadId":"44323","inReplyTo":"CAHd499D_fehBHsNdH9PLQj+H_WAJqbbMhDwv_nCT3fZcX60ENA@mail.gmail.com","subject":"Re: Integrating submodules with no side effects","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-10-19T19:45:50Z","receivedAt":"2016-10-19T19:45:58Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Oct 19, 2016 at 12:19 PM, Robert Dailey\n wrote:\n> On Wed, Oct 19, 2016 at 11:23 AM, Stefan Beller wrote:\n>> You could try this patch series:\n>> https://github.com/jlehmann/git-submod-enhancements/tree/git-checkout-recurse-submodules\n>> (rebased to a newer version; no functional changes:)\n>> https://github.com/stefanbeller/git/tree/submodule-co\n>> (I'll rebase that later to origin/master)\n>>\n>>>\n>>> Do you have any info on how I can prevent that error? Ideally I want\n>>> the integration to go smoothly and transparently, not just for the\n>>> person doing the actual transition (me) but for everyone else that\n>>> gets those changes from upstream. They should not even notice that it\n>>> happened (i.e. no failed commands, awkward behavior, or manual steps).\n>>\n>> It depends on how long you want to postpone the transition, but I plan to\n>> upstream the series referenced above in the near future,\n>> which would enable your situation to Just Work (tm). ;)\n>\n> At first glance, what you've linked to essentially looks like\n> automated `git submodule update` for every `git checkout`. Am I\n> misunderstanding?\n\nEssentially yes, except with stricter rules than the actual submodule update\nIIRC.\n\n>\n> If I'm correct, this is not the same as what I'm talking about. The\n> problem appears to be more internal: When a submodule is removed, the\n> physical files that were there are not removed by Git.\n\nThat is also done by that series: submodules ought to be treated as files:\nIf you checkout a new version where a file is deleted, the checkout command\nwill actually remove the file for you (and e.g. solve any\ndirectory/file conflicts\nthat may happen in the transition.)\n\n> It leaves them\n> there in the working copy as untracked files.\n\nThat is the current behavior as checkout tries hard to ignore submodules.\n\n> The next step Git takes\n> (again, just from outside observation) is to add those very same files\n> to the working copy, since they were added to a commit. However, at\n> this point Git fails because it's trying to create (write) files to\n> the working copy when an exact file of that name already exists there.\n> Git will not overwrite untracked files, so at this point it fails.\n>\n> What needs to happen, somehow, is Git sees that the files were\n> actually part of a submodule (which was removed) and remove the\n> physical files as well, assuming that they were not modified in the\n> submodule itself. This will ensure that the next step (creating the\n> files) will succeed since the files no longer block it.\n\nYep.\n"},{"id":"304411","messageId":"CAHd499DKz+tpP3zRrXX3_serhoC_GCZst2y75JtC0Eiy1zfEew@mail.gmail.com","threadId":"44323","inReplyTo":"CAGZ79kZWaShG9iG565bD3aVYFcb53+opbpnJH-v3G6hrQL2tzA@mail.gmail.com","subject":"Re: Integrating submodules with no side effects","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2016-10-19T19:51:52Z","receivedAt":"2016-10-19T19:51:59Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Wed, Oct 19, 2016 at 2:45 PM, Stefan Beller wrote:\n> On Wed, Oct 19, 2016 at 12:19 PM, Robert Dailey\n> wrote:\n>> On Wed, Oct 19, 2016 at 11:23 AM, Stefan Beller wrote:\n>>> You could try this patch series:\n>>> https://github.com/jlehmann/git-submod-enhancements/tree/git-checkout-recurse-submodules\n>>> (rebased to a newer version; no functional changes:)\n>>> https://github.com/stefanbeller/git/tree/submodule-co\n>>> (I'll rebase that later to origin/master)\n>>>\n>>>>\n>>>> Do you have any info on how I can prevent that error? Ideally I want\n>>>> the integration to go smoothly and transparently, not just for the\n>>>> person doing the actual transition (me) but for everyone else that\n>>>> gets those changes from upstream. They should not even notice that it\n>>>> happened (i.e. no failed commands, awkward behavior, or manual steps).\n>>>\n>>> It depends on how long you want to postpone the transition, but I plan to\n>>> upstream the series referenced above in the near future,\n>>> which would enable your situation to Just Work (tm). ;)\n>>\n>> At first glance, what you've linked to essentially looks like\n>> automated `git submodule update` for every `git checkout`. Am I\n>> misunderstanding?\n>\n> Essentially yes, except with stricter rules than the actual submodule update\n> IIRC.\n>\n>>\n>> If I'm correct, this is not the same as what I'm talking about. The\n>> problem appears to be more internal: When a submodule is removed, the\n>> physical files that were there are not removed by Git.\n>\n> That is also done by that series: submodules ought to be treated as files:\n> If you checkout a new version where a file is deleted, the checkout command\n> will actually remove the file for you (and e.g. solve any\n> directory/file conflicts\n> that may happen in the transition.)\n>\n>> It leaves them\n>> there in the working copy as untracked files.\n>\n> That is the current behavior as checkout tries hard to ignore submodules.\n>\n>> The next step Git takes\n>> (again, just from outside observation) is to add those very same files\n>> to the working copy, since they were added to a commit. However, at\n>> this point Git fails because it's trying to create (write) files to\n>> the working copy when an exact file of that name already exists there.\n>> Git will not overwrite untracked files, so at this point it fails.\n>>\n>> What needs to happen, somehow, is Git sees that the files were\n>> actually part of a submodule (which was removed) and remove the\n>> physical files as well, assuming that they were not modified in the\n>> submodule itself. This will ensure that the next step (creating the\n>> files) will succeed since the files no longer block it.\n>\n> Yep.\n\nIt's great we're finally on the same page ;-)\n\nHowever, I don't see how this problem can be solved with your script,\nor solved in general outside of that. Does this mean that Git needs to\nchange to treat submodules as it does normal files, per your previous\nassertion, which means submodules should *not* be left behind in the\nworking copy as untracked files?\n"},{"id":"304844","messageId":"CAHd499B605aWFKJa1DmuEcB9WR5pubeURk8ToTrnJP7saTG2Kg@mail.gmail.com","threadId":"44323","inReplyTo":"CAHd499DKz+tpP3zRrXX3_serhoC_GCZst2y75JtC0Eiy1zfEew@mail.gmail.com","subject":"Re: Integrating submodules with no side effects","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2016-10-25T15:05:23Z","receivedAt":"2016-10-25T15:05:32Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Wed, Oct 19, 2016 at 2:51 PM, Robert Dailey wrote:\n> On Wed, Oct 19, 2016 at 2:45 PM, Stefan Beller wrote:\n>> On Wed, Oct 19, 2016 at 12:19 PM, Robert Dailey\n>> wrote:\n>>> On Wed, Oct 19, 2016 at 11:23 AM, Stefan Beller wrote:\n>>>> You could try this patch series:\n>>>> https://github.com/jlehmann/git-submod-enhancements/tree/git-checkout-recurse-submodules\n>>>> (rebased to a newer version; no functional changes:)\n>>>> https://github.com/stefanbeller/git/tree/submodule-co\n>>>> (I'll rebase that later to origin/master)\n>>>>\n>>>>>\n>>>>> Do you have any info on how I can prevent that error? Ideally I want\n>>>>> the integration to go smoothly and transparently, not just for the\n>>>>> person doing the actual transition (me) but for everyone else that\n>>>>> gets those changes from upstream. They should not even notice that it\n>>>>> happened (i.e. no failed commands, awkward behavior, or manual steps).\n>>>>\n>>>> It depends on how long you want to postpone the transition, but I plan to\n>>>> upstream the series referenced above in the near future,\n>>>> which would enable your situation to Just Work (tm). ;)\n>>>\n>>> At first glance, what you've linked to essentially looks like\n>>> automated `git submodule update` for every `git checkout`. Am I\n>>> misunderstanding?\n>>\n>> Essentially yes, except with stricter rules than the actual submodule update\n>> IIRC.\n>>\n>>>\n>>> If I'm correct, this is not the same as what I'm talking about. The\n>>> problem appears to be more internal: When a submodule is removed, the\n>>> physical files that were there are not removed by Git.\n>>\n>> That is also done by that series: submodules ought to be treated as files:\n>> If you checkout a new version where a file is deleted, the checkout command\n>> will actually remove the file for you (and e.g. solve any\n>> directory/file conflicts\n>> that may happen in the transition.)\n>>\n>>> It leaves them\n>>> there in the working copy as untracked files.\n>>\n>> That is the current behavior as checkout tries hard to ignore submodules.\n>>\n>>> The next step Git takes\n>>> (again, just from outside observation) is to add those very same files\n>>> to the working copy, since they were added to a commit. However, at\n>>> this point Git fails because it's trying to create (write) files to\n>>> the working copy when an exact file of that name already exists there.\n>>> Git will not overwrite untracked files, so at this point it fails.\n>>>\n>>> What needs to happen, somehow, is Git sees that the files were\n>>> actually part of a submodule (which was removed) and remove the\n>>> physical files as well, assuming that they were not modified in the\n>>> submodule itself. This will ensure that the next step (creating the\n>>> files) will succeed since the files no longer block it.\n>>\n>> Yep.\n>\n> It's great we're finally on the same page ;-)\n>\n> However, I don't see how this problem can be solved with your script,\n> or solved in general outside of that. Does this mean that Git needs to\n> change to treat submodules as it does normal files, per your previous\n> assertion, which means submodules should *not* be left behind in the\n> working copy as untracked files?\n\nI'll assume (due to the lack of responses) that the only viable\nsolution here is to integrate the submodule using a different\ndirectory name than the one used by the submodule itself. It's\nunfortunate but I'll do it if I have no other option.\n"}]}