{"thread":{"id":"47591","subject":"Unable to de-init stubborn submodule","startedAt":"2018-01-11T14:12:22Z","lastAt":"2018-01-11T18:17:49Z","messageCount":3,"participants":["Ævar Arnfjörð Bjarmason","Brandon Williams","Stefan Beller"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"336460","messageId":"87k1woa3i9.fsf@evledraar.gmail.com","threadId":"47591","inReplyTo":null,"subject":"Unable to de-init stubborn submodule","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-01-11T14:12:14Z","receivedAt":"2018-01-11T14:12:22Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"What if we wanted to drop sha1collisiondetection/ as a submodule and\nreplace it with a copy of what's now in sha1dc/? I ran into this with\nanother project, but here's a way to reproduce it on git.git:\n\n    (\n        rm -rf /tmp/git &&\n        git clone git@github.com:git/git.git /tmp/git\n        cd /tmp/git &&\n        git tag nuke-before &&\n        git submodule update --init &&\n        git rm -r .gitmodules sha1collisiondetection &&\n        git commit -m\"Nuke sha1dc submodule\" &&\n        cp -Rvp sha1dc sha1collisiondetection &&\n        git add sha1collisiondetection &&\n        git commit -m\"Now it's not a submodule\" &&\n        git tag nuke-after &&\n        git reset --hard nuke-before &&\n        git submodule update --init && # skip this and the below won't fail\n        git reset --hard nuke-after && # Emulate someone doing a pull\n        git ls-tree HEAD | grep sha1collisiondetection && # OK, shows \"tree\" not \"commit\"\n        test $(git rev-parse HEAD) == $(git -C sha1collisiondetection/ log -1 --pretty=format:%H) && echo OK || echo WTF\n    )\n\nThis results in a really bizarre state where according to ls-tree\nsha1collisiondetection is a tree at the current commit:\n\n    040000 tree 81583289d96bdde4b366c243ab524ea28d895ea5    sha1collisiondetection\n\nBut git still believes there's a submodule there for some reason, and\nshows the log for the upstream sha1collisiondetection project:\n\n    git -C sha1collisiondetection/ log -1\n    commit 19d97bf (HEAD, origin/master, origin/HEAD, master)\n    Merge: 3f14d1b c93f0b4\n    Author: Dan Shumow <shumow@gmail.com>\n    Date:   Sat Jul 1 12:36:15 2017 -0700\n\n        Merge pull request #37 from avar/fixup-pull-request-34\n\n        Fix endian detection logic for Sparc, little endian BSD etc.\n\nDoing:\n\n    git submodule deinit sha1collisiondetection\n\nDoes nothing to help, then I thought it might be:\n\n    git config -f .git/config -l|grep ^submodule\n    submodule.sha1collisiondetection.active=true\n    submodule.sha1collisiondetection.url=https://github.com/cr-marcstevens/sha1collisiondetection.git\n\nBut running:\n\n    git config --remove-section submodule.sha1collisiondetection\n\nDoesn't help either, neither does removing the index:\n\n    rm .git/index &&\n    git reset --hard\n\nIf you then do:\n\n    rm -rf .git/modules\n\nYou'll get this error:\n\n    git -C sha1collisiondetection/ log -1\n    fatal: Not a git repository: /tmp/git/sha1collisiondetection/../.git/modules/sha1collisiondetection\n\nBut I can't see what's still referencing it.\n\nThis problem is avoided if, as noted with a comment I skip:\n\n    git submodule update --init\n\nBut I shouldn't need to remember to de-init a submodule before moving to\na new commit that doesn't have it, least I end up in some seemingly\nunrecoverable state.\n\nAm I missing something obvious here?\n"},{"id":"336463","messageId":"20180111175035.GC91567@google.com","threadId":"47591","inReplyTo":"87k1woa3i9.fsf@evledraar.gmail.com","subject":"Re: Unable to de-init stubborn submodule","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-01-11T17:50:35Z","receivedAt":"2018-01-11T17:50:46Z","isPatch":false,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 01/11, Ævar Arnfjörð Bjarmason wrote:\n> What if we wanted to drop sha1collisiondetection/ as a submodule and\n> replace it with a copy of what's now in sha1dc/? I ran into this with\n> another project, but here's a way to reproduce it on git.git:\n> \n>     (\n>         rm -rf /tmp/git &&\n>         git clone git@github.com:git/git.git /tmp/git\n>         cd /tmp/git &&\n>         git tag nuke-before &&\n>         git submodule update --init &&\n>         git rm -r .gitmodules sha1collisiondetection &&\n>         git commit -m\"Nuke sha1dc submodule\" &&\n>         cp -Rvp sha1dc sha1collisiondetection &&\n>         git add sha1collisiondetection &&\n>         git commit -m\"Now it's not a submodule\" &&\n>         git tag nuke-after &&\n>         git reset --hard nuke-before &&\n>         git submodule update --init && # skip this and the below won't fail\n>         git reset --hard nuke-after && # Emulate someone doing a pull\n>         git ls-tree HEAD | grep sha1collisiondetection && # OK, shows \"tree\" not \"commit\"\n>         test $(git rev-parse HEAD) == $(git -C sha1collisiondetection/ log -1 --pretty=format:%H) && echo OK || echo WTF\n>     )\n> \n> This results in a really bizarre state where according to ls-tree\n> sha1collisiondetection is a tree at the current commit:\n> \n>     040000 tree 81583289d96bdde4b366c243ab524ea28d895ea5    sha1collisiondetection\n> \n> But git still believes there's a submodule there for some reason, and\n> shows the log for the upstream sha1collisiondetection project:\n> \n>     git -C sha1collisiondetection/ log -1\n>     commit 19d97bf (HEAD, origin/master, origin/HEAD, master)\n>     Merge: 3f14d1b c93f0b4\n>     Author: Dan Shumow <shumow@gmail.com>\n>     Date:   Sat Jul 1 12:36:15 2017 -0700\n> \n>         Merge pull request #37 from avar/fixup-pull-request-34\n> \n>         Fix endian detection logic for Sparc, little endian BSD etc.\n> \n> Doing:\n> \n>     git submodule deinit sha1collisiondetection\n> \n> Does nothing to help, then I thought it might be:\n> \n>     git config -f .git/config -l|grep ^submodule\n>     submodule.sha1collisiondetection.active=true\n>     submodule.sha1collisiondetection.url=https://github.com/cr-marcstevens/sha1collisiondetection.git\n> \n> But running:\n> \n>     git config --remove-section submodule.sha1collisiondetection\n> \n> Doesn't help either, neither does removing the index:\n> \n>     rm .git/index &&\n>     git reset --hard\n> \n> If you then do:\n> \n>     rm -rf .git/modules\n> \n> You'll get this error:\n> \n>     git -C sha1collisiondetection/ log -1\n>     fatal: Not a git repository: /tmp/git/sha1collisiondetection/../.git/modules/sha1collisiondetection\n> \n> But I can't see what's still referencing it.\n> \n> This problem is avoided if, as noted with a comment I skip:\n> \n>     git submodule update --init\n> \n> But I shouldn't need to remember to de-init a submodule before moving to\n> a new commit that doesn't have it, least I end up in some seemingly\n> unrecoverable state.\n> \n> Am I missing something obvious here?\n\nOne thing you could try is adding --recurse-submodule flags to the reset\ncommands.  IIRC reset ignores submodules unless you specify that flag.\n\n-- \nBrandon Williams\n"},{"id":"336466","messageId":"CAGZ79kajv9Pf4qc_1wgDD=NTimT1Ss7Wz-8fNNbmkgLO40VVGg@mail.gmail.com","threadId":"47591","inReplyTo":"87k1woa3i9.fsf@evledraar.gmail.com","subject":"Re: Unable to de-init stubborn submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-01-11T18:17:40Z","receivedAt":"2018-01-11T18:17:49Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Jan 11, 2018 at 6:12 AM, Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n> What if we wanted to drop sha1collisiondetection/ as a submodule and\n> replace it with a copy of what's now in sha1dc/? I ran into this with\n> another project, but here's a way to reproduce it on git.git:\n>\n>     (\n>         rm -rf /tmp/git &&\n>         git clone git@github.com:git/git.git /tmp/git\n>         cd /tmp/git &&\n>         git tag nuke-before &&\n>         git submodule update --init &&\n\nAt this point\n    $ tail -n 3 .git/config\n    [submodule \"sha1collisiondetection\"]\n        active = true\n        url = https://github.com/cr-marcstevens/sha1collisiondetection.git\n\n\n>         git rm -r .gitmodules sha1collisiondetection &&\n\nno need to delete the .gitmodules here. Git detects you're deleting a submodule\nand adjusts the .gitmodules file (it is empty after just \"git rm sha1coll...\")\n\n>         git commit -m\"Nuke sha1dc submodule\" &&\n>         cp -Rvp sha1dc sha1collisiondetection &&\n>         git add sha1collisiondetection &&\n>         git commit -m\"Now it's not a submodule\" &&\n>         git tag nuke-after &&\n>         git reset --hard nuke-before &&\n\nAs bmwill said, you may want to reset with --recurse-submodules here,\n\"git ls-tree HEAD |grep sha1c\" will show commit and \"git status\" thinks\neverything is fine on disk. I have no suspicion to believe otherwise.\nBut we digress, you chose to not use that flag.\n\n>         git submodule update --init && # skip this and the below won't fail\n>         git reset --hard nuke-after && # Emulate someone doing a pull\n\n$ git reset --hard nuke-after\nwarning: unable to rmdir 'sha1collisiondetection': Directory not empty\nHEAD is now at f683a1b81 Now it's not a submodule\n$ git reset --hard nuke-after --recurse-submodules\nHEAD is now at f683a1b81 Now it's not a submodule\n\n\n>         git ls-tree HEAD | grep sha1collisiondetection && # OK, shows \"tree\" not \"commit\"\n>         test $(git rev-parse HEAD) == $(git -C sha1collisiondetection/ log -1 --pretty=format:%H) && echo OK || echo WTF\n>     )\n>\n> This results in a really bizarre state where according to ls-tree\n> sha1collisiondetection is a tree at the current commit:\n>\n>     040000 tree 81583289d96bdde4b366c243ab524ea28d895ea5    sha1collisiondetection\n>\n> But git still believes there's a submodule there for some reason, and\n> shows the log for the upstream sha1collisiondetection project:\n>\n>     git -C sha1collisiondetection/ log -1\n\n(A)\nThat is because the reset without flag do touch submodules kept the\nsubmodule in place and the -C in this command tells git to cd into that\ndirectory, (which is the submodule, an 'independent' repo) and shows the\nlog of said repo.\n\n>     commit 19d97bf (HEAD, origin/master, origin/HEAD, master)\n>     Merge: 3f14d1b c93f0b4\n>     Author: Dan Shumow <shumow@gmail.com>\n>     Date:   Sat Jul 1 12:36:15 2017 -0700\n>\n>         Merge pull request #37 from avar/fixup-pull-request-34\n>\n>         Fix endian detection logic for Sparc, little endian BSD etc.\n>\n> Doing:\n>\n>     git submodule deinit sha1collisiondetection\n\nDoing this after you reset to nuke-after, there are no submodules from\nthe superprojects point of view, hence no submodule is touched/modified. :/\n(There just happens to be a stray repository at a place where we'd want\nto have a tree).\n\n> Does nothing to help, then I thought it might be:\n>\n>     git config -f .git/config -l|grep ^submodule\n>     submodule.sha1collisiondetection.active=true\n>     submodule.sha1collisiondetection.url=https://github.com/cr-marcstevens/sha1collisiondetection.git\n>\n> But running:\n>\n>     git config --remove-section submodule.sha1collisiondetection\n\nThis made the submodule not active any more, (note that at the current\ntree there\nis no submodule to begin with... so what effect to we want here?)\n\n> Doesn't help either, neither does removing the index:\n>\n>     rm .git/index &&\n>     git reset --hard\n>\n> If you then do:\n>\n>     rm -rf .git/modules\n\nGetting out the big hammer, eh?\n\ninstead of deleting the submodules git repo, remove its worktree:\n\n    rm -rf sha1collisiondetection\n\n(This is best done at time (A) above)\n\n>\n> You'll get this error:\n>\n>     git -C sha1collisiondetection/ log -1\n>     fatal: Not a git repository: /tmp/git/sha1collisiondetection/../.git/modules/sha1collisiondetection\n>\n> But I can't see what's still referencing it.\n\n$ls -la /tmp/git/sha1collisiondetection\n...\n-rw-r--r--  1 sbeller eng    47 Jan 11 10:02 .git\n...\n$ cat sha1collisiondetection/.git\ngitdir: ../.git/modules/sha1collisiondetection\n\nThis is a crucial file, telling you there is a repo with the worktree\nat the place where\n.git lives and the git dir is at the location indicated (which you removed)\n\n> This problem is avoided if, as noted with a comment I skip:\n>\n>     git submodule update --init\n>\n> But I shouldn't need to remember to de-init a submodule before moving to\n> a new commit that doesn't have it, least I end up in some seemingly\n> unrecoverable state.\n>\n> Am I missing something obvious here?\n\nThe \"git submodule update/init/deinit\" is a \"raw\" command,\n(not to be confused with plumbing, I just made up \"raw commands\" ;)\nthat is very good to deal with submodules, but that tool has sharp edges.\nTry this:\n\n  $ git clone --recurse-submodules=. git@github.com:git/git.git testgit\n  $ cd testgit/\n  $ git config --list |grep submod\nsubmodule.active=.\nsubmodule.sha1collisiondetection.url=https://github.com/cr-marcstevens/sha1collisiondetection.git\n  $ git config submodule.recurse true\n  $ git tag before-nuke\n  $ git rm sha1collisiondetection/\n  rm 'sha1collisiondetection'\n  $ git commit -a -m delete_sub\n  $ git tag after-nuke\n  $ git checkout before-nuke\n  $ git status # and more checks\n  $ git checkout after-nuke\n  $ git status # and more checks\n"}]}