{"thread":{"id":"49961","subject":"[wishlist] git submodule update --reset-hard","startedAt":"2018-12-06T18:02:10Z","lastAt":"2018-12-14T04:22:13Z","messageCount":18,"participants":["Yaroslav Halchenko","Stefan Beller","Yaroslav O Halchenko"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"364694","messageId":"20181206173554.GH4633@hopa.kiewit.dartmouth.edu","threadId":"49961","inReplyTo":null,"subject":"[wishlist] git submodule update --reset-hard","fromName":"Yaroslav Halchenko","fromEmail":"yoh@onerussian.com","sentAt":"2018-12-06T17:35:54Z","receivedAt":"2018-12-06T18:02:10Z","isPatch":false,"sender":{"key":"yoh@onerussian.com","avatar":"https://gravatar.com/avatar/8901b82415ae451a83aea49409708912726e53620e3ac92320bf1f86548d97e9?d=mp&s=160"},"body":"Dear Git Gurus,\n\nI wondered what would be your take on my wishlist request to add\n--reset-hard option, which would be very similar to regular \"update\" which\nchecks out necessary commit, but I want it to remain in the branch.\n\nRationale: In DataLad we heavily rely on submodules, and we have established\neasy ways to do some manipulations across full hierarchies of them. E.g. a\nsingle command could introduce a good number of commits across deep hierarchy\nof submodules, e.g. while committing changes within deep submodule, while also\ndoing all necessary commits in the repositories leading to that submodule so\nthe entire tree of them stays in a \"clean\" state. The difficulty comes when\nthere is a need to just \"forget\" some changes. The obvious way is to e.g. \n\n git reset --hard PREVIOUS_STATE\n\nin the top level repository. But that leaves all the submodules now in\nthe undesired state. If I do\n\n git submodule update --recursive\n\nI would end up in the detached HEADs within submodules. \n\nWhat I want is to retain current branch they are at (or may be possible\n\"were in\"? reflog records might have that information)\n\nExample:\n\n# Have to use datalad install since git clone --recurse-submodules\n# seems to not consider alternative locations for submodules' .git/\n# with url being just a relative path, and where submodules aren't \n# all residing up under toplevel URL .git/\n\n\t$> datalad install -r http://datasets.datalad.org/labs/gobbini/.git\n\t[INFO ] Cloning http://datasets.datalad.org/labs/gobbini/.git into '/tmp/gobbini' \n\tinstall(ok): /tmp/gobbini (dataset) \n\t[INFO ] Installing recursively \n\t[INFO ] Cloning http://datasets.datalad.org/labs/gobbini/famface/.git into '/tmp/gobbini/famface' \n\t[INFO ] Cloning http://datasets.datalad.org/labs/gobbini/famface/data/.git into '/tmp/gobbini/famface/data' \n\t[INFO ] access to dataset sibling \"datasets.datalad.org\" not auto-enabled, enable with: \n\t| \t\tdatalad siblings -d \"/tmp/gobbini/famface/data\" enable -s datasets.datalad.org \n\t[INFO ] Cloning http://datasets.datalad.org/labs/gobbini/famface/data/scripts/mridefacer/.git [2 other candidates] into '/tmp/gobbini/famface/data/scripts/mridefacer' \n\taction summary: \n\t install (ok: 4)\n\nso I have a hierarchy in a good state and all checked out in master\nbranch\n\n\t$> cd gobbini\n\n\t$> git submodule status --recursive \n\t b9071a6bc9f7665f7c75549c63d29f16d40e8af7 famface (heads/master)\n\t e59ba76b42f219bdf14b6b547dd6d9cc0ed5227f famface/data (BIDS-v1.0.1-3-ge59ba76b)\n\t 5d8036c0aaeebb448a00df6296ddc9f799efdd1f famface/data/scripts/mridefacer (heads/master)\n\n\t$> git submodule foreach --recursive cat .git/HEAD \n\tEntering 'famface'\n\tref: refs/heads/master\n\tEntering 'famface/data'\n\tref: refs/heads/master\n\tEntering 'famface/data/scripts/mridefacer'\n\tref: refs/heads/master\n\n\nand if I do roll back\n\n\t$> git reset --hard HEAD^^^ \n\tHEAD is now at 9b4296d [DATALAD] aggregated meta data\n\tchanges on filesystem: \n\t famface | 2 +-\n\nand default update --recursive\n\n\t$> git submodule update --recursive\n\tSubmodule path 'famface': checked out '2569ab436501a832d35afbbe9cc20ffeb6077eb1'\n\tSubmodule path 'famface/data': checked out 'f1e8c9b8b025c311424283b9711efc6bc906ba2b'\n\tSubmodule path 'famface/data/scripts/mridefacer': checked out '49b0fe42696724c2a8492f999736056e51b77358'\n\nI end up in detached HEADs\n\n\t$> git submodule status --recursive \n\t 2569ab436501a832d35afbbe9cc20ffeb6077eb1 famface (2569ab4)\n\t f1e8c9b8b025c311424283b9711efc6bc906ba2b famface/data (BIDS-v1.0.1)\n\t 49b0fe42696724c2a8492f999736056e51b77358 famface/data/scripts/mridefacer (49b0fe4)\n\n\nI do see that there is a \"custom command\" way to do it via\n\"submodule..update\" config setting, but that is not easy to use for my\ncase since all the `` would be different to specify !git reset --hard for\nall of them via config option and I could not find any way to \"glob\" config\n(like submodule.*.update). But in effect that is probably what I need:\n\n\t# restarting from a clean state here\n\t$> git -c submodule.famface.update='!git reset --hard' submodule update --recursive \n\tHEAD is now at 2569ab4 [DATALAD] aggregated meta data\n\tSubmodule path 'famface': 'git reset --hard 2569ab436501a832d35afbbe9cc20ffeb6077eb1'\n\tSubmodule path 'famface/data': checked out 'f1e8c9b8b025c311424283b9711efc6bc906ba2b'\n\tSubmodule path 'famface/data/scripts/mridefacer': checked out '49b0fe42696724c2a8492f999736056e51b77358'\n\n\t$> git submodule status --recursive \n\t 2569ab436501a832d35afbbe9cc20ffeb6077eb1 famface (heads/master)\n\t f1e8c9b8b025c311424283b9711efc6bc906ba2b famface/data (BIDS-v1.0.1)\n\t 49b0fe42696724c2a8492f999736056e51b77358 famface/data/scripts/mridefacer (49b0fe4)\n\n\nCorner cases I see which might make it trickier for a full blown\nsolution (might be relevant to current state as well for other\nstrategies):\n\n- If between those commits we got an additional submodule added (in\n immediate repository or within one of the subdatasets), ideally it\n should also be wiped out\n\n-- \nYaroslav O. Halchenko\nCenter for Open Neuroscience http://centerforopenneuroscience.org\nDartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755\nPhone: +1 (603) 646-9834 Fax: +1 (603) 646-1419\nWWW: http://www.linkedin.com/in/yarik \n"},{"id":"364697","messageId":"CAGZ79kY8uv8zDm3f8Jb6aC-nit7OZduixyOekGYWa_xnqFqw-w@mail.gmail.com","threadId":"49961","inReplyTo":"20181206173554.GH4633@hopa.kiewit.dartmouth.edu","subject":"Re: [wishlist] git submodule update --reset-hard","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-12-06T18:29:28Z","receivedAt":"2018-12-06T18:29:44Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Dec 6, 2018 at 10:02 AM Yaroslav Halchenko wrote:\n>\n> Dear Git Gurus,\n>\n> I wondered what would be your take on my wishlist request to add\n> --reset-hard option, which would be very similar to regular \"update\" which\n> checks out necessary commit, but I want it to remain in the branch.\n\nWhat if the branch differs from the sha1 recorded in the superproject?\n\n> Rationale: In DataLad we heavily rely on submodules, and we have established\n> easy ways to do some manipulations across full hierarchies of them. E.g. a\n> single command could introduce a good number of commits across deep hierarchy\n> of submodules, e.g. while committing changes within deep submodule, while also\n> doing all necessary commits in the repositories leading to that submodule so\n> the entire tree of them stays in a \"clean\" state. The difficulty comes when\n> there is a need to just \"forget\" some changes. The obvious way is to e.g.\n>\n> git reset --hard PREVIOUS_STATE\n\n git reset --hard --recurse-submodules HEAD\n\nwould do the trick\n\n> in the top level repository. But that leaves all the submodules now in\n> the undesired state. If I do\n\nundesirable in the sense of still having local changes (that is what\nthe above reset with `--recurse` would fix) or changed the branch\nstate? (i.e. is detached but was on a branch before?)\n\n> git submodule update --recursive\n>\n> I would end up in the detached HEADs within submodules.\n>\n> What I want is to retain current branch they are at (or may be possible\n> \"were in\"? reflog records might have that information)\n\nSo something like\n\n git submodule foreach --recursive git reset --hard\n\n?\n\nYou may be interested in\nhttps://public-inbox.org/git/20180927221603.148025-1-sbeller@google.com/\nwhich introduces a switch `submodule.repoLike [ = true]`, which\nwhen set would not detach HEAD in submodules.\n\nCan you say more about the first question above:\nWould you typically have situations where the\nsubmodule branch is out of sync with the superproject\nand how do you deal with that?\n\nAdding another mode to `git submodule update` sounds\nreasonable to me, too.\n\nStefan\n"},{"id":"364707","messageId":"20181206212459.GN4633@hopa.kiewit.dartmouth.edu","threadId":"49961","inReplyTo":"CAGZ79kY8uv8zDm3f8Jb6aC-nit7OZduixyOekGYWa_xnqFqw-w@mail.gmail.com","subject":"Re: [wishlist] git submodule update --reset-hard","fromName":"Yaroslav Halchenko","fromEmail":"yoh@onerussian.com","sentAt":"2018-12-06T21:24:59Z","receivedAt":"2018-12-06T21:25:10Z","isPatch":false,"sender":{"key":"yoh@onerussian.com","avatar":"https://gravatar.com/avatar/8901b82415ae451a83aea49409708912726e53620e3ac92320bf1f86548d97e9?d=mp&s=160"},"body":"\nOn Thu, 06 Dec 2018, Stefan Beller wrote:\n\n> On Thu, Dec 6, 2018 at 10:02 AM Yaroslav Halchenko wrote:\n\n> > Dear Git Gurus,\n\n> > I wondered what would be your take on my wishlist request to add\n> > --reset-hard option, which would be very similar to regular \"update\" which\n> > checks out necessary commit, but I want it to remain in the branch.\n\n> What if the branch differs from the sha1 recorded in the superproject?\n\ngit reset --hard itself is an operation which should be done with some\nlevel of competence in doing \"the right thing\" by calling it. You\ncan hop branches even in current (without any submodules in question)\nrepository with it and cause as much chaos as you desire.\n\nIf desired though, a number of prevention mechanisms could be in place (but\nwould require option(s) to overcome) to allow submodule to be reset --hard'ed\nonly when some conditions met (e.g. only to the commit which is among parent\ncommits path of the current branch). This way wild hops would be prevented,\nalthough you might still end up in some feature branch. But since \"reset\n--hard\" itself doesn't have any safe-guards, I do not really think they should\nbe implemented here either.\n\n> > Rationale: In DataLad we heavily rely on submodules, and we have established\n> > easy ways to do some manipulations across full hierarchies of them. E.g. a\n> > single command could introduce a good number of commits across deep hierarchy\n> > of submodules, e.g. while committing changes within deep submodule, while also\n> > doing all necessary commits in the repositories leading to that submodule so\n> > the entire tree of them stays in a \"clean\" state. The difficulty comes when\n> > there is a need to just \"forget\" some changes. The obvious way is to e.g.\n\n> > git reset --hard PREVIOUS_STATE\n\n> git reset --hard --recurse-submodules HEAD\n\n> would do the trick\n\nit does indeed some trick(s) but not all seems to be the ones I desire:\n\n1. Seems to migrate submodule's .git directories into the top level\n.git/modules\n\n\t$> git reset --hard --recurse-submodules HEAD^^^\n\tMigrating git directory of 'famface' from \n\t'/tmp/gobbini/famface/.git' to\n\t'/tmp/gobbini/.git/modules/famface'\n\tMigrating git directory of 'famface/data' from\n\t'/tmp/gobbini/famface/data/.git' to\n\t'/tmp/gobbini/.git/modules/famface/modules/data'\n\tMigrating git directory of 'famface/data/scripts/mridefacer' from\n\t'/tmp/gobbini/famface/data/scripts/mridefacer/.git' to\n\t'/tmp/gobbini/.git/modules/famface/modules/data/modules/scripts/mridefacer'\n\tHEAD is now at 9b4296d [DATALAD] aggregated meta data\n\nwe might eventually adopt this default already for years model (git annex seems\nto be ok, in that it then replaces .git symlink file with the actual\nsymlink .git -> ../../.git/modules/... So things seems to keep working\nfor annex)\n\n2. It still does the detached HEAD for me\n\n\t$> git submodule status --recursive \n\t 2569ab436501a832d35afbbe9cc20ffeb6077eb1 famface (2569ab4)\n\t f1e8c9b8b025c311424283b9711efc6bc906ba2b famface/data (BIDS-v1.0.1)\n\t 49b0fe42696724c2a8492f999736056e51b77358 famface/data/scripts/mridefacer (49b0fe4)\n\n\n> > in the top level repository. But that leaves all the submodules now in\n> > the undesired state. If I do\n\n> undesirable in the sense of still having local changes (that is what\n> the above reset with `--recurse` would fix) or changed the branch\n> state? (i.e. is detached but was on a branch before?)\n\nright -- I meant the local changes and indeed reset --recurse-submodules\nindeed seems to recurse nicely. Then the undesired effect remaining only\nthe detached HEAD\n\n> > git submodule update --recursive\n\n> > I would end up in the detached HEADs within submodules.\n\n> > What I want is to retain current branch they are at (or may be possible\n> > \"were in\"? reflog records might have that information)\n\n> So something like\n\n> git submodule foreach --recursive git reset --hard\n\n> ?\n\nnot quite -- this would just kill all local changes within each submodule, not\nto reset it to the desired state, which wouldn't be specified in such\ninvocation, and is only known to the repo containing it\n\n> You may be interested in\n> https://public-inbox.org/git/20180927221603.148025-1-sbeller@google.com/\n> which introduces a switch `submodule.repoLike [ = true]`, which\n> when set would not detach HEAD in submodules.\n\nThanks! looks interesting -- was there more discussion/activity beyond those 5\nposts in the thread?\nhttps://public-inbox.org/git/87h8i9ift4.fsf@evledraar.gmail.com/#r \n\nThis feature might indeed come handy but if I got it right, it is somewhat\ncomplimentary to just having submodule update --reset-hard . E.g. submodules\nmight be in different branches (if I am not tracking based on branch names), so\nI would not want a recursive checkout with -b|-B. But we would indeed benefit\nfrom such functionality, since this difficulty of managing branches of\nsubmodules I think would be elevated with it! (e.g. in one use case we probably\nwill end up with a few thousands of submodules, and at least 3 branches in each\nwhich would need to be in sync, and typically you wouldn't want different\nbranches to be checked out in different submodules)\n\n> Can you say more about the first question above:\n> Would you typically have situations where the\n> submodule branch is out of sync with the superproject\n> and how do you deal with that?\n\ntypically I do not have anything out of sync. My primary use-case is to\n\"cancel\" recent changes in the entire tree of repositories. I guess for\nmy use case, instead of needing two commands\n\n git reset --hard PREVIOUSPOINT\n git submodule update --reset--hard --recursive\n\nI wish there was just one\n\n git reset --hard --recursive PREVIOUSPOINT\n\nbut I felt that submodule update might be a better starting point\nsince it already provides different modes for update. If I was even greedier,\nI would have asked for \n\n git revert --recursive ...\n git rebase --recursive [-i] ...\n\nwhich I also frequently desire (could elaborate on the use cases etc).\n\nNB or --recurse-submodules to avoid confusion with recursive merge\nstrategy?\n\n\nBut for a complete answer -- if submodule branch is ahead of the superproject's\nrecord, I just commit new state for it in superproject. Or if I see that all\nwhat I have done was actually a throw away -- \"reset --hard\" to that needed\nstate, again manually in submodule. With submodule update --reset-hard\n--recursive or git reset --hard --recursive I would be just ready\nregardless of the depth and complexity of the hierarchy ;-)\n\n> Adding another mode to `git submodule update` sounds\n> reasonable to me, too.\n\ncool, thanks!\n\n-- \nYaroslav O. Halchenko\nCenter for Open Neuroscience http://centerforopenneuroscience.org\nDartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755\nPhone: +1 (603) 646-9834 Fax: +1 (603) 646-1419\nWWW: http://www.linkedin.com/in/yarik \n"},{"id":"364710","messageId":"CAGZ79kYoGqWW4tv4-caA18SHKe+y2mnDT84AEWVksDtDObLq0g@mail.gmail.com","threadId":"49961","inReplyTo":"20181206212459.GN4633@hopa.kiewit.dartmouth.edu","subject":"Re: [wishlist] git submodule update --reset-hard","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-12-06T21:55:03Z","receivedAt":"2018-12-06T21:55:18Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Dec 6, 2018 at 1:25 PM Yaroslav Halchenko wrote:\n>\n>\n> On Thu, 06 Dec 2018, Stefan Beller wrote:\n>\n> > On Thu, Dec 6, 2018 at 10:02 AM Yaroslav Halchenko wrote:\n>\n> > > Dear Git Gurus,\n>\n> > > I wondered what would be your take on my wishlist request to add\n> > > --reset-hard option, which would be very similar to regular \"update\" which\n> > > checks out necessary commit, but I want it to remain in the branch.\n>\n> > What if the branch differs from the sha1 recorded in the superproject?\n>\n> git reset --hard itself is an operation which should be done with some\n> level of competence in doing \"the right thing\" by calling it. You\n> can hop branches even in current (without any submodules in question)\n> repository with it and cause as much chaos as you desire.\n\nRight.\n\ngit reset --hard would the branch (as well as the working tree) to the\ngiven sha1, which is confusing as submodules get involved.\n\nThe Right Thing as of now is the sha1 as found in the\nsuperprojects gitlink. But as that can be different from any branch\nin the submodule, we'd rather detach the HEAD to make it\ndeterministic.\n\nThere was a proposal to \"re-attach HEAD\" in the submodule, i.e.\nif the branch branch points at the same commit, we don't need\na detached HEAD, but could go with the branch instead.\n\n> If desired though, a number of prevention mechanisms could be in place (but\n> would require option(s) to overcome) to allow submodule to be reset --hard'ed\n> only when some conditions met (e.g. only to the commit which is among parent\n> commits path of the current branch). This way wild hops would be prevented,\n> although you might still end up in some feature branch. But since \"reset\n> --hard\" itself doesn't have any safe-guards, I do not really think they should\n> be implemented here either.\n\nSo are you looking for\na) \"stay on submodule branch (i.e. HEAD still points at $branch), and\nreset --hard\"\n such that the submodule has a clean index and at that $branch or\nb) \"stay on submodule branch (i.e. HEAD still points at $branch), but $branch is\n set to the gitlink from the superproject, and then a reset --hard\nwill have the worktree\n set to it as well.\n\n(a) is what the referenced submodule.repoLike option implements.\n\nI'd understand the desire for (b) as well, as it is a \"real\" hard reset on\nthe superproject level, without detaching branches.\n\n> > git reset --hard --recurse-submodules HEAD\n\n> it does indeed some trick(s) but not all seems to be the ones I desire:\n>\n> 1. Seems to migrate submodule's .git directories into the top level\n> .git/modules\n\nAh yes, that happens too. This will help once you want to git-rm\na submodule and checkout states before and after.\n\n> > undesirable in the sense of still having local changes (that is what\n> > the above reset with `--recurse` would fix) or changed the branch\n> > state? (i.e. is detached but was on a branch before?)\n>\n> right -- I meant the local changes and indeed reset --recurse-submodules\n> indeed seems to recurse nicely. Then the undesired effect remaining only\n> the detached HEAD\n\nFor that we may want to revive discussions in\nhttps://public-inbox.org/git/20170501180058.8063-5-sbeller@google.com/\n\n\n> > > git submodule update --recursive\n>\n> > > I would end up in the detached HEADs within submodules.\n>\n> > > What I want is to retain current branch they are at (or may be possible\n> > > \"were in\"? reflog records might have that information)\n>\n> > So something like\n>\n> > git submodule foreach --recursive git reset --hard\n>\n> > ?\n>\n> not quite -- this would just kill all local changes within each submodule, not\n> to reset it to the desired state, which wouldn't be specified in such\n> invocation, and is only known to the repo containing it\n\nWith this answer it sounds like you'd want (b) from above.\n\n> > You may be interested in\n> > https://public-inbox.org/git/20180927221603.148025-1-sbeller@google.com/\n> > which introduces a switch `submodule.repoLike [ = true]`, which\n> > when set would not detach HEAD in submodules.\n>\n> Thanks! looks interesting -- was there more discussion/activity beyond those 5\n> posts in the thread?\n\nUnfortunately there was not.\n\n> This feature might indeed come handy but if I got it right, it is somewhat\n> complimentary to just having submodule update --reset-hard . E.g. submodules\n> might be in different branches (if I am not tracking based on branch names), so\n> I would not want a recursive checkout with -b|-B. But we would indeed benefit\n> from such functionality, since this difficulty of managing branches of\n> submodules I think would be elevated with it! (e.g. in one use case we probably\n> will end up with a few thousands of submodules, and at least 3 branches in each\n> which would need to be in sync, and typically you wouldn't want different\n> branches to be checked out in different submodules)\n>\n> > Can you say more about the first question above:\n> > Would you typically have situations where the\n> > submodule branch is out of sync with the superproject\n> > and how do you deal with that?\n>\n> typically I do not have anything out of sync. My primary use-case is to\n> \"cancel\" recent changes in the entire tree of repositories. I guess for\n> my use case, instead of needing two commands\n>\n> git reset --hard PREVIOUSPOINT\n> git submodule update --reset--hard --recursive\n>\n> I wish there was just one\n>\n> git reset --hard --recursive PREVIOUSPOINT\n\nMaybe this could learn options like\n\n git reset --hard --recursive=hard,keep-branch PREVIOUSPOINT\n\nwhich then could be put into options like\n\n git config reset.recurseSubmodules hard,keep-branch &&\n # maybe not needed, depending on the exact meaning\n # of reset.recurseSubmodules:\n git config submodule.recurse\n\nand then\n\n git reset --hard PREVIOUS\n\nwould do what you'd desire.\n\n> but I felt that submodule update might be a better starting point\n> since it already provides different modes for update. If I was even greedier,\n> I would have asked for\n>\n> git revert --recursive ...\n> git rebase --recursive [-i] ...\n>\n> which I also frequently desire (could elaborate on the use cases etc).\n\nThese would be nice to have. It would be nice if you'd elaborate on the\nuse cases for future reference in the mailing list archive. :-)\n\n>\n> NB or --recurse-submodules to avoid confusion with recursive merge\n> strategy?\n\n... and sometimes recursing in the file system, c.f. `ls-tree -r`.\n"},{"id":"364731","messageId":"20181207012256.GR4633@hopa.kiewit.dartmouth.edu","threadId":"49961","inReplyTo":"CAGZ79kYoGqWW4tv4-caA18SHKe+y2mnDT84AEWVksDtDObLq0g@mail.gmail.com","subject":"Re: [wishlist] git submodule update --reset-hard","fromName":"Yaroslav Halchenko","fromEmail":"yoh@onerussian.com","sentAt":"2018-12-07T01:22:56Z","receivedAt":"2018-12-07T01:23:05Z","isPatch":false,"sender":{"key":"yoh@onerussian.com","avatar":"https://gravatar.com/avatar/8901b82415ae451a83aea49409708912726e53620e3ac92320bf1f86548d97e9?d=mp&s=160"},"body":"Hi Stefan,\n\nThanks for the dialogue! Replies are embedded below.\n\nOn Thu, 06 Dec 2018, Stefan Beller wrote:\n> ...\n> > > What if the branch differs from the sha1 recorded in the superproject?\n\n> > git reset --hard itself is an operation which should be done with some\n> > level of competence in doing \"the right thing\" by calling it. You\n> > can hop branches even in current (without any submodules in question)\n> > repository with it and cause as much chaos as you desire.\n\n> Right.\n\n> git reset --hard would the branch (as well as the working tree) to the\n> given sha1, which is confusing as submodules get involved.\n\n> The Right Thing as of now is the sha1 as found in the\n> superprojects gitlink. But as that can be different from any branch\n> in the submodule, we'd rather detach the HEAD to make it\n> deterministic.\n\nyeap, makes total sense to be the thing do that by default ;-)\n\n> There was a proposal to \"re-attach HEAD\" in the submodule, i.e.\n> if the branch branch points at the same commit, we don't need\n> a detached HEAD, but could go with the branch instead.\n\nif I got the idea right, if we are talking about any branch, it\nwould also non-deterministic since who knows what left over branch(es)\npoint to that commit. Not sure if I would have used that ;)\n\n> > If desired though, a number of prevention mechanisms could be in place (but\n> > would require option(s) to overcome) to allow submodule to be reset --hard'ed\n> > only when some conditions met (e.g. only to the commit which is among parent\n> > commits path of the current branch). This way wild hops would be prevented,\n> > although you might still end up in some feature branch. But since \"reset\n> > --hard\" itself doesn't have any safe-guards, I do not really think they should\n> > be implemented here either.\n\n> So are you looking for\n> a) \"stay on submodule branch (i.e. HEAD still points at $branch), and\n> reset --hard\" such that the submodule has a clean index and at that $branch \n> or\n> b) \"stay on submodule branch (i.e. HEAD still points at $branch), but $branch is\n> set to the gitlink from the superproject, and then a reset --hard\n> will have the worktree set to it as well.\n\nyes!\n\nNB \"gitlink\" -- just now discovered the thing for me. Thought it would be\ncalled a subproject echoing what git diff/log -p shows for submodule commits.\n\n> (a) is what the referenced submodule.repoLike option implements.\n\nsounds like it indeed, thanks for spelling out\n\n> I'd understand the desire for (b) as well, as it is a \"real\" hard reset on\n> the superproject level, without detaching branches.\n\nyeap\n\n> > > git reset --hard --recurse-submodules HEAD\n\n> > it does indeed some trick(s) but not all seems to be the ones I desire:\n\n> > 1. Seems to migrate submodule's .git directories into the top level\n> > .git/modules\n\n> Ah yes, that happens too. This will help once you want to git-rm\n> a submodule and checkout states before and after.\n\nyeap ;-) \n\n> > > undesirable in the sense of still having local changes (that is what\n> > > the above reset with `--recurse` would fix) or changed the branch\n> > > state? (i.e. is detached but was on a branch before?)\n\n> > right -- I meant the local changes and indeed reset --recurse-submodules\n> > indeed seems to recurse nicely. Then the undesired effect remaining only\n> > the detached HEAD\n\n> For that we may want to revive discussions in\n> https://public-inbox.org/git/20170501180058.8063-5-sbeller@google.com/\n\nwell, isn't that one requires a branch to be specified in .gitmodules?\n\n> > > > git submodule update --recursive\n\n> > > > I would end up in the detached HEADs within submodules.\n\n> > > > What I want is to retain current branch they are at (or may be possible\n> > > > \"were in\"? reflog records might have that information)\n\n> > > So something like\n\n> > > git submodule foreach --recursive git reset --hard\n\n> > > ?\n\n> > not quite -- this would just kill all local changes within each submodule, not\n> > to reset it to the desired state, which wouldn't be specified in such\n> > invocation, and is only known to the repo containing it\n\n> With this answer it sounds like you'd want (b) from above.\n\nyeap\n\n> > > You may be interested in\n> > > https://public-inbox.org/git/20180927221603.148025-1-sbeller@google.com/\n> > > which introduces a switch `submodule.repoLike [ = true]`, which\n> > > when set would not detach HEAD in submodules.\n\n> > Thanks! looks interesting -- was there more discussion/activity beyond those 5\n> > posts in the thread?\n\n> Unfortunately there was not.\n\npity\n\n> > This feature might indeed come handy but if I got it right, it is somewhat\n> > complimentary to just having submodule update --reset-hard . E.g. submodules\n> > might be in different branches (if I am not tracking based on branch names), so\n> > I would not want a recursive checkout with -b|-B. But we would indeed benefit\n> > from such functionality, since this difficulty of managing branches of\n> > submodules I think would be elevated with it! (e.g. in one use case we probably\n> > will end up with a few thousands of submodules, and at least 3 branches in each\n> > which would need to be in sync, and typically you wouldn't want different\n> > branches to be checked out in different submodules)\n\n> > > Can you say more about the first question above:\n> > > Would you typically have situations where the\n> > > submodule branch is out of sync with the superproject\n> > > and how do you deal with that?\n\n> > typically I do not have anything out of sync. My primary use-case is to\n> > \"cancel\" recent changes in the entire tree of repositories. I guess for\n> > my use case, instead of needing two commands\n\n> > git reset --hard PREVIOUSPOINT\n> > git submodule update --reset--hard --recursive\n\n> > I wish there was just one\n\n> > git reset --hard --recursive PREVIOUSPOINT\n\n> Maybe this could learn options like\n\n> git reset --hard --recursive=hard,keep-branch PREVIOUSPOINT\n\n'keep-branch' (given aforementioned keeping the specified in .gitmodules\nbranch) might be confusing. Also what if a submodule already in a\ndetached HEAD? IMHO --recursive=hard, and just saying that it would do\n\"reset --hard\", is imho sufficient. (that is why I like pure\n--reset hard since it doesn't care and neither does anything to the\nbranch)\n\n> which then could be put into options like\n\n> git config reset.recurseSubmodules hard,keep-branch &&\n> # maybe not needed, depending on the exact meaning\n> # of reset.recurseSubmodules:\n> git config submodule.recurse\n\n> and then\n\n> git reset --hard PREVIOUS\n\n> would do what you'd desire.\n\nyou mean\n\n git reset --hard --recurse-submodules PREVIOUS\n\nin principle overall I would love to have it, besides not sure what\nother than 'hard' could be there, and what 'keep-branch' would exactly\ndo ;-)\n\n> > but I felt that submodule update might be a better starting point\n> > since it already provides different modes for update. If I was even greedier,\n> > I would have asked for\n\n> > git revert --recursive ...\n> > git rebase --recursive [-i] ...\n\n> > which I also frequently desire (could elaborate on the use cases etc).\n\n> These would be nice to have. It would be nice if you'd elaborate on the\n> use cases for future reference in the mailing list archive. :-)\n\nok, will try to do so ;-) In summary: they are just a logical extension\nof git support for submodules for anyone actively working with\nsubmodules to keep entire tree in sync. Then quite often the need for\nreverting a specific commit (which also has changes reflected in\nsubmodules) arises. The same with rebase, especially to trim away some\nno longer desired changes reflected in submodules. \n\nthe initial \"git submodule update --reset-hard\" is pretty much a\ncrude workaround for some of those cases, so I would just go earlier in\nthe history, and redo some things, whenever I could just drop or revert\nsome selected set of commits.\n\n> > NB or --recurse-submodules to avoid confusion with recursive merge\n> > strategy?\n\n> ... and sometimes recursing in the file system, c.f. `ls-tree -r`.\n\nah... so it is only submodule command which has --recursive, and the\nrest have --recurse-submodules when talking about recursing into\nsubmodules?\n\n-- \nYaroslav O. Halchenko\nCenter for Open Neuroscience http://centerforopenneuroscience.org\nDartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755\nPhone: +1 (603) 646-9834 Fax: +1 (603) 646-1419\nWWW: http://www.linkedin.com/in/yarik \n"},{"id":"364758","messageId":"CAGZ79kbeAd1C-ySnJye-QU5FFf2jygksUsWtEmbvPZ_dQy_3uA@mail.gmail.com","threadId":"49961","inReplyTo":"20181207012256.GR4633@hopa.kiewit.dartmouth.edu","subject":"Re: [wishlist] git submodule update --reset-hard","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-12-07T21:55:15Z","receivedAt":"2018-12-07T21:55:31Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Dec 6, 2018 at 5:23 PM Yaroslav Halchenko wrote:\n\n> > There was a proposal to \"re-attach HEAD\" in the submodule, i.e.\n> > if the branch branch points at the same commit, we don't need\n> > a detached HEAD, but could go with the branch instead.\n>\n> if I got the idea right, if we are talking about any branch, it\n> would also non-deterministic since who knows what left over branch(es)\n> point to that commit. Not sure if I would have used that ;)\n\nI would think we'd rather want to have it deterministic, i.e. something like\n1) record branch name of the submodule\n2) update submodules HEAD to to superprojects gitlink\n3) if recorded branch (1) matches the sha1 of detached HEAD,\n have HEAD point to the branch instead.\n\nYou notice a small inefficiency here as we write HEAD twice, so it\ncould be reworded as:\n1) compare superprojects gitlink with the submodules branch\n2a) if equal, set submodules HEAD to branch\n2b) if unequal set HEAD to gitlink value, resulting in detached HEAD\n\nNote that this idea of reattaching reflects the idea (a) below.\n\n\n> > a) \"stay on submodule branch (i.e. HEAD still points at $branch), and\n> > reset --hard\" such that the submodule has a clean index and at that $branch\n> > or\n> > b) \"stay on submodule branch (i.e. HEAD still points at $branch), but $branch is\n> > set to the gitlink from the superproject, and then a reset --hard\n> > will have the worktree set to it as well.\n\n\n> NB \"gitlink\" -- just now discovered the thing for me. Thought it would be\n> called a subproject echoing what git diff/log -p shows for submodule commits.\n\nThe terminology is messy:\nThe internal representation in Gits object model is a \"gitlink\" entry in a tree\nobject. Once we have a .gitmodules entry, we call it submodule.\n\nThe term 'subproject' is a historic artifact and will likely not be changed\nin the diff output (or format-patch), because these diffs can be applied using\ngit-am for example. That makes the diff output effectively a transport\nprotocol, and changing protocols is hard if you have no versioning in them.\n\nMore in https://git-scm.com/docs/gitsubmodules (a rather recent new write\nof a man page, going into concepts).\n\n> > > right -- I meant the local changes and indeed reset --recurse-submodules\n> > > indeed seems to recurse nicely. Then the undesired effect remaining only\n> > > the detached HEAD\n>\n> > For that we may want to revive discussions in\n> > https://public-inbox.org/git/20170501180058.8063-5-sbeller@google.com/\n>\n> well, isn't that one requires a branch to be specified in .gitmodules?\n\nAh good point.\n\n> > git reset --hard --recursive=hard,keep-branch PREVIOUSPOINT\n>\n> 'keep-branch' (given aforementioned keeping the specified in .gitmodules\n> branch) might be confusing. Also what if a submodule already in a\n> detached HEAD? IMHO --recursive=hard, and just saying that it would do\n> \"reset --hard\", is imho sufficient. (that is why I like pure\n> --reset hard since it doesn't care and neither does anything to the\n> branch)\n\nFor that we might want to first do the\n\n git submodule update --reset-hard\n\nwhich runs reset --hard inside the submodule, no matter which\nbranch the submodule is on (if any) and resets to the given\nsuperproject sha1.\n\nSee git-submodule.sh in git.git[1] in cmd_update.\nWe'd need to add a command line flag (`--reset-hard`\nwould be the obvious choice?) which would set the `update`\nvariable, which then is evaluated to what needs to be done in\nthe submodule, which in that case would be the hard reset.\n\nhttps://github.com/git/git/blob/master/git-submodule.sh#L606\n\nOnce that is done we'd want to add a test case, presumably\nin t/t7406-submodule-update.sh\n\n> > > I would have asked for\n>\n> > > git revert --recursive ...\n> > > git rebase --recursive [-i] ...\n>\n> > > which I also frequently desire (could elaborate on the use cases etc).\n>\n> > These would be nice to have. It would be nice if you'd elaborate on the\n> > use cases for future reference in the mailing list archive. :-)\n>\n> ok, will try to do so ;-) In summary: they are just a logical extension\n> of git support for submodules for anyone actively working with\n> submodules to keep entire tree in sync. Then quite often the need for\n> reverting a specific commit (which also has changes reflected in\n> submodules) arises. The same with rebase, especially to trim away some\n> no longer desired changes reflected in submodules.\n>\n> the initial \"git submodule update --reset-hard\" is pretty much a\n> crude workaround for some of those cases, so I would just go earlier in\n> the history, and redo some things, whenever I could just drop or revert\n> some selected set of commits.\n\nThat makes sense.\nDo you want to give the implementation a try for the --reset-hard switch?\n\n> ah... so it is only submodule command which has --recursive, and the\n> rest have --recurse-submodules when talking about recursing into\n> submodules?\n\nI don't think we were that cautious in development as it was done by\ndifferent people at different times. There is also just `--submodule` for\nthe diff family, for reference:\nhttps://public-inbox.org/git/20180905225828.17782-1-sbeller@google.com/\n"},{"id":"364785","messageId":"20181208021531.GB4633@hopa.kiewit.dartmouth.edu","threadId":"49961","inReplyTo":"CAGZ79kbeAd1C-ySnJye-QU5FFf2jygksUsWtEmbvPZ_dQy_3uA@mail.gmail.com","subject":"Re: [wishlist] git submodule update --reset-hard","fromName":"Yaroslav Halchenko","fromEmail":"yoh@onerussian.com","sentAt":"2018-12-08T02:15:31Z","receivedAt":"2018-12-08T02:15:39Z","isPatch":false,"sender":{"key":"yoh@onerussian.com","avatar":"https://gravatar.com/avatar/8901b82415ae451a83aea49409708912726e53620e3ac92320bf1f86548d97e9?d=mp&s=160"},"body":"\nOn Fri, 07 Dec 2018, Stefan Beller wrote:\n> > the initial \"git submodule update --reset-hard\" is pretty much a\n> > crude workaround for some of those cases, so I would just go earlier in\n> > the history, and redo some things, whenever I could just drop or revert\n> > some selected set of commits.\n\n> That makes sense.\n> Do you want to give the implementation a try for the --reset-hard switch?\n\nok, will do, thanks for the blessing ;-)\n\n-- \nYaroslav O. Halchenko\nCenter for Open Neuroscience http://centerforopenneuroscience.org\nDartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755\nPhone: +1 (603) 646-9834 Fax: +1 (603) 646-1419\nWWW: http://www.linkedin.com/in/yarik \n"},{"id":"364787","messageId":"20181208042139.GA4827@hopa.kiewit.dartmouth.edu","threadId":"49961","inReplyTo":"20181208021531.GB4633@hopa.kiewit.dartmouth.edu","subject":"Re: [wishlist] git submodule update --reset-hard","fromName":"Yaroslav Halchenko","fromEmail":"yoh@onerussian.com","sentAt":"2018-12-08T04:21:39Z","receivedAt":"2018-12-08T04:21:53Z","isPatch":false,"sender":{"key":"yoh@onerussian.com","avatar":"https://gravatar.com/avatar/8901b82415ae451a83aea49409708912726e53620e3ac92320bf1f86548d97e9?d=mp&s=160"},"body":"\nOn Fri, 07 Dec 2018, Yaroslav Halchenko wrote:\n\n\n> On Fri, 07 Dec 2018, Stefan Beller wrote:\n> > > the initial \"git submodule update --reset-hard\" is pretty much a\n> > > crude workaround for some of those cases, so I would just go earlier in\n> > > the history, and redo some things, whenever I could just drop or revert\n> > > some selected set of commits.\n\n> > That makes sense.\n> > Do you want to give the implementation a try for the --reset-hard switch?\n\n> ok, will do, thanks for the blessing ;-)\n\nThe patch is attached (please advise if should be done\ndifferently) and also submitted as PR\nhttps://github.com/git/git/pull/563\n\nI guess it would need more tests. Took me some time to figure out\nwhy I was getting\n\n\tfatal: bad value for update parameter\n\nafter all my changes to the git-submodule.sh script after looking at an\nexample commit 42b491786260eb17d97ea9fb1c4b70075bca9523 which introduced\n--merge to the update ;-)\n\n-- \nYaroslav O. Halchenko\nCenter for Open Neuroscience http://centerforopenneuroscience.org\nDartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755\nPhone: +1 (603) 646-9834 Fax: +1 (603) 646-1419\nWWW: http://www.linkedin.com/in/yarik \n\n\nFrom 170296dc661b4bc3d942917ce27288df52ff650d Mon Sep 17 00:00:00 2001\nFrom: Yaroslav Halchenko \nDate: Fri, 7 Dec 2018 21:28:29 -0500\nSubject: [PATCH] submodule: Add --reset-hard option for git submodule update\n\nThis patch adds a --reset-hard option for the update command to hard\nreset submodule(s) to the gitlink for the corresponding submodule in\nthe superproject. This feature is desired e.g. to be able to discard\nrecent changes in the entire hierarchy of the submodules after running\n\n git reset --hard PREVIOUS_STATE\n\nin the superproject which leaves submodules in their original state,\nand\n\n git reset --hard --recurse-submodules PREVIOUS_STATE\n\nwould result in the submodules being checked into detached HEADs.\n\nAs in the original git reset --hard no checks or any kind of\nsafe-guards against jumping into some state which was never on the\ncurrent branch is done.\n\nmust_die_on_failure is not set to yes to mimic behavior of a update\n--checkout strategy, which would leave user with a non-clean state\nimmediately apparent via git status so an informed decision/actions\ncould be done manually.\n\nSigned-off-by: Yaroslav Halchenko \n---\n Documentation/git-submodule.txt | 12 +++++++++++-\n Documentation/gitmodules.txt | 4 ++--\n builtin/submodule--helper.c | 3 ++-\n git-submodule.sh | 10 +++++++++-\n submodule.c | 4 ++++\n submodule.h | 1 +\n t/t7406-submodule-update.sh | 17 ++++++++++++++++-\n 7 files changed, 45 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt\nindex ba3c4df550..f90a42d265 100644\n--- a/Documentation/git-submodule.txt\n+++ b/Documentation/git-submodule.txt\n@@ -124,7 +124,7 @@ If you really want to remove a submodule from the repository and commit\n that use linkgit:git-rm[1] instead. See linkgit:gitsubmodules[7] for removal\n options.\n \n-update [--init] [--remote] [-N|--no-fetch] [--[no-]recommend-shallow] [-f|--force] [--checkout|--rebase|--merge] [--reference ] [--depth ] [--recursive] [--jobs ] [--] [...]::\n+update [--init] [--remote] [-N|--no-fetch] [--[no-]recommend-shallow] [-f|--force] [--checkout|--rebase|--merge|--reset-hard] [--reference ] [--depth ] [--recursive] [--jobs ] [--] [...]::\n +\n --\n Update the registered submodules to match what the superproject\n@@ -358,6 +358,16 @@ the submodule itself.\n \tIf the key `submodule.$name.update` is set to `rebase`, this option is\n \timplicit.\n \n+--reset-hard::\n+\tThis option is only valid for the update command.\n+\tHard reset current state to the commit recorded in the\tsuperproject.\n+ If this option is given, the submodule's HEAD will not get detached\n+ if it was not detached before. Note that, like with a regular\n+ git reset --hard no safe-guards are in place to prevent jumping\n+ to a commit which was never part of the current branch.\n+\tIf the key `submodule.$name.update` is set to `reset-hard`, this\n+\toption is implicit.\n+\n --init::\n \tThis option is only valid for the update command.\n \tInitialize all submodules for which \"git submodule init\" has not been\ndiff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt\nindex 312b6f9259..e085dbc01f 100644\n--- a/Documentation/gitmodules.txt\n+++ b/Documentation/gitmodules.txt\n@@ -43,8 +43,8 @@ submodule..update::\n \tcommand in the superproject. This is only used by `git\n \tsubmodule init` to initialize the configuration variable of\n \tthe same name. Allowed values here are 'checkout', 'rebase',\n-\t'merge' or 'none'. See description of 'update' command in\n-\tlinkgit:git-submodule[1] for their meaning. Note that the\n+\t'merge', 'reset-hard' or 'none'. See description of 'update' command\n+\tin linkgit:git-submodule[1] for their meaning. Note that the\n \t'!command' form is intentionally ignored here for security\n \treasons.\n \ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex d38113a31a..31d95c3cd6 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -1481,6 +1481,7 @@ static void determine_submodule_update_strategy(struct repository *r,\n \tif (just_cloned &&\n \t (out->type == SM_UPDATE_MERGE ||\n \t out->type == SM_UPDATE_REBASE ||\n+\t out->type == SM_UPDATE_RESET_HARD ||\n \t out->type == SM_UPDATE_NONE))\n \t\tout->type = SM_UPDATE_CHECKOUT;\n \n@@ -1851,7 +1852,7 @@ static int update_clone(int argc, const char **argv, const char *prefix)\n \t\t\t \"submodule boundaries\")),\n \t\tOPT_STRING(0, \"update\", &update,\n \t\t\t N_(\"string\"),\n-\t\t\t N_(\"rebase, merge, checkout or none\")),\n+\t\t\t N_(\"rebase, merge, checkout, reset-hard or none\")),\n \t\tOPT_STRING_LIST(0, \"reference\", &suc.references, N_(\"repo\"),\n \t\t\t N_(\"reference repository\")),\n \t\tOPT_BOOL(0, \"dissociate\", &suc.dissociate,\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 5e608f8bad..b5d6fad983 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -9,7 +9,7 @@ USAGE=\"[--quiet] add [-b ] [-f|--force] [--name ] [--reference ...]\n or: $dashless [--quiet] init [--] [...]\n or: $dashless [--quiet] deinit [-f|--force] (--all| [--] ...)\n- or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--[no-]recommend-shallow] [--reference ] [--recursive] [--] [...]\n+ or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase|--reset-hard] [--[no-]recommend-shallow] [--reference ] [--recursive] [--] [...]\n or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...]\n or: $dashless [--quiet] foreach [--recursive] \n or: $dashless [--quiet] sync [--recursive] [--] [...]\n@@ -483,6 +483,9 @@ cmd_update()\n \t\t-m|--merge)\n \t\t\tupdate=\"merge\"\n \t\t\t;;\n+\t\t--reset-hard)\n+\t\t\tupdate=\"reset-hard\"\n+\t\t\t;;\n \t\t--recursive)\n \t\t\trecursive=1\n \t\t\t;;\n@@ -621,6 +624,11 @@ cmd_update()\n \t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': merged in '\\$sha1'\")\"\n \t\t\t\tmust_die_on_failure=yes\n \t\t\t\t;;\n+\t\t\treset-hard)\n+\t\t\t\tcommand=\"git reset --hard\"\n+\t\t\t\tdie_msg=\"$(eval_gettext \"Unable to reset --hard to '\\$sha1' in submodule path '\\$displaypath'\")\"\n+\t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': was reset --hard to '\\$sha1'\")\"\n+\t\t\t\t;;\n \t\t\t!*)\n \t\t\t\tcommand=\"${update_module#!}\"\n \t\t\t\tdie_msg=\"$(eval_gettext \"Execution of '\\$command \\$sha1' failed in submodule path '\\$displaypath'\")\"\ndiff --git a/submodule.c b/submodule.c\nindex 6415cc5580..4580cf0944 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -373,6 +373,8 @@ enum submodule_update_type parse_submodule_update_type(const char *value)\n \t\treturn SM_UPDATE_REBASE;\n \telse if (!strcmp(value, \"merge\"))\n \t\treturn SM_UPDATE_MERGE;\n+\telse if (!strcmp(value, \"reset-hard\"))\n+\t\treturn SM_UPDATE_RESET_HARD;\n \telse if (*value == '!')\n \t\treturn SM_UPDATE_COMMAND;\n \telse\n@@ -406,6 +408,8 @@ const char *submodule_strategy_to_string(const struct submodule_update_strategy\n \t\treturn \"checkout\";\n \tcase SM_UPDATE_MERGE:\n \t\treturn \"merge\";\n+\tcase SM_UPDATE_RESET_HARD:\n+\t\treturn \"reset-hard\";\n \tcase SM_UPDATE_REBASE:\n \t\treturn \"rebase\";\n \tcase SM_UPDATE_NONE:\ndiff --git a/submodule.h b/submodule.h\nindex a680214c01..f23ac4630e 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -29,6 +29,7 @@ enum submodule_update_type {\n \tSM_UPDATE_CHECKOUT,\n \tSM_UPDATE_REBASE,\n \tSM_UPDATE_MERGE,\n+\tSM_UPDATE_RESET_HARD,\n \tSM_UPDATE_NONE,\n \tSM_UPDATE_COMMAND\n };\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex e87164aa8f..2e08e0047c 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -6,7 +6,8 @@\n test_description='Test updating submodules\n \n This test verifies that \"git submodule update\" detaches the HEAD of the\n-submodule and \"git submodule update --rebase/--merge\" does not detach the HEAD.\n+submodule and \"git submodule update --rebase/--merge/--reset-hard\" does\n+not detach the HEAD.\n '\n \n . ./test-lib.sh\n@@ -305,6 +306,20 @@ test_expect_success 'submodule update --merge staying on master' '\n \t)\n '\n \n+test_expect_success 'submodule update --reset-hard staying on master' '\n+\t(cd super/submodule &&\n+\t git reset --hard HEAD~1\n+\t) &&\n+\t(cd super &&\n+\t (cd submodule &&\n+\t compare_head\n+\t ) &&\n+\t git submodule update --reset-hard submodule &&\n+\t cd submodule &&\n+\t compare_head\n+\t)\n+'\n+\n test_expect_success 'submodule update - rebase in .git/config' '\n \t(cd super &&\n \t git config submodule.submodule.update rebase\n-- \n2.20.0.rc2.8.g0a3bec4a1c.dirty\n\n"},{"id":"364968","messageId":"CAGZ79kYDa27EFk4A9uEzCnoW7scjb1U8fKwCo0P7rUZESto+Qg@mail.gmail.com","threadId":"49961","inReplyTo":"20181208042139.GA4827@hopa.kiewit.dartmouth.edu","subject":"Re: [wishlist] git submodule update --reset-hard","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-12-10T18:58:18Z","receivedAt":"2018-12-10T18:58:33Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Dec 7, 2018 at 8:21 PM Yaroslav Halchenko wrote:\n>\n>\n> On Fri, 07 Dec 2018, Yaroslav Halchenko wrote:\n>\n>\n> > On Fri, 07 Dec 2018, Stefan Beller wrote:\n> > > > the initial \"git submodule update --reset-hard\" is pretty much a\n> > > > crude workaround for some of those cases, so I would just go earlier in\n> > > > the history, and redo some things, whenever I could just drop or revert\n> > > > some selected set of commits.\n>\n> > > That makes sense.\n> > > Do you want to give the implementation a try for the --reset-hard switch?\n>\n> > ok, will do, thanks for the blessing ;-)\n>\n> The patch is attached (please advise if should be done\n> differently) and also submitted as PR\n> https://github.com/git/git/pull/563\n\nYes, usually we send patches inline\n(Random example:\nhttps://public-inbox.org/git/244bdf2a6fc300f2b535ac8edfc2fbdaf5260266.1544465177.git.gitgitgadget@gmail.com/T/#u\ncompared to https://public-inbox.org/git/20181208042139.GA4827@hopa.kiewit.dartmouth.edu/\n(which I am replying to))\n\nSee Documentation/SubmittingPatches.\n\nThere are some tools that provide a GithubPR -> emailPatch workflow at\nhttps://github.com/gitgitgadget/git\nI think if you'd open your pull request there, then it would be automatically\nmailed to the list correctly.\n\nI left some comments on the PR.\n\n>\n> I guess it would need more tests.\n\nWriting tests is hard, as we don't know what we expect to break. ;-)\n\n> Took me some time to figure out\n> why I was getting\n>\n> fatal: bad value for update parameter\n>\n> after all my changes to the git-submodule.sh script after looking at an\n> example commit 42b491786260eb17d97ea9fb1c4b70075bca9523 which introduced\n> --merge to the update ;-)\n\nYeah I saw you also updated the submodule related C code, was that\nfatal message related to that?\n\nThanks,\nStefan\n"},{"id":"364974","messageId":"835918B7-3AD7-40C8-B438-D267E3879B51@onerussian.com","threadId":"49961","inReplyTo":"CAGZ79kYDa27EFk4A9uEzCnoW7scjb1U8fKwCo0P7rUZESto+Qg@mail.gmail.com","subject":"Re: [wishlist] git submodule update --reset-hard","fromName":"Yaroslav Halchenko","fromEmail":"yoh@onerussian.com","sentAt":"2018-12-10T20:14:27Z","receivedAt":"2018-12-10T20:14:45Z","isPatch":false,"sender":{"key":"yoh@onerussian.com","avatar":"https://gravatar.com/avatar/8901b82415ae451a83aea49409708912726e53620e3ac92320bf1f86548d97e9?d=mp&s=160"},"body":"\n\n\n>\n>> Took me some time to figure out\n>> why I was getting\n>>\n>> fatal: bad value for update parameter\n>>\n>> after all my changes to the git-submodule.sh script after looking at\n>an\n>> example commit 42b491786260eb17d97ea9fb1c4b70075bca9523 which\n>introduced\n>> --merge to the update ;-)\n>\n>Yeah I saw you also updated the submodule related C code, was that\n>fatal message related to that?\n\nYes\n\n-- \nYaroslav O. Halchenko (mobile version)\nCenter for Open Neuroscience http://centerforopenneuroscience.org\nDartmouth College, NH, USA\n"},{"id":"365023","messageId":"20181211040839.17472-1-debian@onerussian.com","threadId":"49961","inReplyTo":"CAGZ79kYDa27EFk4A9uEzCnoW7scjb1U8fKwCo0P7rUZESto+Qg@mail.gmail.com","subject":"[PATCH 1/2] submodule: Add --reset-hard option for git submodule update","fromName":"Yaroslav Halchenko","fromEmail":"debian@onerussian.com","sentAt":"2018-12-11T04:08:38Z","receivedAt":"2018-12-11T04:08:50Z","isPatch":true,"sender":{"key":"debian@onerussian.com","avatar":"https://gravatar.com/avatar/9d2f005048a0274a9c26bc47a51f580e2bf631dfedfdb18370f65e08e4250317?d=mp&s=160"},"body":"This patch adds a --reset-hard option for the update command to hard\nreset submodule(s) to the gitlink for the corresponding submodule in\nthe superproject. This feature is desired e.g. to be able to discard\nrecent changes in the entire hierarchy of the submodules after running\n\n git reset --hard PREVIOUS_STATE\n\nin the superproject which leaves submodules in their original state,\nand\n\n git reset --hard --recurse-submodules PREVIOUS_STATE\n\nwould result in the submodules being checked into detached HEADs.\n\nAs in the original git reset --hard no checks or any kind of\nsafe-guards against jumping into some state which was never on the\ncurrent branch is done.\n\nmust_die_on_failure is not set to yes to mimic behavior of a update\n--checkout strategy, which would leave user with a non-clean state\nimmediately apparent via git status so an informed decision/actions\ncould be done manually.\n\nSigned-off-by: Yaroslav Halchenko \n---\n Documentation/git-submodule.txt | 12 +++++++++++-\n Documentation/gitmodules.txt | 4 ++--\n builtin/submodule--helper.c | 3 ++-\n git-submodule.sh | 10 +++++++++-\n submodule.c | 4 ++++\n submodule.h | 1 +\n t/t7406-submodule-update.sh | 17 ++++++++++++++++-\n 7 files changed, 45 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt\nindex ba3c4df550..f4e0483997 100644\n--- a/Documentation/git-submodule.txt\n+++ b/Documentation/git-submodule.txt\n@@ -124,7 +124,7 @@ If you really want to remove a submodule from the repository and commit\n that use linkgit:git-rm[1] instead. See linkgit:gitsubmodules[7] for removal\n options.\n \n-update [--init] [--remote] [-N|--no-fetch] [--[no-]recommend-shallow] [-f|--force] [--checkout|--rebase|--merge] [--reference ] [--depth ] [--recursive] [--jobs ] [--] [...]::\n+update [--init] [--remote] [-N|--no-fetch] [--[no-]recommend-shallow] [-f|--force] [--checkout|--rebase|--merge|--reset-hard] [--reference ] [--depth ] [--recursive] [--jobs ] [--] [...]::\n +\n --\n Update the registered submodules to match what the superproject\n@@ -358,6 +358,16 @@ the submodule itself.\n \tIf the key `submodule.$name.update` is set to `rebase`, this option is\n \timplicit.\n \n+--reset-hard::\n+\tThis option is only valid for the update command.\n+\tHard reset current state to the commit recorded in the\tsuperproject.\n+\tIf this option is given, the submodule's HEAD will not get detached\n+\tif it was not detached before. Note that, like with a regular\n+\t`git reset --hard` no safe-guards are in place to prevent jumping\n+\tto a commit which was never part of the current branch.\n+\tIf the key `submodule.$name.update` is set to `reset-hard`, this\n+\toption is implicit.\n+\n --init::\n \tThis option is only valid for the update command.\n \tInitialize all submodules for which \"git submodule init\" has not been\ndiff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt\nindex 312b6f9259..e085dbc01f 100644\n--- a/Documentation/gitmodules.txt\n+++ b/Documentation/gitmodules.txt\n@@ -43,8 +43,8 @@ submodule..update::\n \tcommand in the superproject. This is only used by `git\n \tsubmodule init` to initialize the configuration variable of\n \tthe same name. Allowed values here are 'checkout', 'rebase',\n-\t'merge' or 'none'. See description of 'update' command in\n-\tlinkgit:git-submodule[1] for their meaning. Note that the\n+\t'merge', 'reset-hard' or 'none'. See description of 'update' command\n+\tin linkgit:git-submodule[1] for their meaning. Note that the\n \t'!command' form is intentionally ignored here for security\n \treasons.\n \ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex d38113a31a..31d95c3cd6 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -1481,6 +1481,7 @@ static void determine_submodule_update_strategy(struct repository *r,\n \tif (just_cloned &&\n \t (out->type == SM_UPDATE_MERGE ||\n \t out->type == SM_UPDATE_REBASE ||\n+\t out->type == SM_UPDATE_RESET_HARD ||\n \t out->type == SM_UPDATE_NONE))\n \t\tout->type = SM_UPDATE_CHECKOUT;\n \n@@ -1851,7 +1852,7 @@ static int update_clone(int argc, const char **argv, const char *prefix)\n \t\t\t \"submodule boundaries\")),\n \t\tOPT_STRING(0, \"update\", &update,\n \t\t\t N_(\"string\"),\n-\t\t\t N_(\"rebase, merge, checkout or none\")),\n+\t\t\t N_(\"rebase, merge, checkout, reset-hard or none\")),\n \t\tOPT_STRING_LIST(0, \"reference\", &suc.references, N_(\"repo\"),\n \t\t\t N_(\"reference repository\")),\n \t\tOPT_BOOL(0, \"dissociate\", &suc.dissociate,\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 5e608f8bad..b5d6fad983 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -9,7 +9,7 @@ USAGE=\"[--quiet] add [-b ] [-f|--force] [--name ] [--reference ...]\n or: $dashless [--quiet] init [--] [...]\n or: $dashless [--quiet] deinit [-f|--force] (--all| [--] ...)\n- or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--[no-]recommend-shallow] [--reference ] [--recursive] [--] [...]\n+ or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase|--reset-hard] [--[no-]recommend-shallow] [--reference ] [--recursive] [--] [...]\n or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...]\n or: $dashless [--quiet] foreach [--recursive] \n or: $dashless [--quiet] sync [--recursive] [--] [...]\n@@ -483,6 +483,9 @@ cmd_update()\n \t\t-m|--merge)\n \t\t\tupdate=\"merge\"\n \t\t\t;;\n+\t\t--reset-hard)\n+\t\t\tupdate=\"reset-hard\"\n+\t\t\t;;\n \t\t--recursive)\n \t\t\trecursive=1\n \t\t\t;;\n@@ -621,6 +624,11 @@ cmd_update()\n \t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': merged in '\\$sha1'\")\"\n \t\t\t\tmust_die_on_failure=yes\n \t\t\t\t;;\n+\t\t\treset-hard)\n+\t\t\t\tcommand=\"git reset --hard\"\n+\t\t\t\tdie_msg=\"$(eval_gettext \"Unable to reset --hard to '\\$sha1' in submodule path '\\$displaypath'\")\"\n+\t\t\t\tsay_msg=\"$(eval_gettext \"Submodule path '\\$displaypath': was reset --hard to '\\$sha1'\")\"\n+\t\t\t\t;;\n \t\t\t!*)\n \t\t\t\tcommand=\"${update_module#!}\"\n \t\t\t\tdie_msg=\"$(eval_gettext \"Execution of '\\$command \\$sha1' failed in submodule path '\\$displaypath'\")\"\ndiff --git a/submodule.c b/submodule.c\nindex 6415cc5580..4580cf0944 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -373,6 +373,8 @@ enum submodule_update_type parse_submodule_update_type(const char *value)\n \t\treturn SM_UPDATE_REBASE;\n \telse if (!strcmp(value, \"merge\"))\n \t\treturn SM_UPDATE_MERGE;\n+\telse if (!strcmp(value, \"reset-hard\"))\n+\t\treturn SM_UPDATE_RESET_HARD;\n \telse if (*value == '!')\n \t\treturn SM_UPDATE_COMMAND;\n \telse\n@@ -406,6 +408,8 @@ const char *submodule_strategy_to_string(const struct submodule_update_strategy\n \t\treturn \"checkout\";\n \tcase SM_UPDATE_MERGE:\n \t\treturn \"merge\";\n+\tcase SM_UPDATE_RESET_HARD:\n+\t\treturn \"reset-hard\";\n \tcase SM_UPDATE_REBASE:\n \t\treturn \"rebase\";\n \tcase SM_UPDATE_NONE:\ndiff --git a/submodule.h b/submodule.h\nindex a680214c01..f23ac4630e 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -29,6 +29,7 @@ enum submodule_update_type {\n \tSM_UPDATE_CHECKOUT,\n \tSM_UPDATE_REBASE,\n \tSM_UPDATE_MERGE,\n+\tSM_UPDATE_RESET_HARD,\n \tSM_UPDATE_NONE,\n \tSM_UPDATE_COMMAND\n };\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex e87164aa8f..2e08e0047c 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -6,7 +6,8 @@\n test_description='Test updating submodules\n \n This test verifies that \"git submodule update\" detaches the HEAD of the\n-submodule and \"git submodule update --rebase/--merge\" does not detach the HEAD.\n+submodule and \"git submodule update --rebase/--merge/--reset-hard\" does\n+not detach the HEAD.\n '\n \n . ./test-lib.sh\n@@ -305,6 +306,20 @@ test_expect_success 'submodule update --merge staying on master' '\n \t)\n '\n \n+test_expect_success 'submodule update --reset-hard staying on master' '\n+\t(cd super/submodule &&\n+\t git reset --hard HEAD~1\n+\t) &&\n+\t(cd super &&\n+\t (cd submodule &&\n+\t compare_head\n+\t ) &&\n+\t git submodule update --reset-hard submodule &&\n+\t cd submodule &&\n+\t compare_head\n+\t)\n+'\n+\n test_expect_success 'submodule update - rebase in .git/config' '\n \t(cd super &&\n \t git config submodule.submodule.update rebase\n-- \n2.20.0.rc2.8.g0a3bec4a1c.dirty\n\n"},{"id":"365024","messageId":"20181211040839.17472-2-debian@onerussian.com","threadId":"49961","inReplyTo":"20181211040839.17472-1-debian@onerussian.com","subject":"[PATCH 2/2] RF+ENH(TST): compare the entire list of submodule status --recursive to stay intact","fromName":"Yaroslav Halchenko","fromEmail":"debian@onerussian.com","sentAt":"2018-12-11T04:08:39Z","receivedAt":"2018-12-11T04:08:59Z","isPatch":true,"sender":{"key":"debian@onerussian.com","avatar":"https://gravatar.com/avatar/9d2f005048a0274a9c26bc47a51f580e2bf631dfedfdb18370f65e08e4250317?d=mp&s=160"},"body":"For submodule update --reset-hard the best test is comparison of the\nentire status as shown by submodule status --recursive. Upon update\n--reset-hard we should get back to the original state, with all the\nbranches being the same (no detached HEAD) and commits identical to\noriginal (so no merges, new commits, etc).\n\nFor that, I have introduced two helpers: {record,compare}_submodules_status and\nan additional test for --reset-hard in nested submodule.\n\nI have kept this as a separate PATCH to demonstrate the diff from the original\ntest composition as introduced in the prior patch, and this one where\nall tests could be of the same type:\n\n record_submodule_status &&\n perform evil actions &&\n ! compare_submodule_status && # to double check that evil was done\n git submodule --reset-hard . &&\n compare_submodule_status # assure that we are all good\n\nSigned-off-by: Yaroslav Halchenko \n---\n t/t7406-submodule-update.sh | 37 ++++++++++++++++++++++++++++++-------\n 1 file changed, 30 insertions(+), 7 deletions(-)\n\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex 2e08e0047c..1927424f47 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -21,6 +21,17 @@ compare_head()\n test \"$sha_master\" = \"$sha_head\"\n }\n \n+record_submodules_status()\n+{\n+\tgit submodule status --recursive >expect\n+}\n+\n+compare_submodules_status()\n+{\n+\tgit submodule status --recursive >actual &&\n+\ttest_i18ncmp expect actual\n+}\n+\n \n test_expect_success 'setup a submodule tree' '\n \techo file > file &&\n@@ -294,7 +305,7 @@ test_expect_success 'submodule update --rebase staying on master' '\n \n test_expect_success 'submodule update --merge staying on master' '\n \t(cd super/submodule &&\n-\t git reset --hard HEAD~1\n+\t git reset --hard HEAD~1\n \t) &&\n \t(cd super &&\n \t (cd submodule &&\n@@ -307,16 +318,28 @@ test_expect_success 'submodule update --merge staying on master' '\n '\n \n test_expect_success 'submodule update --reset-hard staying on master' '\n-\t(cd super/submodule &&\n-\t git reset --hard HEAD~1\n-\t) &&\n \t(cd super &&\n+\t record_submodules_status &&\n \t (cd submodule &&\n-\t compare_head\n+\t git reset --hard HEAD~1\n \t ) &&\n+\t ! compare_submodules_status &&\n \t git submodule update --reset-hard submodule &&\n-\t cd submodule &&\n-\t compare_head\n+\t compare_submodules_status\n+\t)\n+'\n+\n+test_expect_success 'submodule update --reset-hard in nested submodule' '\n+\t(cd recursivesuper &&\n+\t git submodule update --init --recursive &&\n+\t record_submodules_status &&\n+\t (cd super/submodule &&\n+\t echo 123 >> file &&\n+\t git commit -m \"new commit\" file\n+\t ) &&\n+\t ! compare_submodules_status &&\n+\t git submodule update --reset-hard --recursive &&\n+\t compare_submodules_status\n \t)\n '\n \n-- \n2.20.0.rc2.8.g0a3bec4a1c.dirty\n\n"},{"id":"365195","messageId":"CAGZ79kY17gmEh5Sawa+1fG5cXjOReOgCjDyEmGbbpJ5EE1APdw@mail.gmail.com","threadId":"49961","inReplyTo":"20181211040839.17472-2-debian@onerussian.com","subject":"Re: [PATCH 2/2] RF+ENH(TST): compare the entire list of submodule status --recursive to stay intact","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-12-12T19:48:41Z","receivedAt":"2018-12-12T19:48:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Dec 10, 2018 at 8:09 PM Yaroslav Halchenko\n wrote:\n\nThanks for the patches. The first patch looks good to me!\n\n> [PATCH 2/2] RF+ENH(TST): compare the entire list of submodule status --recursive to stay intact\n\nThe subject is a bit cryptic (specifically the first part before the\ncolon), maybe\n\n t7406: compare entire submodule status for --reset-hard mode\n\n?\n\n\n> For submodule update --reset-hard the best test is comparison of the\n> entire status as shown by submodule status --recursive. Upon update\n> --reset-hard we should get back to the original state, with all the\n> branches being the same (no detached HEAD) and commits identical to\n> original (so no merges, new commits, etc).\n\n\"original state\" can mean different things to different people. I'd think\nwe could be more precise:\n\n ... we should get to the state that the submodule is reset to the\n object id as the superprojects gitlink points at, irrespective of the\n submodule branch.\n\n\n> test_expect_success 'submodule update --merge staying on master' '\n> (cd super/submodule &&\n> - git reset --hard HEAD~1\n> + git reset --hard HEAD~1\n\nunrelated white space change?\n\n> ) &&\n> (cd super &&\n> (cd submodule &&\n> @@ -307,16 +318,28 @@ test_expect_success 'submodule update --merge staying on master' '\n> '\n>\n> test_expect_success 'submodule update --reset-hard staying on master' '\n> [..]\n> +'\n> +\n\nThe tests look good to me, though I wonder if we'd rather want to inline\n{record/compare}_submodule_status as then you'd not need to look it up\nand the functions are rather short?\n"},{"id":"365274","messageId":"20181213164217.GA4633@hopa.kiewit.dartmouth.edu","threadId":"49961","inReplyTo":"CAGZ79kY17gmEh5Sawa+1fG5cXjOReOgCjDyEmGbbpJ5EE1APdw@mail.gmail.com","subject":"Re: [PATCH 2/2] RF+ENH(TST): compare the entire list of submodule status --recursive to stay intact","fromName":"Yaroslav O Halchenko","fromEmail":"debian@onerussian.com","sentAt":"2018-12-13T16:42:17Z","receivedAt":"2018-12-13T16:42:25Z","isPatch":true,"sender":{"key":"debian@onerussian.com","avatar":"https://gravatar.com/avatar/9d2f005048a0274a9c26bc47a51f580e2bf631dfedfdb18370f65e08e4250317?d=mp&s=160"},"body":"Thank you Stefan for the review and please pardon my delay with the\nreply, and sorry it got a bit too long by the end ;)\n\nOn Wed, 12 Dec 2018, Stefan Beller wrote:\n> Thanks for the patches. The first patch looks good to me!\n\nGreat!\n\n> > [PATCH 2/2] RF+ENH(TST): compare the entire list of submodule status --recursive to stay intact\n\n> The subject is a bit cryptic (specifically the first part before the\n> colon), maybe\n\n> t7406: compare entire submodule status for --reset-hard mode\n\n> ?\n\n\n> > For submodule update --reset-hard the best test is comparison of the\n> > entire status as shown by submodule status --recursive. Upon update\n> > --reset-hard we should get back to the original state, with all the\n> > branches being the same (no detached HEAD) and commits identical to\n> > original (so no merges, new commits, etc).\n\n> \"original state\" can mean different things to different people. I'd think\n> we could be more precise:\n\n> ... we should get to the state that the submodule is reset to the\n> object id as the superprojects gitlink points at, irrespective of the\n> submodule branch.\n\nok, I will update the description. But I wonder if there could be some\nshort term to be used to describe the composite \"git submodule status\"\nand \"git status\" (refers to below ;)).\n\n> > test_expect_success 'submodule update --merge staying on master' '\n> > (cd super/submodule &&\n> > - git reset --hard HEAD~1\n> > + git reset --hard HEAD~1\n\n> unrelated white space change?\n\nI was tuning formatting to be uniform and I guess missed that this is in\nthe other (not my) test. I will revert that piece, thanks!\n\nBTW -- should I just squash to PATCHes now? I kept them separate primarily to\nshow the use of those helpers:\n\n> > ) &&\n> > (cd super &&\n> > (cd submodule &&\n> > @@ -307,16 +318,28 @@ test_expect_success 'submodule update --merge staying on master' '\n> > '\n\n> > test_expect_success 'submodule update --reset-hard staying on master' '\n> > [..]\n> > +'\n> > +\n\n> The tests look good to me, though I wonder if we'd rather want to inline\n> {record/compare}_submodule_status as then you'd not need to look it up\n> and the functions are rather short?\n\ncompare_submodules_status is already a compound action, so code would\nbecome quite more \"loaded\" if it is expanded, e.g. instead of \n\n\t(cd super &&\n\t record_submodules_status &&\n\t (cd submodule &&\n\t git reset --hard HEAD~1\n\t ) &&\n\t ! compare_submodules_status &&\n\t git submodule update --reset-hard submodule &&\n\t compare_submodules_status\n\t)\n\nit would become something like this I guess?\n\n\t(cd super &&\n\t git submodule status --recursive >expect &&\n\t (cd submodule &&\n\t git reset --hard HEAD~1\n\t ) &&\n\t ! {git submodule status --recursive >actual && \n test_i18ncmp expect actual;} &&\n\t git submodule update --reset-hard submodule &&\n\t {git submodule status --recursive >actual && \n test_i18ncmp expect actual;}\n\t)\n\nIMHO a bit mouth full. I was thinking also to extend compare_ with additional\ntesting e.g. using \"git status\" since \"git submodule status\" does not care\nabout untracked files etc. For --reset-hard I would like to assure that it is\nnot just some kind of a mixed reset leaving files behind. That would make\ntests even more overloaded.\n\nOn that point: Although I also like explicit calls at times, I also do\nlike test fixtures as a concept to do more testing around the actual\ntest-specific code block, thus minimizing boiler plate, which even if explicit\nmakes code actually harder to grasp (at least to me). \n\nSince for the majority of the --reset-hard tests the fixture and test(s) are\npretty much the same, actually ideally I would have liked to have\nsomething like this:\n\ntest_expect_unchanged_submodule_status 'submodule update --reset-hard staying on master' \\\n super \\\n '(cd submodule && git reset --hard HEAD~1)' \\\n 'git submodule update --reset-hard submodule'\n\nwhere I just pass \n the path to work in, \n the test setup function, \n and the test action. \n\nThe rest (initial cd, record, run setup, verify that there is a change, run\naction, verify there is no changes) is done by the\ntest_expect_unchanged_submodule_status in a uniform way, absorbing all the\nboiler plate. (I am not married to the name, could be more descriptive/generic\nmay be)\n\nThen we could breed a good number of tests with little to no boiler plate, with\nonly relevant pieces and as extended as needed testing done by this\ntest_expect_unchanged_submodule_status helper. e.g smth like\n\ntest_expect_unchanged_submodule_status 'submodule update --reset-hard staying on master when I do a new commit' \\\n super \\\n '(cd submodule && git commit --allow-empty -m \"new one\"' \\\n 'git submodule update --reset-hard submodule'\n\nand kaboom -- we have a new test. If we decide to test more -- just tune up\ntest_expect_unchanged_submodule_status and done -- all the tests remain\nsufficiently prescribed.\n\nWhat do you think?\n-- \nYaroslav O. Halchenko\nCenter for Open Neuroscience http://centerforopenneuroscience.org\nDartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755\nPhone: +1 (603) 646-9834 Fax: +1 (603) 646-1419\nWWW: http://www.linkedin.com/in/yarik \n"},{"id":"365300","messageId":"CAGZ79kZb28bCvM7cOYHC4cpJWpA-3_gcbxS_g-rG0yy=9jXquw@mail.gmail.com","threadId":"49961","inReplyTo":"20181213164217.GA4633@hopa.kiewit.dartmouth.edu","subject":"Re: [PATCH 2/2] RF+ENH(TST): compare the entire list of submodule status --recursive to stay intact","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-12-13T20:44:19Z","receivedAt":"2018-12-13T20:44:34Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Dec 13, 2018 at 8:42 AM Yaroslav O Halchenko\n wrote:\n>\n> Thank you Stefan for the review and please pardon my delay with the\n> reply, and sorry it got a bit too long by the end ;)\n>\n> On Wed, 12 Dec 2018, Stefan Beller wrote:\n> > Thanks for the patches. The first patch looks good to me!\n>\n> Great!\n>\n> > > [PATCH 2/2] RF+ENH(TST): compare the entire list of submodule status --recursive to stay intact\n>\n> > The subject is a bit cryptic (specifically the first part before the\n> > colon), maybe\n>\n> > t7406: compare entire submodule status for --reset-hard mode\n>\n> > ?\n>\n>\n> > > For submodule update --reset-hard the best test is comparison of the\n> > > entire status as shown by submodule status --recursive. Upon update\n> > > --reset-hard we should get back to the original state, with all the\n> > > branches being the same (no detached HEAD) and commits identical to\n> > > original (so no merges, new commits, etc).\n>\n> > \"original state\" can mean different things to different people. I'd think\n> > we could be more precise:\n>\n> > ... we should get to the state that the submodule is reset to the\n> > object id as the superprojects gitlink points at, irrespective of the\n> > submodule branch.\n>\n> ok, I will update the description. But I wonder if there could be some\n> short term to be used to describe the composite \"git submodule status\"\n> and \"git status\" (refers to below ;)).\n>\n> > > test_expect_success 'submodule update --merge staying on master' '\n> > > (cd super/submodule &&\n> > > - git reset --hard HEAD~1\n> > > + git reset --hard HEAD~1\n>\n> > unrelated white space change?\n>\n> I was tuning formatting to be uniform and I guess missed that this is in\n> the other (not my) test. I will revert that piece, thanks!\n\nThe tests in that file are not quite following the coding style that is\ncurrently deemed the best. So if you want to clean that up\nas a preparatory patch, feel welcome to do so. :-)\n(c.f. t/t7400-submodule-basic.sh for good style, specifically\nindentation by tabs and the cd on its own line in\na subshell)\nThe latest style update I found is\n80938c39e2 (pack-objects test: modernize style, 2018-10-30)\nand submodule related test style\n31158c7efc (t7410: update to new style, 2018-08-15)\n\nSo I was not opposed to have style changes, but to have\nmultiple unrelated things in one patch (feature work vs\ncleanup).\n\n> BTW -- should I just squash to PATCHes now? I kept them separate primarily to\n> show the use of those helpers:\n\nThat would make sense.\n\n> compare_submodules_status is already a compound action, so code would\n> become quite more \"loaded\" if it is expanded, e.g. instead of\n...\n> it would become something like this I guess?\n...\n> ! {git submodule status --recursive >actual &&\n\nyou could keep the status out of the negation.\n\n> test_i18ncmp expect actual;} &&\n> git submodule update --reset-hard submodule &&\n> {git submodule status --recursive >actual &&\n> test_i18ncmp expect actual;}\n> )\n>\n> IMHO a bit mouth full. I was thinking also to extend compare_ with additional\n> testing e.g. using \"git status\" since \"git submodule status\" does not care\n> about untracked files etc. For --reset-hard I would like to assure that it is\n> not just some kind of a mixed reset leaving files behind. That would make\n> tests even more overloaded.\n\nok, that makes sense.\n\n> On that point: Although I also like explicit calls at times, I also do\n> like test fixtures as a concept to do more testing around the actual\n> test-specific code block, thus minimizing boiler plate, which even if explicit\n> makes code actually harder to grasp (at least to me).\n>\n> Since for the majority of the --reset-hard tests the fixture and test(s) are\n> pretty much the same, actually ideally I would have liked to have\n> something like this:\n>\n> test_expect_unchanged_submodule_status 'submodule update --reset-hard staying on master' \\\n> super \\\n> '(cd submodule && git reset --hard HEAD~1)' \\\n> 'git submodule update --reset-hard submodule'\n>\n> where I just pass\n> the path to work in,\n> the test setup function,\n> and the test action.\n>\n> The rest (initial cd, record, run setup, verify that there is a change, run\n> action, verify there is no changes) is done by the\n> test_expect_unchanged_submodule_status in a uniform way, absorbing all the\n> boiler plate. (I am not married to the name, could be more descriptive/generic\n> may be)\n\nThe issue with submodules is that we're already deviating from the\n'standard' git test suite at times (See the submodule test suite\nlib-submodule-update.sh that is used via t1013, t2013 or t3906\nand others).\n\nI guess if we keep the test_expect_unchanged_submodule_status\nas a file local function, it could be okay.\n\n> Then we could breed a good number of tests with little to no boiler plate, with\n> only relevant pieces and as extended as needed testing done by this\n> test_expect_unchanged_submodule_status helper. e.g smth like\n>\n> test_expect_unchanged_submodule_status 'submodule update --reset-hard staying on master when I do a new commit' \\\n> super \\\n> '(cd submodule && git commit --allow-empty -m \"new one\"' \\\n\nIn new tests we're a big fan of using -C, as that can save the\nsubshell, i.e. replace the whole line by\n\n git -C submodule commit --allow-empty -m \"new one\"' &&\n\n\n> 'git submodule update --reset-hard submodule'\n>\n> and kaboom -- we have a new test. If we decide to test more -- just tune up\n> test_expect_unchanged_submodule_status and done -- all the tests remain\n> sufficiently prescribed.\n>\n> What do you think?\n\nThat is pretty cool. Maybe my gut reaction on the previous patch\nalso had to do with the numbers, i.e. having 2 extra function for\nonly having 2 tests more legible. A framework is definitely better\nonce we have more tests.\n\nStefan\n"},{"id":"365311","messageId":"20181213224356.GI4633@hopa.kiewit.dartmouth.edu","threadId":"49961","inReplyTo":"CAGZ79kZb28bCvM7cOYHC4cpJWpA-3_gcbxS_g-rG0yy=9jXquw@mail.gmail.com","subject":"Re: [PATCH 2/2] RF+ENH(TST): compare the entire list of submodule status --recursive to stay intact","fromName":"Yaroslav O Halchenko","fromEmail":"debian@onerussian.com","sentAt":"2018-12-13T22:43:56Z","receivedAt":"2018-12-13T22:44:04Z","isPatch":true,"sender":{"key":"debian@onerussian.com","avatar":"https://gravatar.com/avatar/9d2f005048a0274a9c26bc47a51f580e2bf631dfedfdb18370f65e08e4250317?d=mp&s=160"},"body":"\nOn Thu, 13 Dec 2018, Stefan Beller wrote:\n\n> > and kaboom -- we have a new test. If we decide to test more -- just tune up\n> > test_expect_unchanged_submodule_status and done -- all the tests remain\n> > sufficiently prescribed.\n\n> > What do you think?\n\n> That is pretty cool. Maybe my gut reaction on the previous patch\n> also had to do with the numbers, i.e. having 2 extra function for\n> only having 2 tests more legible. A framework is definitely better\n> once we have more tests.\n\ncool, thanks for the feedback - I will then try to make it happen\n\nquick one (so when I get to it I know): should I replicate all those\ntests you have for other update strategies? (treating of config\nspecifications etc) There is no easy way to parametrize them somehow?\n;) In Python world I might have mocked the actual underlying call to\nupdate, to see what option it would be getting and assure that it is the\none I specified via config, and then sweepped through all of them\nto make sure nothing interim changes it. Just wondering if may be\nsomething like that exists in git's tests support.\n\n\nBTW - sorry if RTFM and unrelated, is there a way to \n\n update --merge \n\nbut allowing only fast-forwards? My use case is collection of this\nsubmodules: http://datasets.datalad.org/?dir=/openneuro which all\nshould come from github and I should not have any changes of my own.\nSure thing if all is clean etc, merge should result in fast-forward. I\njust do not want to miss a case where there was some (temporary?) \"dirt\"\nwhich I forgot to reset and it would then get merged etc.\n\n\n-- \nYaroslav O. Halchenko\nCenter for Open Neuroscience http://centerforopenneuroscience.org\nDartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755\nPhone: +1 (603) 646-9834 Fax: +1 (603) 646-1419\nWWW: http://www.linkedin.com/in/yarik \n"},{"id":"365312","messageId":"CAGZ79kaLpMa+AYwtNh+HUkYk3ORDephxZ74hcTbS=z1CQ+bf6A@mail.gmail.com","threadId":"49961","inReplyTo":"20181213224356.GI4633@hopa.kiewit.dartmouth.edu","subject":"Re: [PATCH 2/2] RF+ENH(TST): compare the entire list of submodule status --recursive to stay intact","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-12-13T23:58:51Z","receivedAt":"2018-12-14T00:05:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Dec 13, 2018 at 2:44 PM Yaroslav O Halchenko\n wrote:\n>\n>\n> On Thu, 13 Dec 2018, Stefan Beller wrote:\n>\n> > > and kaboom -- we have a new test. If we decide to test more -- just tune up\n> > > test_expect_unchanged_submodule_status and done -- all the tests remain\n> > > sufficiently prescribed.\n>\n> > > What do you think?\n>\n> > That is pretty cool. Maybe my gut reaction on the previous patch\n> > also had to do with the numbers, i.e. having 2 extra function for\n> > only having 2 tests more legible. A framework is definitely better\n> > once we have more tests.\n>\n> cool, thanks for the feedback - I will then try to make it happen\n>\n> quick one (so when I get to it I know): should I replicate all those\n> tests you have for other update strategies? (treating of config\n> specifications etc)\n\nIf there is a sensible way to do so?\nI have the impression that there are enough differences, that it\nmay not be possible to replicate all tests meaningfully from the\nother modes.\n\n> There is no easy way to parametrize them somehow?\n\nThere is t/lib-submodule-update.sh, which brings this to\nan extreme, as it makes a \"test suite in a test suite\"; and I would\nnot follow that example for this change.\n\n> ;) In Python world I might have mocked the actual underlying call to\n> update, to see what option it would be getting and assure that it is the\n> one I specified via config, and then sweepped through all of them\n> to make sure nothing interim changes it. Just wondering if may be\n> something like that exists in git's tests support.\n\ngits tests are very heavy on end to end testing, i.e. run a whole command\nand observe its output. This makes our command setup code, (i.e. finding\nthe repository, parsing options, reading possible config, etc) a really well\nexercised code path. ;-)\n\nThere is a recent push towards testing only units, most of\nt/helper is used for that, e.g. c.f. 4c7bb45269 (test-reach:\ntest get_reachable_subset, 2018-11-02).\n\nSo if you have a good idea how to focus the submodule\ntests more on the (new) unit that you add, that would be cool.\n\n> BTW - sorry if RTFM and unrelated, is there a way to\n>\n> update --merge\n>\n> but allowing only fast-forwards? My use case is collection of this\n> submodules: http://datasets.datalad.org/?dir=/openneuro which all\n> should come from github and I should not have any changes of my own.\n\nSo you want the merge option --ff-only\nto be passed to the submodule merge command. I guess you could make\na patch, that update takes another option (--ff-only, only useful when\n--merge is given), which is then propagated.\n\nI am not sure if we could have a more generalized option passing,\nwhich would allow to pass any option (for its respective command)\nto the command that is run in the update mode.\n\n> Sure thing if all is clean etc, merge should result in fast-forward. I\n> just do not want to miss a case where there was some (temporary?) \"dirt\"\n> which I forgot to reset and it would then get merged etc.\n\nmaybe use --rebase, such that your potential change would bubble\nup and possibly produce a merge conflict?\n"},{"id":"365326","messageId":"20181214042205.GJ4633@hopa.kiewit.dartmouth.edu","threadId":"49961","inReplyTo":"CAGZ79kaLpMa+AYwtNh+HUkYk3ORDephxZ74hcTbS=z1CQ+bf6A@mail.gmail.com","subject":"Re: [PATCH 2/2] RF+ENH(TST): compare the entire list of submodule status --recursive to stay intact","fromName":"Yaroslav O Halchenko","fromEmail":"debian@onerussian.com","sentAt":"2018-12-14T04:22:05Z","receivedAt":"2018-12-14T04:22:13Z","isPatch":true,"sender":{"key":"debian@onerussian.com","avatar":"https://gravatar.com/avatar/9d2f005048a0274a9c26bc47a51f580e2bf631dfedfdb18370f65e08e4250317?d=mp&s=160"},"body":"\nOn Thu, 13 Dec 2018, Stefan Beller wrote:\n\n> > cool, thanks for the feedback - I will then try to make it happen\n\n> > quick one (so when I get to it I know): should I replicate all those\n> > tests you have for other update strategies? (treating of config\n> > specifications etc)\n\n> If there is a sensible way to do so?\n> I have the impression that there are enough differences, that it\n> may not be possible to replicate all tests meaningfully from the\n> other modes.\n\noh, by replicate I just meant to copy/paste and adjust for expected for\n--reset-hard test behavior (and possibly introduced helper),\nnothing fancy, just duplication as for replication ;-) \n\n> > There is no easy way to parametrize them somehow?\n\n> There is t/lib-submodule-update.sh, which brings this to\n> an extreme, as it makes a \"test suite in a test suite\"; and I would\n> not follow that example for this change.\n\nok\n\n> > ;) In Python world I might have mocked the actual underlying call to\n> > update, to see what option it would be getting and assure that it is the\n> > one I specified via config, and then sweepped through all of them\n> > to make sure nothing interim changes it. Just wondering if may be\n> > something like that exists in git's tests support.\n\n> gits tests are very heavy on end to end testing, i.e. run a whole command\n> and observe its output. This makes our command setup code, (i.e. finding\n> the repository, parsing options, reading possible config, etc) a really well\n> exercised code path. ;-)\n\n> There is a recent push towards testing only units, most of\n> t/helper is used for that, e.g. c.f. 4c7bb45269 (test-reach:\n> test get_reachable_subset, 2018-11-02).\n\n> So if you have a good idea how to focus the submodule\n> tests more on the (new) unit that you add, that would be cool.\n\nno, not really any good ideas -- I am new here, but I will keep an eye open.\n\n> > BTW - sorry if RTFM and unrelated, is there a way to\n\n> > update --merge\n\n> > but allowing only fast-forwards? My use case is collection of this\n> > submodules: http://datasets.datalad.org/?dir=/openneuro which all\n> > should come from github and I should not have any changes of my own.\n\n> So you want the merge option --ff-only\n> to be passed to the submodule merge command. I guess you could make\n> a patch, that update takes another option (--ff-only, only useful when\n> --merge is given), which is then propagated.\n\n> I am not sure if we could have a more generalized option passing,\n> which would allow to pass any option (for its respective command)\n> to the command that is run in the update mode.\n\nwouldn't it be (theoretically) possible, in principle, to pass\nthem via some config variable? e.g. instead of \n\nsubmodule update --reset-hard\n\nhave\n\n-c submodule.update.reset.opts=--hard update --reset\n\nand then analogously\n\n-c submodule.update.merge.opts=--ff-only update --merge\n\n(--ff-only I guess would make no sense for any \"supermodule\" - a repo\nwith submodules)\n\n> > Sure thing if all is clean etc, merge should result in fast-forward. I\n> > just do not want to miss a case where there was some (temporary?) \"dirt\"\n> > which I forgot to reset and it would then get merged etc.\n\n> maybe use --rebase, such that your potential change would bubble\n> up and possibly produce a merge conflict?\n\nthat is a good idea as a workaround, thanks!\n\n-- \nYaroslav O. Halchenko\nCenter for Open Neuroscience http://centerforopenneuroscience.org\nDartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755\nPhone: +1 (603) 646-9834 Fax: +1 (603) 646-1419\nWWW: http://www.linkedin.com/in/yarik \n"}]}