{"thread":{"id":"30488","subject":"submodule update --force","startedAt":"2012-05-09T23:55:40Z","lastAt":"2012-05-14T20:16:15Z","messageCount":12,"participants":["Stefan Zager","Junio C Hamano","Heiko Voigt","Phil Hord"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"191255","messageId":"CAHOQ7J9xCYL=x=_nbq-3ksC2nF7L0=kxu9JX6M60xM-Bxmyfag@mail.gmail.com","threadId":"30488","inReplyTo":"CAHOQ7J8r4m2rz57BdkM9CADHdHE1yDFwExyF87u=DCEXjqzcqw@mail.gmail.com","subject":"submodule update --force","fromName":"Stefan Zager","fromEmail":"szager@google.com","sentAt":"2012-05-09T23:55:40Z","receivedAt":"2012-05-09T23:55:40Z","isPatch":false,"sender":{"key":"szager@google.com","avatar":null},"body":"I have a situation where I have a full source tree -- top-level\nrepository and all submodules -- generated via `git clone -n`.  So,\nthe directory structure and .git directories are intact, but no actual\nsource files have been checked out.\n\nIf I run `git submodule update` from the top level, some submodules\nget checked out, but not others.  Weird.\n\nRoot cause is in the cmd_update function in git-submodule.sh, which\ndoes essentially this:\n\nsha1 = submodule revision as registered in top-level module\nsubsha1 = HEAD revision in submodule checkout\n\nif test \"$subsha1\" != \"$sha1\"\nthen\n    git checkout $sha1\nfi\n\nIn the submodule checkouts, HEAD is refs/heads/master.  In *some* of\nthe submodules, the revision registered in the top-level repository is\nthe same as HEAD.  So, for those submodules, `git submodule update`\nrun from the top level is a no-op, because $sha1 = $subsha1.  That's\ntrue even though there are no actual source files checked out in the\nsubmodule.\n\nFor other submodules, $sha1 != $subsha1, and `git submodule update`\nchecks out the source code as expected.\n\nConfusing!\n\nAccording to the docs for git-submodule:\n\n       -f, --force\n           This option is only valid for add and update commands. When\nrunning add, allow adding an otherwise ignored\n           submodule path. When running update, throw away local\nchanges in submodules when switching to a different\n           commit.\n\nI'd like to propose amending the documentation thusly:\n\nAccording to the docs:\n\n       -f, --force\n           This option is only valid for add and update commands. When\nrunning add, allow adding an otherwise ignored\n           submodule path. When running update, throw away local\nchanges in submodules when switching to a different\n           commit; if not switching to a different commit, a checkout\nto HEAD will still be run.\n\n... and here's the patch to implement it:\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 64a70d6..8b045d9 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -536,7 +536,7 @@ Maybe you want to use 'update --init'?\")\"\n                        die \"$(eval_gettext \"Unable to find current\nrevision in submodule path '\\$sm_path'\")\"\n                fi\n\n-               if test \"$subsha1\" != \"$sha1\"\n+               if test \"$subsha1\" != \"$sha1\" -o -n \"$force\"\n                then\n                        subforce=$force\n                        # If we don't already have a -f flag and the\nsubmodule has never been checked out\n\n\n\nThoughts?\n\nThanks,\n\nStefan\n"},{"id":"191258","messageId":"7vobpwpoyi.fsf@alter.siamese.dyndns.org","threadId":"30488","inReplyTo":"CAHOQ7J9xCYL=x=_nbq-3ksC2nF7L0=kxu9JX6M60xM-Bxmyfag@mail.gmail.com","subject":"Re: submodule update --force","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-05-10T05:47:17Z","receivedAt":"2012-05-10T05:47:17Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Zager <szager@google.com> writes:\n\n> I'd like to propose amending the documentation thusly:\n>\n> According to the docs:\n>\n>        -f, --force\n>            This option is only valid for add and update commands. When\n> running add, allow adding an otherwise ignored\n>            submodule path. When running update, throw away local\n> changes in submodules when switching to a different\n>            commit; if not switching to a different commit, a checkout\n> to HEAD will still be run.\n>\n> ... and here's the patch to implement it:\n>\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 64a70d6..8b045d9 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -536,7 +536,7 @@ Maybe you want to use 'update --init'?\")\"\n>                         die \"$(eval_gettext \"Unable to find current\n> revision in submodule path '\\$sm_path'\")\"\n>                 fi\n>\n> -               if test \"$subsha1\" != \"$sha1\"\n> +               if test \"$subsha1\" != \"$sha1\" -o -n \"$force\"\n>                 then\n>                         subforce=$force\n>                         # If we don't already have a -f flag and the\n> submodule has never been checked out\n>\n> Thoughts?\n\nEven though I admit that I do not use submodule heavily myself, I think\nthis is a sane thing to do.  After all, the user explicitly said \"I want\nto force update it\", and it is a strong sign that what is in the working\ntree is suspect and the user wants to make sure everything is in sync.\n\nThis is a tangent, but what strikes me odd with the code before this patch\nis that the decision to recurse into the submodule repository is made\nsolely on the status of the submodule, and there is no way for the user to\nsay \"I do not want it to recurse\" (in other words, \"--recursive\" option\nfrom the command line does not have any effect on this part of the code).\n\nPerhaps that is because we consider submodules that have been \"init\"ed\nalways interesting, and if that is the case that may not be a big deal,\nbut it might not be a bad idea to allow \"--no-recursive\" option to mean\nsomething stronger than not giving --recursive option, i.e. not recursing\nin a situation where it normally would even when run without --recursive.\n"},{"id":"191269","messageId":"7vk40kpnia.fsf@alter.siamese.dyndns.org","threadId":"30488","inReplyTo":"7vobpwpoyi.fsf@alter.siamese.dyndns.org","subject":"Re: submodule update --force","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-05-10T06:18:37Z","receivedAt":"2012-05-10T06:18:37Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Stefan Zager <szager@google.com> writes:\n>\n>> I'd like to propose amending the documentation thusly:\n>>\n>> According to the docs:\n>>\n>>        -f, --force\n>>            This option is only valid for add and update commands. When\n>> running add, allow adding an otherwise ignored\n>>            submodule path. When running update, throw away local\n>> changes in submodules when switching to a different\n>>            commit; if not switching to a different commit, a checkout\n>> to HEAD will still be run.\n>>\n>> ... and here's the patch to implement it:\n>>\n>> diff --git a/git-submodule.sh b/git-submodule.sh\n>> index 64a70d6..8b045d9 100755\n>> --- a/git-submodule.sh\n>> +++ b/git-submodule.sh\n>> @@ -536,7 +536,7 @@ Maybe you want to use 'update --init'?\")\"\n>>                         die \"$(eval_gettext \"Unable to find current\n>> revision in submodule path '\\$sm_path'\")\"\n>>                 fi\n>>\n>> -               if test \"$subsha1\" != \"$sha1\"\n>> +               if test \"$subsha1\" != \"$sha1\" -o -n \"$force\"\n>>                 then\n>>                         subforce=$force\n>>                         # If we don't already have a -f flag and the\n>> submodule has never been checked out\n>>\n>> Thoughts?\n>\n> Even though I admit that I do not use submodule heavily myself, I think\n> this is a sane thing to do.  After all, the user explicitly said \"I want\n> to force update it\", and it is a strong sign that what is in the working\n> tree is suspect and the user wants to make sure everything is in sync.\n>\n> This is a tangent, but what strikes me odd with the code before this patch\n> is that the decision to recurse into the submodule repository is made\n> solely on the status of the submodule, and there is no way for the user to\n> say \"I do not want it to recurse\" (in other words, \"--recursive\" option\n> from the command line does not have any effect on this part of the code).\n>\n> Perhaps that is because we consider submodules that have been \"init\"ed\n> always interesting, and if that is the case that may not be a big deal,\n> but it might not be a bad idea to allow \"--no-recursive\" option to mean\n> something stronger than not giving --recursive option, i.e. not recursing\n> in a situation where it normally would even when run without --recursive.\n\nAnd similarly, it might be a good idea to make the presense of \"--recursive\"\na bit stronger than the command line that did not say either \"--recursive\"\nnor \"--no-recursive\", so that a subcommand that normally inspects the\nstate of the submodule and decide to recurse (or not) can be told to\nalways recurse into it.\n\nIf we follow that line of thought, it may make more sense not to implement\nyour feature like the above patch, but instead make it so\n\n\tif the user told us never to recurse\n        then\n\t\tnothing\n\telif the user told us to always recurse ||\n             subsha1 != sha1\n        then\n\t\tdo the \"recurse\" thing\n\tfi\n\nso that you can still force it recurse into the submodule, even when you\ndo not necessarily want the \"force checkout\" thing to happen to clobber\nthe working tree.\n"},{"id":"191272","messageId":"CAHOQ7J_6+sfU6egjvVSPj-FAS6zjSUT=a057=kz_wYbogHLMMA@mail.gmail.com","threadId":"30488","inReplyTo":"7vk40kpnia.fsf@alter.siamese.dyndns.org","subject":"Re: submodule update --force","fromName":"Stefan Zager","fromEmail":"szager@google.com","sentAt":"2012-05-10T07:20:19Z","receivedAt":"2012-05-10T07:20:19Z","isPatch":false,"sender":{"key":"szager@google.com","avatar":null},"body":"On Wed, May 9, 2012 at 11:18 PM, Junio C Hamano <gitster@pobox.com> wrote:\n\n> If we follow that line of thought, it may make more sense not to implement\n> your feature like the above patch, but instead make it so\n>\n>        if the user told us never to recurse\n>        then\n>                nothing\n>        elif the user told us to always recurse ||\n>             subsha1 != sha1\n>        then\n>                do the \"recurse\" thing\n>        fi\n>\n> so that you can still force it recurse into the submodule, even when you\n> do not necessarily want the \"force checkout\" thing to happen to clobber\n> the working tree.\n\nI'm a bit confused by your use of the term recursion here.  It sounds\nlike you consider any operation run on a submodule to be a form of\nrecursion, which is not the way I think about it.  To my mind, any\n`git submodule` command should *always* run on the first level of\nsubmodules.  If you're going to specify --no-recurse, then why are you\nrunning `git submodule` at all?  I think 'recursion' only applies to\nmoving beyond the first level of submodules.\n\nI think your solution would work for our project, because we only use\none level of submodules.  But IMO the functionality, as you have\nwritten it, is less generally useful.\n\nStefan\n"},{"id":"191299","messageId":"7v8vh0ozge.fsf@alter.siamese.dyndns.org","threadId":"30488","inReplyTo":"CAHOQ7J_6+sfU6egjvVSPj-FAS6zjSUT=a057=kz_wYbogHLMMA@mail.gmail.com","subject":"Re: submodule update --force","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-05-10T14:58:09Z","receivedAt":"2012-05-10T14:58:09Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Zager <szager@google.com> writes:\n\n> ...  To my mind, any\n> `git submodule` command should *always* run on the first level of\n> submodules.  If you're going to specify --no-recurse, then why are you\n> running `git submodule` at all?  I think 'recursion' only applies to\n> moving beyond the first level of submodules.\n\nVery true.\n\nSubmodule folks, any opinion on the Stefan's approach?\n"},{"id":"191337","messageId":"20120510185738.GE76400@book.hvoigt.net","threadId":"30488","inReplyTo":"7v8vh0ozge.fsf@alter.siamese.dyndns.org","subject":"Re: submodule update --force","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2012-05-10T18:57:38Z","receivedAt":"2012-05-10T18:57:38Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"Hallo all,\n\nOn Thu, May 10, 2012 at 07:58:09AM -0700, Junio C Hamano wrote:\n> Stefan Zager <szager@google.com> writes:\n> \n> > ...  To my mind, any\n> > `git submodule` command should *always* run on the first level of\n> > submodules.  If you're going to specify --no-recurse, then why are you\n> > running `git submodule` at all?  I think 'recursion' only applies to\n> > moving beyond the first level of submodules.\n> \n> Very true.\n> \n> Submodule folks, any opinion on the Stefan's approach?\n\nThe distinction between first level of submodules and deeper is only\npresent in the \"git submodule\" subcommand and I think mainly for\nhistorical reasons. I do not see a use case where this would be helpful.\nTo skip uninteresting submodules one can always use the\nsubmodule.$name.update option set to 'none'. (I just found that its\ndocumentation is in the wrong place but I will send a seperate patch\nabout that).\n\nIn the non submodule commands we usually name this option\n--recurse-submodules=always and have another\n--recurse-submodules=on-demand option for the current behavior. Those\noptions would either recurse or do nothing with the submodule. Such a\nbehavior, as pointed out, does not make sense for 'submodule update'.\nSimilar options names for 'submodule update' would probably be\n--recurse=always and --recurse=on-demand.\n\nNonetheless is force a term where the user probably wants to skip all\noptimizations which the sha1 equality provides. So to make the current\nbehavior more consistent I would be fine with adding this change.\n\nOne thing which might make force even more useful would be to also skip\nthe \"is the sha1 available\"-check for fetch that is possibly run before\nthe checkout and just always run the fetch.\n\nIn the long term, once checkout has learned things 'submodule update' is\ncurrently doing, it probably makes sense to let 'submodule update'\nalways recurse into all checked out submodules. Since then it does not\nmake sense to run 'submodule update' for much more than resetting\nthings or changing the currently registered commits anymore. So in the\nbright new future the 'on-demand' part will probably move away from\n'submodule update' and as such it does not make sense to implement\nthe seperate recurse options I described above.\n\nWhat do others think?\n\nCheers Heiko\n"},{"id":"191418","messageId":"CABURp0rFQ+330X8g3C2rmozQ77zxqhZhReZhaYMi1FE4uKeQtA@mail.gmail.com","threadId":"30488","inReplyTo":"20120510185738.GE76400@book.hvoigt.net","subject":"Re: submodule update --force","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2012-05-11T20:56:07Z","receivedAt":"2012-05-11T20:56:07Z","isPatch":false,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"On Thu, May 10, 2012 at 2:57 PM, Heiko Voigt <hvoigt@hvoigt.net> wrote:\n>\n> Hallo all,\n>\n> On Thu, May 10, 2012 at 07:58:09AM -0700, Junio C Hamano wrote:\n> > Stefan Zager <szager@google.com> writes:\n> >\n> > > ...  To my mind, any\n> > > `git submodule` command should *always* run on the first level of\n> > > submodules.  If you're going to specify --no-recurse, then why are you\n> > > running `git submodule` at all?  I think 'recursion' only applies to\n> > > moving beyond the first level of submodules.\n> >\n> > Very true.\n> >\n> > Submodule folks, any opinion on the Stefan's approach?\n>\n> The distinction between first level of submodules and deeper is only\n> present in the \"git submodule\" subcommand and I think mainly for\n> historical reasons. I do not see a use case where this would be helpful.\n\nDo I understand you to mean that you think the git-submodule ...\n--recursive option is archaic?  I would agree that one might expect it\nto be the default option, but I do not think it should be deprecated\nin any way.\n\n> To skip uninteresting submodules one can always use the\n> submodule.$name.update option set to 'none'. (I just found that its\n> documentation is in the wrong place but I will send a seperate patch\n> about that).\n>\n> In the non submodule commands we usually name this option\n> --recurse-submodules=always and have another\n> --recurse-submodules=on-demand option for the current behavior. Those\n> options would either recurse or do nothing with the submodule.  Such a\n> behavior, as pointed out, does not make sense for 'submodule update'.\n> Similar options names for 'submodule update' would probably be\n> --recurse=always and --recurse=on-demand.\n>\n> Nonetheless is force a term where the user probably wants to skip all\n> optimizations which the sha1 equality provides. So to make the current\n> behavior more consistent I would be fine with adding this change.\n>\n> One thing which might make force even more useful would be to also skip\n> the \"is the sha1 available\"-check for fetch that is possibly run before\n> the checkout and just always run the fetch.\n>\n> In the long term, once checkout has learned things 'submodule update' is\n> currently doing, it probably makes sense to let 'submodule update'\n> always recurse into all checked out submodules. Since then it does not\n> make sense to run 'submodule update' for much more than resetting\n> things or changing the currently registered commits anymore. So in the\n> bright new future the 'on-demand' part will probably move away from\n> 'submodule update' and as such it does not make sense to implement\n> the seperate recurse options I described above.\n>\n> What do others think?\n\nI think there are three cases:\n\n1. I want to update any sha1-mismatching submodules so\n    their HEAD matches the superproject gitlink.\n\n    git submodule update\n\n2. Same as (1) above, but also check out files for all\n    submodules which are not already checked out.\n\n    git submodule update &&\n    git submodule foreach 'git checkout HEAD || :'\n\n3. I want to update exactly to the gitlinks in the superproject\n   and discard any local or staged changes.\n\n    git submodule update -f\n\n(2) above is the case Junio was trying to cover.  I cannot think of an\nelegant name for the switch for such an option, but I would be\nsurprised it to find it is not the default behavior if I also\nencountered it like Stefan did.  We should try to eliminate surprises\nto help dispel the notion that submodules are unwieldy.\n\n(3) is too heavy when I really only wanted (2).\n\nI do not understand that use case that led Stefan to the predicament\nhe was in where he had submodules with HEADs but with no checked out\nfiles.  But I do not begrudge his being there.\n\n\nRegards,\nPhil\n"},{"id":"191505","messageId":"20120514165231.GB58058@book.hvoigt.net","threadId":"30488","inReplyTo":"CABURp0rFQ+330X8g3C2rmozQ77zxqhZhReZhaYMi1FE4uKeQtA@mail.gmail.com","subject":"Re: submodule update --force","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2012-05-14T16:52:32Z","receivedAt":"2012-05-14T16:52:32Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"Hi,\n\nOn Fri, May 11, 2012 at 04:56:07PM -0400, Phil Hord wrote:\n> On Thu, May 10, 2012 at 2:57 PM, Heiko Voigt <hvoigt@hvoigt.net> wrote:\n> > On Thu, May 10, 2012 at 07:58:09AM -0700, Junio C Hamano wrote:\n> > > Stefan Zager <szager@google.com> writes:\n> > >\n> > > > ... ?To my mind, any\n> > > > `git submodule` command should *always* run on the first level of\n> > > > submodules. ?If you're going to specify --no-recurse, then why are you\n> > > > running `git submodule` at all? ?I think 'recursion' only applies to\n> > > > moving beyond the first level of submodules.\n> > >\n> > > Very true.\n> > >\n> > > Submodule folks, any opinion on the Stefan's approach?\n> >\n> > The distinction between first level of submodules and deeper is only\n> > present in the \"git submodule\" subcommand and I think mainly for\n> > historical reasons. I do not see a use case where this would be helpful.\n> \n> Do I understand you to mean that you think the git-submodule ...\n> --recursive option is archaic?  I would agree that one might expect it\n> to be the default option, but I do not think it should be deprecated\n> in any way.\n\nIn a way yes its archaic since I do not know why one would distinguish\nbetween the first level of populated submodules and below. For example\nif you have nested submodules and want them all be populated you need to\nuse\n\n\tgit submodule update --init --recursive\n\nThe sequence\n\n\tgit submodule init --recursive\n\tgit submodule update --recursive\n\ndoes not do the same thing but would have to be called multiple times\nuntil you have reached the deepest level. IMO that is confusing but not\nonly a problem of this option.\n\n> > To skip uninteresting submodules one can always use the\n> > submodule.$name.update option set to 'none'. (I just found that its\n> > documentation is in the wrong place but I will send a seperate patch\n> > about that).\n> >\n> > In the non submodule commands we usually name this option\n> > --recurse-submodules=always and have another\n> > --recurse-submodules=on-demand option for the current behavior. Those\n> > options would either recurse or do nothing with the submodule.  Such a\n> > behavior, as pointed out, does not make sense for 'submodule update'.\n> > Similar options names for 'submodule update' would probably be\n> > --recurse=always and --recurse=on-demand.\n> >\n> > Nonetheless is force a term where the user probably wants to skip all\n> > optimizations which the sha1 equality provides. So to make the current\n> > behavior more consistent I would be fine with adding this change.\n> >\n> > One thing which might make force even more useful would be to also skip\n> > the \"is the sha1 available\"-check for fetch that is possibly run before\n> > the checkout and just always run the fetch.\n> >\n> > In the long term, once checkout has learned things 'submodule update' is\n> > currently doing, it probably makes sense to let 'submodule update'\n> > always recurse into all checked out submodules. Since then it does not\n> > make sense to run 'submodule update' for much more than resetting\n> > things or changing the currently registered commits anymore. So in the\n> > bright new future the 'on-demand' part will probably move away from\n> > 'submodule update' and as such it does not make sense to implement\n> > the seperate recurse options I described above.\n> >\n> > What do others think?\n> \n> I think there are three cases:\n> \n> 1. I want to update any sha1-mismatching submodules so\n>     their HEAD matches the superproject gitlink.\n> \n>     git submodule update\n> \n> 2. Same as (1) above, but also check out files for all\n>     submodules which are not already checked out.\n> \n>     git submodule update &&\n>     git submodule foreach 'git checkout HEAD || :'\n> \n> 3. I want to update exactly to the gitlinks in the superproject\n>    and discard any local or staged changes.\n> \n>     git submodule update -f\n> \n> (2) above is the case Junio was trying to cover.  I cannot think of an\n> elegant name for the switch for such an option, but I would be\n> surprised it to find it is not the default behavior if I also\n> encountered it like Stefan did.  We should try to eliminate surprises\n> to help dispel the notion that submodules are unwieldy.\n\nYes we should eliminate surprises thats true. On the other hand there is\nno way to setup submodules in the way Stefan had them by using the git\nsubmodule command or is there? So for his use case the command sequence\nyou described seems to be more appropriate but I am not sure whether\nthat justifies a separate option for it.\n\n> (3) is too heavy when I really only wanted (2).\n> \n> I do not understand that use case that led Stefan to the predicament\n> he was in where he had submodules with HEADs but with no checked out\n> files.  But I do not begrudge his being there.\n\nYes, but currently -f is wrong in the way that when the submodules HEAD\nsha1 is the same as registered in the superproject it will skip the\ncheckout. That is wrong when you have local uncommitted changes in the\nworktree. In such a state I would expect it to throw away those local\nchanges and checkout HEAD. So I think Stefans patch makes sense anyway\neven though it might actually be to heavy for his use case.\n\nCheers Heiko\n"},{"id":"191507","messageId":"CAHOQ7J_O=8NL0wp0Pu6pfjN_Y6NDJhKZUft9G2FL0vUWL7aEBw@mail.gmail.com","threadId":"30488","inReplyTo":"20120514165231.GB58058@book.hvoigt.net","subject":"Re: submodule update --force","fromName":"Stefan Zager","fromEmail":"szager@google.com","sentAt":"2012-05-14T17:17:53Z","receivedAt":"2012-05-14T17:17:53Z","isPatch":false,"sender":{"key":"szager@google.com","avatar":null},"body":"On Mon, May 14, 2012 at 9:52 AM, Heiko Voigt <hvoigt@hvoigt.net> wrote:\n\n> Yes we should eliminate surprises thats true. On the other hand there is\n> no way to setup submodules in the way Stefan had them by using the git\n> submodule command or is there? So for his use case the command sequence\n> you described seems to be more appropriate but I am not sure whether\n> that justifies a separate option for it.\n>\n>> (3) is too heavy when I really only wanted (2).\n>>\n>> I do not understand that use case that led Stefan to the predicament\n>> he was in where he had submodules with HEADs but with no checked out\n>> files.  But I do not begrudge his being there.\n>\n> Yes, but currently -f is wrong in the way that when the submodules HEAD\n> sha1 is the same as registered in the superproject it will skip the\n> checkout. That is wrong when you have local uncommitted changes in the\n> worktree. In such a state I would expect it to throw away those local\n> changes and checkout HEAD. So I think Stefans patch makes sense anyway\n> even though it might actually be to heavy for his use case.\n\nTo satisfy your curiosity, although it probably won't help me make my\ncase: I'm working on very large project with a lot of commit history.\nCloning this repository is prohibitively slow, so I'm trying to speed\nit up by periodically creating a snapshot of the repository (and all\nsubmodule repositories) that can be downloaded and cloned locally.\n\nI first tried using `git bundle`, but cloning from the bundle files\nwas still very slow, because it took a long time to replay all the\ncommit history to recreate the index.  So, I hit upon another\nsolution: run `git clone -n` on the top-level repository and all\nsubmodule repositories, and create a zip file of the empty checkout.\nThe resulting zip file is the same size as the git bundle file, but\nunpacking it on the client (basically, `unzip` followed by `git\ncheckout HEAD`) is much faster.\n\nStefan\n"},{"id":"191520","messageId":"20120514191802.GE58058@book.hvoigt.net","threadId":"30488","inReplyTo":"CAHOQ7J_O=8NL0wp0Pu6pfjN_Y6NDJhKZUft9G2FL0vUWL7aEBw@mail.gmail.com","subject":"Re: submodule update --force","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2012-05-14T19:18:02Z","receivedAt":"2012-05-14T19:18:02Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"Hi,\n\nOn Mon, May 14, 2012 at 10:17:53AM -0700, Stefan Zager wrote:\n> On Mon, May 14, 2012 at 9:52 AM, Heiko Voigt <hvoigt@hvoigt.net> wrote:\n> \n> > Yes we should eliminate surprises thats true. On the other hand there is\n> > no way to setup submodules in the way Stefan had them by using the git\n> > submodule command or is there? So for his use case the command sequence\n> > you described seems to be more appropriate but I am not sure whether\n> > that justifies a separate option for it.\n> >\n> >> (3) is too heavy when I really only wanted (2).\n> >>\n> >> I do not understand that use case that led Stefan to the predicament\n> >> he was in where he had submodules with HEADs but with no checked out\n> >> files. ?But I do not begrudge his being there.\n> >\n> > Yes, but currently -f is wrong in the way that when the submodules HEAD\n> > sha1 is the same as registered in the superproject it will skip the\n> > checkout. That is wrong when you have local uncommitted changes in the\n> > worktree. In such a state I would expect it to throw away those local\n> > changes and checkout HEAD. So I think Stefans patch makes sense anyway\n> > even though it might actually be to heavy for his use case.\n> \n> To satisfy your curiosity, although it probably won't help me make my\n> case: I'm working on very large project with a lot of commit history.\n> Cloning this repository is prohibitively slow, so I'm trying to speed\n> it up by periodically creating a snapshot of the repository (and all\n> submodule repositories) that can be downloaded and cloned locally.\n> \n> I first tried using `git bundle`, but cloning from the bundle files\n> was still very slow, because it took a long time to replay all the\n> commit history to recreate the index.  So, I hit upon another\n> solution: run `git clone -n` on the top-level repository and all\n> submodule repositories, and create a zip file of the empty checkout.\n> The resulting zip file is the same size as the git bundle file, but\n> unpacking it on the client (basically, `unzip` followed by `git\n> checkout HEAD`) is much faster.\n\nFor this use case running\n\n\tgit submodule update -f\n\nis just fine. The only objection I had was against a new option. So I am\nall for making -f skip this sha1 comparing optimization like your patch\ndid.\n\nOn a side note: I am surprised that cloning through git is really that\nmuch slower than copying a zip from the network. Do you run git gc\nregularly enough on the server?\n\nCheers Heiko\n"},{"id":"191521","messageId":"CAHOQ7J8Wq+jHdghgNEGD+7aWNCv3rpPu1erCK4V_pu3sgXQLxA@mail.gmail.com","threadId":"30488","inReplyTo":"20120514191802.GE58058@book.hvoigt.net","subject":"Re: submodule update --force","fromName":"Stefan Zager","fromEmail":"szager@google.com","sentAt":"2012-05-14T19:29:02Z","receivedAt":"2012-05-14T19:29:02Z","isPatch":false,"sender":{"key":"szager@google.com","avatar":null},"body":"On Mon, May 14, 2012 at 12:18 PM, Heiko Voigt <hvoigt@hvoigt.net> wrote:\n\n> On a side note: I am surprised that cloning through git is really that\n> much slower than copying a zip from the network. Do you run git gc\n> regularly enough on the server?\n\nDon't know about running `git gc`, but I can tell you that the\n'resolving deltas' step on the client side is very slow; anecdotally,\nit appears to take longer than the network transfer.  We also would\nlike to relieve the server of the burden of creating large pack files\nmany times a day (the repo is frequently cloned).  I'll check to see\nwhether `git gc` can help us.\n\nThanks,\n\nStefan\n"},{"id":"191526","messageId":"20120514201615.GI58058@book.hvoigt.net","threadId":"30488","inReplyTo":"CAHOQ7J8Wq+jHdghgNEGD+7aWNCv3rpPu1erCK4V_pu3sgXQLxA@mail.gmail.com","subject":"Re: submodule update --force","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2012-05-14T20:16:15Z","receivedAt":"2012-05-14T20:16:15Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Mon, May 14, 2012 at 12:29:02PM -0700, Stefan Zager wrote:\n> On Mon, May 14, 2012 at 12:18 PM, Heiko Voigt <hvoigt@hvoigt.net> wrote:\n> \n> > On a side note: I am surprised that cloning through git is really that\n> > much slower than copying a zip from the network. Do you run git gc\n> > regularly enough on the server?\n> \n> Don't know about running `git gc`, but I can tell you that the\n> 'resolving deltas' step on the client side is very slow; anecdotally,\n> it appears to take longer than the network transfer.  We also would\n> like to relieve the server of the burden of creating large pack files\n> many times a day (the repo is frequently cloned).  I'll check to see\n> whether `git gc` can help us.\n\nI do not think you need to do this many times a day a cronjob during the\nnight should be sufficient. Maybe also do a gc --aggressive one a week?\n\nCheers Heiko\n"}]}