{"thread":{"id":"28896","subject":"[RFC/PATCH] add update to branch support for \"floating submodules\"","startedAt":"2011-11-09T17:40:27Z","lastAt":"2012-02-06T21:32:07Z","messageCount":27,"participants":["Heiko Voigt","Junio C Hamano","Leif Gruenwoldt","Jonathan Nieder","Gioele Barabucci","Andreas T.Auer","Jens Lehmann","Phil Hord","Brandon Casey","Marc Branchaud"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"179213","messageId":"20111109174027.GA28825@book.fritz.box","threadId":"28896","inReplyTo":null,"subject":"[RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2011-11-09T17:40:27Z","receivedAt":"2011-11-09T17:40:27Z","isPatch":true,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"This adds the capability to configure a branch which submodule update\nwill use to checkout the tips sha1 instead of the registered one.\n\nIt will first attempt to read the configuration directly from the\ncurrently checked out .gitmodules file from the key\nsubmodule.$name.branch. This configuration can be overridden by local\nuser configuration values. The parameter --branch can be used to\nspecify/override the branch using the commandline. The parameter\n--checkout can be used to switch to the exact model for all submodules.\n\nSuch a thing is helpful if a user wants to follow a defined branches tip\nin the submodule. Image such a branch is the stable branch for some\ncentral library or similar.\n\nWhen the newly checked out tip will not match the registered sha1 in the\nsuperproject it will show up as a change as usual. You can imagine this\nas a configuration which lets the upstream project tell a user the\nbranch it usually updates to. The usual revision control is still in\nplace.\n---\n\nThis is almost ready but I would like to know what users of the\n\"floating submodule\" think about this.\n\n Documentation/git-submodule.txt | 26 +++++++++--\n git-submodule.sh | 47 ++++++++++++++++++++\n t/t7406-submodule-update.sh | 93 +++++++++++++++++++++++++++++++++++++++\n 3 files changed, 162 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt\nindex 6ec3fef..b8affa3 100644\n--- a/Documentation/git-submodule.txt\n+++ b/Documentation/git-submodule.txt\n@@ -133,9 +133,11 @@ init::\n update::\n \tUpdate the registered submodules, i.e. clone missing submodules and\n \tcheckout the commit specified in the index of the containing repository.\n-\tThis will make the submodules HEAD be detached unless `--rebase` or\n-\t`--merge` is specified or the key `submodule.$name.update` is set to\n-\t`rebase`, `merge` or `none`.\n+\tThis will make the submodules HEAD be detached. This will not\n+\thappen if `--rebase`, `--merge` or `--branch` are specified.\n+\tAlso if the key `submodule.$name.update` is set to `rebase`,\n+\t`merge` or `none`. If `submodule.$name.branch` is set to some\n+\tlocal branch this will also not happen.\n +\n If the submodule is not yet initialized, and you just want to use the\n setting as stored in .gitmodules, you can automatically initialize the\n@@ -146,7 +148,16 @@ registered submodules, and update any nested submodules within.\n +\n If the configuration key `submodule.$name.update` is set to `none` the\n submodule with name `$name` will not be updated by default. This can be\n-overriden by adding `--checkout` to the command.\n+overriden by adding `--checkout` to the command. `--checkout` can also\n+be used to enforce exact checkout of submodule sha1's.\n++\n+If the configuration key `submodule.$name.branch` is set to some valid\n+branch in the submodule named by `$name` the submodule will be updated\n+to the tip of that branch instead of the registered sha1. This option\n+can either be set in .gitmodules or via git's configuration. Gits local\n+configuration takes precedence over .gitmodules. If you want to override\n+the branch checkout you can use the value `HEAD` to tell git to checkout\n+exactly the registered sha1.\n \n summary::\n \tShow commit summary between the given commit (defaults to HEAD) and\n@@ -252,6 +263,13 @@ OPTIONS\n \tIf the key `submodule.$name.update` is set to `rebase`, this option is\n \timplicit.\n \n+--branch::\n+\tThis option is only valid for the update command. You can use\n+\tthis parameter to specify which branch you want to update all\n+\tsubmodules to. This is helpful if you want to update all\n+\tsubmodules to the tip of a certain branch or need to work on a\n+\tbranch for all submodules for some time.\n+\n --init::\n \tThis option is only valid for the update command.\n \tInitialize all submodules for which \"git submodule init\" has not been\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 3adab93..a4b117b 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -465,6 +465,14 @@ cmd_update()\n \t\t--checkout)\n \t\t\tupdate=\"checkout\"\n \t\t\t;;\n+\t\t--branch=*)\n+\t\t\tcase \"$1\" in\n+\t\t\t*=*)\n+\t\t\t\tupdate=\"checkout\"\n+\t\t\t\tbranch=`expr \"z$1\" : 'z--[^=]*=\\(.*\\)'` ;;\n+\t\t\t*)\n+\t\t\t\tusage ;;\n+\t\t\tesac ;;\n \t\t--)\n \t\t\tshift\n \t\t\tbreak\n@@ -504,6 +512,45 @@ cmd_update()\n \t\t\tupdate_module=$(git config submodule.\"$name\".update)\n \t\tfi\n \n+\t\tif ! test -z \"$branch\"\n+\t\tthen\n+\t\t\tbranch_module=$branch\n+\t\telse\n+\t\t\tif test \"$update\" != \"checkout\"\n+\t\t\tthen\n+\t\t\t\tbranch_module=$(git config submodule.\"$name\".branch)\n+\t\t\t\tif test -z \"$branch_module\"\n+\t\t\t\tthen\n+\t\t\t\t\tbranch_module=$(git config -f .gitmodules --get submodule.\"$name\".branch)\n+\t\t\t\tfi\n+\t\t\tfi\n+\t\tfi\n+\n+\t\tif test \"$branch_module\" = \"HEAD\"\n+\t\tthen\n+\t\t\tbranch_module=\n+\t\tfi\n+\n+\t\tif ! test -z \"$branch_module\"\n+\t\tthen\n+\t\t\t(clear_local_git_env; cd \"$path\" &&\n+\t\t\t if test ! $nofetch\n+\t\t\t then\n+\t\t\t\tgit-fetch --all >/dev/null 2>/dev/null || exit 1\n+\t\t\t fi) ||\n+\t\t\tdie \"$(eval_gettext \"Unable to fetch submodule in path '\\$path'\")\"\n+\n+\t\t\tsha1=$(clear_local_git_env; cd \"$path\" &&\n+\t\t\t\tgit rev-parse $branch_module) ||\n+\t\t\tsay \"$(eval_gettext \"Unable to find branch '\\$branch_module' in submodule path '\\$path'\")\"\n+\t\tfi\n+\n+\t\tif test \"$branch\" -a \"$update\" != \"checkout\"\n+\t\tthen\n+\t\t\tdie \"$(eval_gettext \"You can not set update='\\$update' and\n+use a branch for submodule '\\$path'\")\"\n+\t\tfi\n+\n \t\tif test \"$update_module\" = \"none\"\n \t\tthen\n \t\t\techo \"Skipping submodule '$path'\"\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex 33b292b..517ed83 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -611,4 +611,97 @@ test_expect_success 'submodule update places git-dir in superprojects git-dir re\n \t)\n '\n \n+test_expect_success '--branch updates follow the given branch' '\n+\tgit clone . branch &&\n+\t(cd branch &&\n+\t\tgit submodule add ./submodule submodule1 &&\n+\t\tgit submodule add ./submodule submodule2 &&\n+\t\t(cd submodule1 &&\n+\t\t\tgit rev-parse HEAD >../expected1 &&\n+\t\t\tgit checkout HEAD^) &&\n+\t\t(cd submodule2 &&\n+\t\t\tgit rev-parse HEAD >../expected2 &&\n+\t\t\tgit checkout HEAD^) &&\n+\t\tgit add submodule1 &&\n+\t\tgit add submodule2 &&\n+\t\tgit commit -m \"add submodule1 and submodule2\" &&\n+\t\tgit submodule update --branch=origin/master &&\n+\t\t(cd submodule1 && git rev-parse HEAD >../actual1) &&\n+\t\t(cd submodule2 && git rev-parse HEAD >../actual2) &&\n+\t\ttest_cmp expected1 actual1 &&\n+\t\ttest_cmp expected2 actual2\n+\t)\n+'\n+\n+cat >branch/expect_status <../expected1 &&\n+\t\tgit rev-parse HEAD >../actual1) &&\n+\t(cd submodule2 &&\n+\t\tgit rev-parse origin/master^ >../expected2 &&\n+\t\tgit rev-parse HEAD >../actual2) &&\n+\ttest_cmp expected1 actual1 &&\n+\ttest_cmp expected2 actual2 &&\n+\tgit status --porcelain --untracked-files=no >actual_status &&\n+\ttest_cmp expect_status actual_status\n+}\n+\n+check_both_submodules_exact()\n+{\n+\t(cd submodule1 &&\n+\t\tgit rev-parse origin/master^ >../expected1 &&\n+\t\tgit rev-parse HEAD >../actual1) &&\n+\t(cd submodule2 &&\n+\t\tgit rev-parse origin/master^ >../expected2 &&\n+\t\tgit rev-parse HEAD >../actual2) &&\n+\ttest_cmp expected1 actual1 &&\n+\ttest_cmp expected2 actual2 &&\n+\ttest -z \"$(git status --porcelain --untracked-files=no)\"\n+}\n+\n+test_expect_success 'local branch configuration follows branch' '\n+\t(cd branch &&\n+\t\tgit submodule update &&\n+\t\tcheck_both_submodules_exact &&\n+\t\tgit config submodule.submodule1.branch origin/master &&\n+\t\tgit submodule update &&\n+\t\tcheck_submodule_one_follows\n+\t)\n+'\n+\n+test_expect_success '.gitmodules branch configuration follows branch' '\n+\t(cd branch &&\n+\t\tgit config --unset submodule.submodule1.branch &&\n+\t\tgit submodule update &&\n+\t\tcheck_both_submodules_exact &&\n+\t\tgit config -f .gitmodules submodule.submodule1.branch origin/master &&\n+\t\tgit add .gitmodules &&\n+\t\tgit commit -m \".gitmodules follows branch\" &&\n+\t\tgit submodule update &&\n+\t\tcheck_submodule_one_follows\n+\t)\n+'\n+\n+test_expect_success '--checkout commandline overrides branch config' '\n+\t(cd branch &&\n+\t\tgit submodule update --checkout &&\n+\t\tcheck_both_submodules_exact\n+\t)\n+'\n+\n+test_expect_success 'local config overrides .gitmodules branch config' '\n+\t(cd branch &&\n+\t\tgit submodule update &&\n+\t\tcheck_submodule_one_follows &&\n+\t\tgit config submodule.submodule1.branch HEAD &&\n+\t\tgit submodule update &&\n+\t\tcheck_both_submodules_exact\n+\t)\n+'\n+\n test_done\n-- \n1.7.7.433.gcf1e7\n"},{"id":"179214","messageId":"7vr51htbsy.fsf@alter.siamese.dyndns.org","threadId":"28896","inReplyTo":"20111109174027.GA28825@book.fritz.box","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-09T18:01:33Z","receivedAt":"2011-11-09T18:01:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Heiko Voigt writes:\n\n> This is almost ready but I would like to know what users of the\n> \"floating submodule\" think about this.\n\nThanks for working on this.\n\nI do like to hear from potential users as well, because the general\nimpression we got was that floating submodules is not a real need of\nanybody, but it is merely an inertia of people who (perhaps mistakenly)\nthought svn externals that are not anchored to a particular revision is a\nfeature when it is just a limitation in reality. During the GitTogether'11\nwe learned that Android that uses floating model does not really have to.\n"},{"id":"180122","messageId":"20111129220854.GB2812@sandbox-rc.fritz.box","threadId":"28896","inReplyTo":"7vr51htbsy.fsf@alter.siamese.dyndns.org","subject":"Re: Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2011-11-29T22:08:55Z","receivedAt":"2011-11-29T22:08:55Z","isPatch":true,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"Hi,\n\nOn Wed, Nov 09, 2011 at 10:01:33AM -0800, Junio C Hamano wrote:\n> Heiko Voigt writes:\n> \n> > This is almost ready but I would like to know what users of the\n> > \"floating submodule\" think about this.\n> \n> Thanks for working on this.\n> \n> I do like to hear from potential users as well, because the general\n> impression we got was that floating submodules is not a real need of\n> anybody, but it is merely an inertia of people who (perhaps mistakenly)\n> thought svn externals that are not anchored to a particular revision is a\n> feature when it is just a limitation in reality. During the GitTogether'11\n> we learned that Android that uses floating model does not really have to.\n\nSince we did not get any reply from potential floating submodule users I\ndo not mind to drop this patch for now. It is archived in the mailing list\nand it should be easy to revive once there is real world need for it.\n\nOnce we have the \"exact\" model support for checkout and friends this\nmight be a handy tool to update submodules before releases and such. But\ncurrently I would like to focus on the \"exact\" front first.\n\nCheers Heiko\n"},{"id":"180743","messageId":"loom.20111210T062013-538@post.gmane.org","threadId":"28896","inReplyTo":"20111129220854.GB2812@sandbox-rc.fritz.box","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Leif Gruenwoldt","fromEmail":"leifer@gmail.com","sentAt":"2011-12-10T05:50:59Z","receivedAt":"2011-12-10T05:50:59Z","isPatch":true,"sender":{"key":"leifer@gmail.com","avatar":"https://gravatar.com/avatar/5c17a063e2df05a22275a8cf0e00e16a75c1ce4f9a59970be15eafedd50629d1?d=mp&s=160"},"body":"Heiko Voigt hvoigt.net> writes:\n\n> \n> Hi,\n> \n> On Wed, Nov 09, 2011 at 10:01:33AM -0800, Junio C Hamano wrote:\n> > Heiko Voigt hvoigt.net> writes:\n> > \n> > > This is almost ready but I would like to know what users of the\n> > > \"floating submodule\" think about this.\n> > \n> > Thanks for working on this.\n> > \n> > I do like to hear from potential users as well, because the general\n> > impression we got was that floating submodules is not a real need of\n> > anybody, but it is merely an inertia of people who (perhaps mistakenly)\n> > thought svn externals that are not anchored to a particular revision is a\n> > feature when it is just a limitation in reality. During the GitTogether'11\n> > we learned that Android that uses floating model does not really have to.\n> \n> Since we did not get any reply from potential floating submodule users I\n> do not mind to drop this patch for now. It is archived in the mailing list\n> and it should be easy to revive once there is real world need for it.\n> \n> Once we have the \"exact\" model support for checkout and friends this\n> might be a handy tool to update submodules before releases and such. But\n> currently I would like to focus on the \"exact\" front first.\n> \n> Cheers Heiko\n> \n\nIf I understand the description of \"floating submodules\", it's something I have \nbeen wanting for a while now! The lack of it is currently a deal breaker for \nusing submodules within my organisation.\n\nOur use case is as follows. We have several repositories for our common code \n(commonA.git, commonB.git, etc) and a few different products that leverage these \ncommon repos (productA.git, productB.git, etc). When one of the products is in \nheavy development we often need to do a lot of work in the common repos. Having \nto increment the sha1 of the submodules to track the latest tip would be overly \narduous. (Obviously when development of the product stabilizes we would want to \nchange to anchoring to a specific sha1 in the common repos).\n"},{"id":"180745","messageId":"20111210061906.GA11326@elie.hsd1.il.comcast.net","threadId":"28896","inReplyTo":"loom.20111210T062013-538@post.gmane.org","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-12-10T06:19:06Z","receivedAt":"2011-12-10T06:19:06Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"(restoring cc list)\nHi Leif,\n\nLeif Gruenwoldt wrote:\n\n> If I understand the description of \"floating submodules\", it's something I have \n> been wanting for a while now! The lack of it is currently a deal breaker for \n> using submodules within my organisation.\n>\n> Our use case is as follows.\n[...]\n> When one of the products is in \n> heavy development we often need to do a lot of work in the common repos. Having \n> to increment the sha1 of the submodules to track the latest tip would be overly \n> arduous.\n\nWhat happens when a bug was introduced in this period of heavy development\nand someone wants to look back in the development history and build each\nversion to find which introduced the bug?\n\nIf I were part of such a project, I would be tempted to follow one of two\nrules. Either\n\n A. Each commit of productA strives to work with the latest version of\n the common code possible. Which version of the common code that was\n tested against gets recorded (perhaps by some record-submodule-versions-\n and-commit script, or even a pre-commit hook) so others can\n reproduce the results.\n\nor\n\n B. Occasionally (e.g., daily or weekly) the \"baseline\" version of the\n common code that can be relied on gets bumped, and each commit of\n productA should work with that version and all later versions for a\n while. Everyday development might typically happen with the tip\n version of the common code which may be faster, have more\n bugfixes, and otherwise be more pleasant to work with, but commits\n should work against the baseline version as well. When it is time\n to bump the baseline, that fact gets recorded (in a separate\n commit).\n\n For this, the '[submodule \"\"] ignore' setting described in\n gitmodules(5) might be helpful.\n\nThough of course other variations are possible.\n\nWould you be able to try out using Heiko's patch for a while, adapt it\nto your needs as necessary, and let us know how it goes?\n\nThanks very much, and good luck,\nJonathan\n"},{"id":"180746","messageId":"7vborhaqgw.fsf@alter.siamese.dyndns.org","threadId":"28896","inReplyTo":"loom.20111210T062013-538@post.gmane.org","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-12-10T06:30:23Z","receivedAt":"2011-12-10T06:30:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Leif Gruenwoldt writes:\n\n> Our use case is as follows. We have several repositories for our common code \n> (commonA.git, commonB.git, etc) and a few different products that leverage these \n> common repos (productA.git, productB.git, etc). When one of the products is in \n> heavy development we often need to do a lot of work in the common repos. Having \n> to increment the sha1 of the submodules to track the latest tip would be overly \n> arduous. (Obviously when development of the product stabilizes we would want to \n> change to anchoring to a specific sha1 in the common repos).\n\nNobody forces you to update the commit in the submodule bound to the\nsuperproject tree every time you update areas that are unrelated to or\nindependent from that frequently updated submodule.\n\nDuring the period the submodule is so often updated that you feel \"having\nto increment ... would be overly arduous\", it does not matter which exact\ncommit in that submodule is used in the tree for your other modules and\nthe superproject. Otherwise you _would_ want to say something like \"for\nthis entire tree state from the top-level superproject to correctly work,\nwe absolutely need to have this commit, not any commit that is older and\nis known to be broken, from this submodule\", and cannot afford to use\nfloating.\n\nWhich means by definition anybody who wants floating can instead let such\nan often updated submodule stay somewhat stale by not running \"submodule\nupdate\" for it unnecessarily. In a well-modularized set of projects, the\ninterface to the busy submodule may be stable and I can imagine that kind\nof arrangement would well be not just possible but practical, and probably\nyours may be such a project.\n\nSo that use case does not sound like a good rationale to require addition\nof floating submodules.\n"},{"id":"180798","messageId":"4EE369B6.7060602@svario.it","threadId":"28896","inReplyTo":"7vr51htbsy.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Gioele Barabucci","fromEmail":"gioele@svario.it","sentAt":"2011-12-10T14:16:22Z","receivedAt":"2011-12-10T14:16:22Z","isPatch":true,"sender":{"key":"gioele@svario.it","avatar":null},"body":"On 09/11/2011 18:01, Junio C Hamano wrote:\n>> This is almost ready but I would like to know what users of the\n>> \"floating submodule\" think about this.\n >\n> I do like to hear from potential users as well, because the general\n> impression we got was that floating submodules is not a real need of\n> anybody,\n\nFloating modules are something much sought after by those who use Git \nfor non-development purposes, like those who have most of their $HOME \nversioned with Git [1]. For example, part of what Joey Hess's `mr` tool \n[2] does is to simulate floating submodules for Git-versioned $HOMEs.\n\nIn the context of versioned $HOMEs, or with backups in general, precise \ntracking of submodules updates is not that important. To quote [3]: \n«Last, change tracking is a bit more lenient with home directories. I \nmay shuffle some stuff around, and I don't need to explain the changes \nto anyone else.». In my case, I want my ~/Documents dir (that is in a \ndifferent repo from $HOME) to be always updated; I would prefer not to \ndeal with submodule updates, merges and detached HEADs.\n\nBye,\n\n[1] http://vcs-home.branchable.com/\n[2] http://kitenet.net/~joey/code/mr/\n[3] http://joshcarter.com/productivity/svn_hg_git_for_home_directory\n\n--\nGioele Barabucci \n"},{"id":"180800","messageId":"CALFF=ZQKRgx_AodBQH17T9cSe_JFtoKie7DoMMfkTXCyCFospw@mail.gmail.com","threadId":"28896","inReplyTo":"7vborhaqgw.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Leif Gruenwoldt","fromEmail":"leifer@gmail.com","sentAt":"2011-12-10T15:27:28Z","receivedAt":"2011-12-10T15:27:28Z","isPatch":true,"sender":{"key":"leifer@gmail.com","avatar":"https://gravatar.com/avatar/5c17a063e2df05a22275a8cf0e00e16a75c1ce4f9a59970be15eafedd50629d1?d=mp&s=160"},"body":"On Sat, Dec 10, 2011 at 1:30 AM, Junio C Hamano wrote:\n\n> So that use case does not sound like a good rationale to require addition\n> of floating submodules.\n\nOk I will try another scenario :)\n\nImagine again products A, B and C and a common library. The products are in\na stable state of development and track a stable branch of the common lib.\nThen imagine an important security fix gets made to the common library. On\nthe next pull of products A, B, and C they get this fix for free\nbecause they were\nfloating. They didn't need to communicate with the maintainer of the common\nrepo to know this. In fact they don't really care. They just want the\nlatest stable\ncode for that release branch.\n\nThis is how package management on many linux systems works. Dependencies\nget updated and all products reap the benefit (or catastrophe) automatically.\n"},{"id":"180932","messageId":"4EE61EED.50604@ursus.ath.cx","threadId":"28896","inReplyTo":"CALFF=ZQKRgx_AodBQH17T9cSe_JFtoKie7DoMMfkTXCyCFospw@mail.gmail.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Andreas T.Auer","fromEmail":"andreas.t.auer_gtml_37453@ursus.ath.cx","sentAt":"2011-12-12T15:34:05Z","receivedAt":"2011-12-12T15:34:05Z","isPatch":true,"sender":{"key":"andreas.t.auer_gtml_37453@ursus.ath.cx","avatar":null},"body":"\n\nOn 10.12.2011 16:27 Leif Gruenwoldt wrote:\n> On Sat, Dec 10, 2011 at 1:30 AM, Junio C Hamano \n> wrote:\n>\n> > So that use case does not sound like a good rationale to require\n> > addition of floating submodules.\n>\n> Ok I will try another scenario :)\n>\n> Imagine again products A, B and C and a common library. The products\n> are in a stable state of development and track a stable branch of\n> the common lib. Then imagine an important security fix gets made to\n> the common library. On the next pull of products A, B, and C they get\n> this fix for free because they were floating. They didn't need to\n> communicate with the maintainer of the common repo to know this. In\n> fact they don't really care. They just want the latest stable code\n> for that release branch.\n\nSo you don't want to have a stale submodule as Junio suggested, which is \nolder than the gitlinked commit in the superproject, but you want to \nhave the newest stable version, which is not yet gitlinked in the \nsuperproject, right?\n\nWouldn't ( cd commonlib ; git pull stable ) instead of\ngit submodule update commonlib\nwork as you want?\n\nTo be able to configure this update behavior in .gitmodules for _some_ \nsubmodules, could be helpful in this case.\n\nSo you don't want to add a new commit to the products A, B and C repos \nwhenever the stable branch of the submodule changes, but on the other \nhand when you commit changes to the products it would still make sense \nto update the gitlink to the current commonlib version together with \nyour changes, too, right?\n\n\n> This is how package management on many linux systems works.\n> Dependencies get updated and all products reap the benefit (or\n> catastrophe) automatically.\nIf I have e.g. the Debian testing distro, which is more floating than \nthe most other Linux distro releases, then I still get only those \nversions of the packages that are referenced by this \"Debian testing\" \nsuperproject, unless I specify a different superproject (e.g. \"Debian \nunstable\") to get a newer version, but they are still tracked in some \nsuperproject. I'm not aware of a way to get the newest version of a \npackage before it is in some \"superproject\", except downloading it \nexplictily somewhere else. But I don't think this is what you want.\n"},{"id":"180939","messageId":"CALFF=ZRYB1LkAY5WSC4Eydu-N0KNnWLLM2CfbSXZji18yO82gw@mail.gmail.com","threadId":"28896","inReplyTo":"4EE61EED.50604@ursus.ath.cx","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Leif Gruenwoldt","fromEmail":"leifer@gmail.com","sentAt":"2011-12-12T18:04:42Z","receivedAt":"2011-12-12T18:04:42Z","isPatch":true,"sender":{"key":"leifer@gmail.com","avatar":"https://gravatar.com/avatar/5c17a063e2df05a22275a8cf0e00e16a75c1ce4f9a59970be15eafedd50629d1?d=mp&s=160"},"body":"On Mon, Dec 12, 2011 at 10:34 AM, Andreas T.Auer\n wrote:\n\n> So you don't want to have a stale submodule as Junio suggested, which is\n> older than the gitlinked commit in the superproject, but you want to have\n> the newest stable version, which is not yet gitlinked in the superproject,\n> right?\n\nRight.\n\n> Wouldn't  ( cd commonlib ; git pull stable ) instead of\n> git submodule update commonlib\n> work as you want?\n\nYes that's how we perform business now.\n\n> To be able to configure this update behavior in .gitmodules for _some_\n> submodules, could be helpful in this case.\n\nYes my thoughts exactly.\n\n\n> So you don't want to add a new commit to the products A, B and C repos\n> whenever the stable branch of the submodule changes, but on the other hand\n> when you commit changes to the products it would still make sense to update\n> the gitlink to the current commonlib version together with your changes,\n>  too, right?\n\nHmm I supose that does make sense. If the commonlib version was auto recorded\nduring a commit of the product it would be nice. Then if/when the user\nreconfigured\nthe submodule from \"floating\" to \"strict\" mode it would then have a\nsubmodule sha1\nreference. I like how this sounds.\n"},{"id":"180948","messageId":"4EE64B04.8080405@ursus.ath.cx","threadId":"28896","inReplyTo":"CALFF=ZRYB1LkAY5WSC4Eydu-N0KNnWLLM2CfbSXZji18yO82gw@mail.gmail.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Andreas T.Auer","fromEmail":"andreas.t.auer_gtml_37453@ursus.ath.cx","sentAt":"2011-12-12T18:42:12Z","receivedAt":"2011-12-12T18:42:12Z","isPatch":true,"sender":{"key":"andreas.t.auer_gtml_37453@ursus.ath.cx","avatar":null},"body":"\n\nOn 12.12.2011 19:04 Leif Gruenwoldt wrote:\n> On Mon, Dec 12, 2011 at 10:34 AM, Andreas T.Auer\n> wrote:\n>\n> > So you don't want to add a new commit to the products A, B and C\n> > repos whenever the stable branch of the submodule changes, but on\n> > the other hand when you commit changes to the products it would\n> > still make sense to update the gitlink to the current commonlib\n> > version together with your changes, too, right?\n>\n> Hmm I supose that does make sense. If the commonlib version was auto\n> recorded during a commit of the product it would be nice. Then\n> if/when the user reconfigured the submodule from \"floating\" to\n> \"strict\" mode it would then have a submodule sha1 reference. I like\n> how this sounds.\n\nThe next question is: Wouldn't you like to have the new stable branch \nonly pulled in, when the projectX (as the superproject) is currently on \nthat new development branch (maybe master)?\n\nBut if you checkout that fixed released version 1.2.9.8, wouldn't it be \nbetter that in that case the gitlinked version of the submodule is \nchecked out instead of some unrelated new version? I mean, when the \ngitlinks are tracked with the projectX commits, this should work well.\n\nAnd what about a maintenance branch, which is not a fixed version but a \nquite stable branch which should only have bugfixes. Shouldn't the \nauto-pull be disabled in that case, too?\n\nI think the \"auto-pull\" behavior should depend on the currently checked \nout branch. So the configuration options should allow the definition of \none or more mappings.\n"},{"id":"180952","messageId":"CALFF=ZRB7qjj7VMhzr12ySdHmZsySoqceu5brFht8rX1+W3NPg@mail.gmail.com","threadId":"28896","inReplyTo":"4EE64B04.8080405@ursus.ath.cx","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Leif Gruenwoldt","fromEmail":"leifer@gmail.com","sentAt":"2011-12-12T19:13:45Z","receivedAt":"2011-12-12T19:13:45Z","isPatch":true,"sender":{"key":"leifer@gmail.com","avatar":"https://gravatar.com/avatar/5c17a063e2df05a22275a8cf0e00e16a75c1ce4f9a59970be15eafedd50629d1?d=mp&s=160"},"body":"On Mon, Dec 12, 2011 at 1:42 PM, Andreas T.Auer\n wrote:\n\n> The next question is: Wouldn't you like to have the new stable branch only\n> pulled in, when the projectX (as the superproject) is currently on that new\n> development branch (maybe master)?\n>\n> But if you checkout that fixed released version 1.2.9.8, wouldn't it be\n> better that in that case the gitlinked version of the submodule is checked\n> out instead of some unrelated new version? I mean, when the gitlinks are\n> tracked with the projectX commits, this should work well.\n>\n> And what about a maintenance branch, which is not a fixed version but a\n> quite stable branch which should only have bugfixes. Shouldn't the auto-pull\n> be disabled in that case, too?\n>\n> I think the \"auto-pull\" behavior should depend on the currently checked out\n> branch. So the configuration options should allow the definition of one or\n> more mappings.\n\nYes. I think you nailed it. The floating behaviour would best be\nconfigured per branch.\n\nAn aside. Would this mean a \"git pull\" on the product repo would\nautomatically do a pull (git submodule update) on the submodule too?\n"},{"id":"180955","messageId":"7vaa6x4m5l.fsf@alter.siamese.dyndns.org","threadId":"28896","inReplyTo":"CALFF=ZQKRgx_AodBQH17T9cSe_JFtoKie7DoMMfkTXCyCFospw@mail.gmail.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-12-12T19:36:54Z","receivedAt":"2011-12-12T19:36:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Leif Gruenwoldt writes:\n\n> On Sat, Dec 10, 2011 at 1:30 AM, Junio C Hamano wrote:\n>\n>> So that use case does not sound like a good rationale to require addition\n>> of floating submodules.\n>\n> Ok I will try another scenario :)\n>\n> Imagine again products A, B and C and a common library. The products are in\n> a stable state of development and track a stable branch of the common lib.\n> Then imagine an important security fix gets made to the common library. On\n> the next pull of products A, B, and C they get this fix for free\n> because they were\n> floating. They didn't need to communicate with the maintainer of the common\n> repo to know this. In fact they don't really care. They just want the\n> latest stable\n> code for that release branch.\n>\n> This is how package management on many linux systems works. Dependencies\n> get updated and all products reap the benefit (or catastrophe) automatically.\n\nDistro package dependency tracking is a poor analogy for many reasons, but\nI'll only touch a few.\n\nIf you have a common library L and application packages A, B and C, first\nof all, you do not build distro package of A from the sources of A and\nL. Instead, package A has a build dependency on package L-devel (in other\nwords, \"in order to build A from the source, you need L-devel package\ninstalled in your build environment\"), build A from its source, link it\nagainst L's binary without having the source of L. So the source code\narrangement is very different from the typical submodule case, in that you\ndo not even need to have A and L appear in the same working tree, let\nalone bound in a same superproject as two submodules.\n\nA library L may have two API versions, and application A and B may be\nwritten for its v1.0 and v2.0 API, respectively. Distro packaging makes\nthe binary package A and B _know_ about their own dependency requirements\nby recording \"A depends on L (v1.0<=, On Mon, Dec 12, 2011 at 1:42 PM, Andreas T.Auer\n> wrote:\n> \n>> The next question is: Wouldn't you like to have the new stable branch only\n>> pulled in, when the projectX (as the superproject) is currently on that new\n>> development branch (maybe master)?\n>>\n>> But if you checkout that fixed released version 1.2.9.8, wouldn't it be\n>> better that in that case the gitlinked version of the submodule is checked\n>> out instead of some unrelated new version? I mean, when the gitlinks are\n>> tracked with the projectX commits, this should work well.\n>>\n>> And what about a maintenance branch, which is not a fixed version but a\n>> quite stable branch which should only have bugfixes. Shouldn't the auto-pull\n>> be disabled in that case, too?\n>>\n>> I think the \"auto-pull\" behavior should depend on the currently checked out\n>> branch. So the configuration options should allow the definition of one or\n>> more mappings.\n> \n> Yes. I think you nailed it. The floating behaviour would best be\n> configured per branch.\n\nWhy not use .gitmodules and make the \"branch\" setting work without having to\nsync it?\n\n> An aside. Would this mean a \"git pull\" on the product repo would\n> automatically do a pull (git submodule update) on the submodule too?\n\nMe thinks that only \"git submodule update\" should do that. Or what should\nhappen for people not using pull but just doing fetch and checkout?\n"},{"id":"180988","messageId":"CABURp0rFOGQ9kAbAn65W3UAHTWbk5prH7spjJnFvL5fqzbFp1w@mail.gmail.com","threadId":"28896","inReplyTo":"CALFF=ZRB7qjj7VMhzr12ySdHmZsySoqceu5brFht8rX1+W3NPg@mail.gmail.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2011-12-12T22:56:52Z","receivedAt":"2011-12-12T22:56:52Z","isPatch":true,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"On Mon, Dec 12, 2011 at 2:13 PM, Leif Gruenwoldt wrote:\n> On Mon, Dec 12, 2011 at 1:42 PM, Andreas T.Auer\n> wrote:\n>\n>> The next question is: Wouldn't you like to have the new stable branch only\n>> pulled in, when the projectX (as the superproject) is currently on that new\n>> development branch (maybe master)?\n>>\n>> But if you checkout that fixed released version 1.2.9.8, wouldn't it be\n>> better that in that case the gitlinked version of the submodule is checked\n>> out instead of some unrelated new version? I mean, when the gitlinks are\n>> tracked with the projectX commits, this should work well.\n>>\n>> And what about a maintenance branch, which is not a fixed version but a\n>> quite stable branch which should only have bugfixes. Shouldn't the auto-pull\n>> be disabled in that case, too?\n>>\n>> I think the \"auto-pull\" behavior should depend on the currently checked out\n>> branch. So the configuration options should allow the definition of one or\n>> more mappings.\n>\n> Yes. I think you nailed it. The floating behaviour would best be\n> configured per branch.\n\nYes, I think you nailed it too. I've been thinking the same thing for\na while now, but I didn't know how to express it completely. Some of\nthe discussion on here last week gelled the last bits in my mind.\n\nTo wit, I think I would want something like this in my project:\n\nUse gitlinks when the superproject HEAD is one of these:\n refs/heads/maint/*\n refs/heads/svn/* (historic branches)\n refs/tags/*\n (detached)\n\nFloat on the rest, using the branch given in .gitmodules (which may be\n* to mean \"use the same branch as the superproject\".)\n\nBut maybe it is foolish of me to keep branches where I really want\nlightweight tags. If so, I could get away with this:\n\n Float if .git/HEAD begins with \"refs/heads\"\n Else, use the SHA1.\n\n\n> An aside. Would this mean a \"git pull\" on the product repo would\n> automatically do a pull (git submodule update) on the submodule too?\n\nGood question.\n\nI want to say \"eventually, but not yet.\" But someone else may\ndisagree. \"git pull --recurse-submodules=yes\" does not do this yet.\nA separate git-submodule-update is still required. But I think this\nis a separate issue from floating submodules.\n\nPhil\n"},{"id":"180997","messageId":"CA+sFfMfH33hoRxdB01a6h=msVVjf3MwZXd7SFE6UVoVbxJVBoA@mail.gmail.com","threadId":"28896","inReplyTo":"CALFF=ZQKRgx_AodBQH17T9cSe_JFtoKie7DoMMfkTXCyCFospw@mail.gmail.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Brandon Casey","fromEmail":"drafnel@gmail.com","sentAt":"2011-12-13T00:12:11Z","receivedAt":"2011-12-13T00:12:11Z","isPatch":true,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"On Sat, Dec 10, 2011 at 9:27 AM, Leif Gruenwoldt wrote:\n> On Sat, Dec 10, 2011 at 1:30 AM, Junio C Hamano wrote:\n>\n>> So that use case does not sound like a good rationale to require addition\n>> of floating submodules.\n>\n> Ok I will try another scenario :)\n>\n> Imagine again products A, B and C and a common library. The products are in\n> a stable state of development and track a stable branch of the common lib.\n> Then imagine an important security fix gets made to the common library. On\n> the next pull of products A, B, and C they get this fix for free\n> because they were\n> floating. They didn't need to communicate with the maintainer of the common\n> repo to know this. In fact they don't really care. They just want the\n> latest stable\n> code for that release branch.\n>\n> This is how package management on many linux systems works. Dependencies\n> get updated and all products reap the benefit (or catastrophe) automatically.\n\nWhat happens if the update to the floating submodule introduces a bug\nthat prevents A, B, and/or C from building/running correctly? If the\nsubmodule states are not recorded, how would the previously working\nsubmodule version be restored so that development on A, B, and C,\ncould proceed? I guess each developer could manually checkout @{1} in\nthe submodule in their working directory, though that wouldn't work\nfor a new clone, and it's not very elegant.\n\nI presume that if A, B, and C, do not care to know exactly what was\nfixed in the common library, they probably do not care to investigate\nthe breakage in that repo either. Or, they may not have the\nexpertise.\n\n-Brandon\n"},{"id":"181039","messageId":"CABURp0pPqpkWXdC+97wR8HZeX=Nbi0bn-3ji+k9LQnj0kFjCnQ@mail.gmail.com","threadId":"28896","inReplyTo":"7vaa6x4m5l.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2011-12-13T14:17:28Z","receivedAt":"2011-12-13T14:17:28Z","isPatch":true,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"On Mon, Dec 12, 2011 at 2:36 PM, Junio C Hamano wrote:\n[...]\n> Distro package dependency tracking is a poor analogy for many reasons, but\n> I'll only touch a few.\n[...]\n> Naively, one might think that two branches, branch-1.0 and branch-2.0, can\n> be defined in the repository of L, tell somebody (like \"superproject that\n> covers all the packages in the distro\") that A wants branch-1.0 and B\n> wants branch-2.0 of L respectively, to emulate this, but if one thinks\n> further, one would realize that it is insufficient. For one thing, it is\n> unclear what should happen when both A and B are asked to be checked out,\n> but more importantly, in dependency requirements on real distro packaging,\n> the application C could say \"I want v1.0 API but v1.4 is broken and not\n> compatible with me\", which won't fit on the two-branches model. A\n> workaround to add more branches to L could be devised but any workaround\n> cannot be a good solution that allows a random application C among 47\n> others to dictate how the branch structure of L project should look like.\n>\n> Fortunately, the dependency management is a solved problem by distro\n> package management and build systems, and they do so without using\n> anything from submodules. There is no point reinventing these logic in git\n> submodules and emulating poorly.\n>\n> The only remotely plausible analogy around distro packaging would be a\n> superproject full of all the packages in a distro as its submodules, and\n> records exact versions of each and every package that goes on a release\n> CD (or DVD). In that case, you do want to have a central registry that\n> records what exact version of each package is used to cut the CD and the\n> mother of all modules superproject could be one way to implement it. But\n> that is not an example of floating, but is a direct opposite.\n>\n> This exchange convinced me further that anybody who wishes to use\n> \"floating\" is better off either by doing one or both of the following:\n>\n>  - using \"exact\" but not updating religiously, as the interdepency\n>   requirement in their project is not strict; or\n>\n>  - not using submodules at all, but merely keeping these unrelated A, B, C\n>   and L as standalone repositories next to each other in the directory\n>   structure.\n\nMy interdependency requirements are not so cut-and-dry. We use\nsubmodules to isolate controlled regions of code. We may need to\nshare our project with a contractor who is allowed to see code\npertaining to \"vendorA\" but not that for \"vendorB\" or \"VendorN\". But\nour in-house developers want to have all the vendor code in one place\nfor convenient integration. Submodules do this nicely for us. We can\ngive the contractor just the main modules and the VendorA modules and\nhe'll be fine. In-house devs get all the submodules (using the\nvendor-ALL superproject).\n\nBut this necessarily means there is too much coupling for comfort\nbetween our submodules. For example, when an API changes in the main\nsubmodule, each of the vendor submodules is affected because they each\nimplement that API in a custom method. Some of those vendor modules\nbelong to different people. Submodule synchronization becomes a real\nchore.\n\nFloating would help, I think. Instead I do this:\n\n git pull origin topic_foo && git submodule foreach 'git pull origin topic_foo'\n\n git submodule foreach 'git push origin topic_foo' && git push origin topic_foo\n\nBut not all my developers are git-gurus yet, and they sometimes mess\nup their ad hoc scripts or miss important changes they forgot to push\nin one submodule or another. Or worse, their pull or push fails and\nthey can't see the problem for all the noise. So they email it to me.\n\nOn my git server, I have a hook that automatically propagates each\npush to \"master\" from the submodules into the superproject. But this\nis tedious and limited. And it relies on a centralized server.\n\nYou may say this itch is all in my head, but it sure seems real to me.\n\nPhil\n"},{"id":"181049","messageId":"4EE770D0.5080702@xiplink.com","threadId":"28896","inReplyTo":"CABURp0rFOGQ9kAbAn65W3UAHTWbk5prH7spjJnFvL5fqzbFp1w@mail.gmail.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2011-12-13T15:35:44Z","receivedAt":"2011-12-13T15:35:44Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 11-12-12 05:56 PM, Phil Hord wrote:\n> On Mon, Dec 12, 2011 at 2:13 PM, Leif Gruenwoldt wrote:\n>> On Mon, Dec 12, 2011 at 1:42 PM, Andreas T.Auer\n>> wrote:\n>>\n>>> The next question is: Wouldn't you like to have the new stable branch only\n>>> pulled in, when the projectX (as the superproject) is currently on that new\n>>> development branch (maybe master)?\n>>>\n>>> But if you checkout that fixed released version 1.2.9.8, wouldn't it be\n>>> better that in that case the gitlinked version of the submodule is checked\n>>> out instead of some unrelated new version? I mean, when the gitlinks are\n>>> tracked with the projectX commits, this should work well.\n>>>\n>>> And what about a maintenance branch, which is not a fixed version but a\n>>> quite stable branch which should only have bugfixes. Shouldn't the auto-pull\n>>> be disabled in that case, too?\n>>>\n>>> I think the \"auto-pull\" behavior should depend on the currently checked out\n>>> branch. So the configuration options should allow the definition of one or\n>>> more mappings.\n>>\n>> Yes. I think you nailed it. The floating behaviour would best be\n>> configured per branch.\n> \n> Yes, I think you nailed it too. I've been thinking the same thing for\n> a while now, but I didn't know how to express it completely. Some of\n> the discussion on here last week gelled the last bits in my mind.\n> \n> To wit, I think I would want something like this in my project:\n> \n> Use gitlinks when the superproject HEAD is one of these:\n> refs/heads/maint/*\n> refs/heads/svn/* (historic branches)\n> refs/tags/*\n> (detached)\n> \n> Float on the rest, using the branch given in .gitmodules (which may be\n> * to mean \"use the same branch as the superproject\".)\n> \n> But maybe it is foolish of me to keep branches where I really want\n> lightweight tags. If so, I could get away with this:\n> \n> Float if .git/HEAD begins with \"refs/heads\"\n> Else, use the SHA1.\n\nWouldn't this break creating a bugfix topic branch based on an earlier\nrevision of the repo? I wouldn't want such a branch to automatically give me\nthe latest submodules.\n\nI'd prefer to have floating be explicitly configured on a per-branch (or\nper-branch-glob) basis. So in addition to what Jens described yesterday [1]\nto configure an individual submodule's floating branch, I suggest there also\nbe a new section in the .gitmodules file for configuring the super-repo's\nfloating branches, e.g.\n\n\t[super]\n\t\tfloaters = refs/heads/master refs/heads/dev*\n\n\t[submodule \"Sub1\"]\n\t\tpath = foo/bar\n\t\tbranch = maint\n\t\turl = ...\n\n\t[submodule \"Sub2\"]\n\t\tpath = other/place\n\t\turl = ...\n\nThis would mean that whenever the super-repo checks out either the \"master\"\nbranch or a branch whose name starts with \"dev\" (assuming recursive checkouts\nare on):\n\n * The Sub1 submodule automatically checks out the tip of its\n \"maint\" branch.\n\n * The Sub2 submodule (lacking a \"branch\" variable) would not float\n and would check out the commit recorded in the super-repo.\n\nA super-repo recursive-checkout that doesn't match a floaters pattern would\nwork in the regular, non-floating way.\n\n\t\tM.\n\n[1] http://article.gmane.org/gmane.comp.version-control.git/186969\n"},{"id":"181079","messageId":"4EE7BEF5.6050205@web.de","threadId":"28896","inReplyTo":"CABURp0pPqpkWXdC+97wR8HZeX=Nbi0bn-3ji+k9LQnj0kFjCnQ@mail.gmail.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-12-13T21:09:09Z","receivedAt":"2011-12-13T21:09:09Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 13.12.2011 15:17, schrieb Phil Hord:\n> On Mon, Dec 12, 2011 at 2:36 PM, Junio C Hamano wrote:\n> [...]\n>> Distro package dependency tracking is a poor analogy for many reasons, but\n>> I'll only touch a few.\n> [...]\n>> Naively, one might think that two branches, branch-1.0 and branch-2.0, can\n>> be defined in the repository of L, tell somebody (like \"superproject that\n>> covers all the packages in the distro\") that A wants branch-1.0 and B\n>> wants branch-2.0 of L respectively, to emulate this, but if one thinks\n>> further, one would realize that it is insufficient. For one thing, it is\n>> unclear what should happen when both A and B are asked to be checked out,\n>> but more importantly, in dependency requirements on real distro packaging,\n>> the application C could say \"I want v1.0 API but v1.4 is broken and not\n>> compatible with me\", which won't fit on the two-branches model. A\n>> workaround to add more branches to L could be devised but any workaround\n>> cannot be a good solution that allows a random application C among 47\n>> others to dictate how the branch structure of L project should look like.\n>>\n>> Fortunately, the dependency management is a solved problem by distro\n>> package management and build systems, and they do so without using\n>> anything from submodules. There is no point reinventing these logic in git\n>> submodules and emulating poorly.\n>>\n>> The only remotely plausible analogy around distro packaging would be a\n>> superproject full of all the packages in a distro as its submodules, and\n>> records exact versions of each and every package that goes on a release\n>> CD (or DVD). In that case, you do want to have a central registry that\n>> records what exact version of each package is used to cut the CD and the\n>> mother of all modules superproject could be one way to implement it. But\n>> that is not an example of floating, but is a direct opposite.\n>>\n>> This exchange convinced me further that anybody who wishes to use\n>> \"floating\" is better off either by doing one or both of the following:\n>>\n>> - using \"exact\" but not updating religiously, as the interdepency\n>> requirement in their project is not strict; or\n>>\n>> - not using submodules at all, but merely keeping these unrelated A, B, C\n>> and L as standalone repositories next to each other in the directory\n>> structure.\n> \n> My interdependency requirements are not so cut-and-dry. We use\n> submodules to isolate controlled regions of code. We may need to\n> share our project with a contractor who is allowed to see code\n> pertaining to \"vendorA\" but not that for \"vendorB\" or \"VendorN\". But\n> our in-house developers want to have all the vendor code in one place\n> for convenient integration. Submodules do this nicely for us. We can\n> give the contractor just the main modules and the VendorA modules and\n> he'll be fine. In-house devs get all the submodules (using the\n> vendor-ALL superproject).\n> \n> But this necessarily means there is too much coupling for comfort\n> between our submodules. For example, when an API changes in the main\n> submodule, each of the vendor submodules is affected because they each\n> implement that API in a custom method. Some of those vendor modules\n> belong to different people. Submodule synchronization becomes a real\n> chore.\n\nHmm, maybe having vendor-specific branches in the superproject would\nhelp here. But that is hard to tell without knowing more details about\nyour setup. But I suspect your vendor-ALL superproject is exactly the\nright spot to deal with these kind of problems (and if that isn't easy\nthat might be a result of the difficulty of the problem you are trying\nto solve here, keeping different vendors in sync with your API ;-).\n\n> Floating would help, I think. Instead I do this:\n> \n> git pull origin topic_foo && git submodule foreach 'git pull origin topic_foo'\n> \n> git submodule foreach 'git push origin topic_foo' && git push origin topic_foo\n\nThis sounds to me like you would need the \"--recurse-submodules\" option\nimplemented for \"git pull\" and \"git push\", no? And I miss to see how\nfloating would help when the tips of some submodules are not ready to\nwork with other submodules tips ...\n\n> But not all my developers are git-gurus yet, and they sometimes mess\n> up their ad hoc scripts or miss important changes they forgot to push\n> in one submodule or another.\n\nSure, even though current git should help you some by showing changes\nin the submodules.\n\n> Or worse, their pull or push fails and\n> they can't see the problem for all the noise. So they email it to me.\n\nWe circumvent that by not pulling, but fetching and merging in the\nsubmodule first and after that in the superproject. You have much more\ncontrol about what is going wrong where (and can have more\ngit-experienced people help with - or even do - the merges).\n\n> On my git server, I have a hook that automatically propagates each\n> push to \"master\" from the submodules into the superproject. But this\n> is tedious and limited. And it relies on a centralized server.\n\nBut for closely related stuff that is a good option. Our continuous\nintegration server shows us quite some breakage between submodules\nbefore they hit a superproject, which is really helpful.\n\n> You may say this itch is all in my head, but it sure seems real to me.\n\nThis definitely is a real problem. Lets see how far git can help you\nhere...\n"},{"id":"181080","messageId":"4EE7C15A.3040501@web.de","threadId":"28896","inReplyTo":"4EE770D0.5080702@xiplink.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-12-13T21:19:22Z","receivedAt":"2011-12-13T21:19:22Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 13.12.2011 16:35, schrieb Marc Branchaud:\n> I'd prefer to have floating be explicitly configured on a per-branch (or\n> per-branch-glob) basis. So in addition to what Jens described yesterday [1]\n> to configure an individual submodule's floating branch, I suggest there also\n> be a new section in the .gitmodules file for configuring the super-repo's\n> floating branches, e.g.\n> \n> \t[super]\n> \t\tfloaters = refs/heads/master refs/heads/dev*\n> \n> \t[submodule \"Sub1\"]\n> \t\tpath = foo/bar\n> \t\tbranch = maint\n> \t\turl = ...\n> \n> \t[submodule \"Sub2\"]\n> \t\tpath = other/place\n> \t\turl = ...\n\nHmm, but you can have different .gitmodules files in different branches of\nthe superproject, no? Why not just have the \"branch = maint\" setting for\n\"Sub1\" in the master and the dev branches .gitmodules file and drop it in\nthe other branches?\n\n> This would mean that whenever the super-repo checks out either the \"master\"\n> branch or a branch whose name starts with \"dev\" (assuming recursive checkouts\n> are on):\n> \n> * The Sub1 submodule automatically checks out the tip of its\n> \"maint\" branch.\n> \n> * The Sub2 submodule (lacking a \"branch\" variable) would not float\n> and would check out the commit recorded in the super-repo.\n> \n> A super-repo recursive-checkout that doesn't match a floaters pattern would\n> work in the regular, non-floating way.\n\nWhich would just work with my proposal too if git would honor the\n.gitmodules file of the currently checked out branch.\n\n> [1] http://article.gmane.org/gmane.comp.version-control.git/186969\n"},{"id":"181088","messageId":"4EE7D4E0.5000305@xiplink.com","threadId":"28896","inReplyTo":"4EE7C15A.3040501@web.de","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2011-12-13T22:42:40Z","receivedAt":"2011-12-13T22:42:40Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 11-12-13 04:19 PM, Jens Lehmann wrote:\n> Am 13.12.2011 16:35, schrieb Marc Branchaud:\n>> I'd prefer to have floating be explicitly configured on a per-branch (or\n>> per-branch-glob) basis. So in addition to what Jens described yesterday [1]\n>> to configure an individual submodule's floating branch, I suggest there also\n>> be a new section in the .gitmodules file for configuring the super-repo's\n>> floating branches, e.g.\n>>\n>> \t[super]\n>> \t\tfloaters = refs/heads/master refs/heads/dev*\n>>\n>> \t[submodule \"Sub1\"]\n>> \t\tpath = foo/bar\n>> \t\tbranch = maint\n>> \t\turl = ...\n>>\n>> \t[submodule \"Sub2\"]\n>> \t\tpath = other/place\n>> \t\turl = ...\n> \n> Hmm, but you can have different .gitmodules files in different branches of\n> the superproject, no?\n\nYes. I'm not sure I see that as a problem though.\n\n> Why not just have the \"branch = maint\" setting for\n> \"Sub1\" in the master and the dev branches .gitmodules file and drop it in\n> the other branches?\n\nBecause I think that's an error-prone approach.\n\nIf the user creates a topic branch off (an ancestor) of master, git doesn't\nknow if the user wants floating submodules or not. If this is a bugfix\ntopic, the user would have to edit .gitmodules to turn off floating. But\nthat modified .gitmodules is too easily committed to the branch, and once it\ngets merged back into master suddenly master loses its \"floating\" feature.\n\nWhat's more, less-sophisticated users would be wary of editing an \"internal\"\nfile like .gitmodules.\n\nInstead I think it's more intuitive for the repository to define which\nbranches get floating submodules and which don't, and IMO a list of\nnames/globs is a good way to do that. The repo's users would need to be\naware of what the magic branch names are, but I think that's easily\ncommunicated in the floating-submodule scenarios I've seen posted. Git could\nalso help by telling the user, when a branch is created or it's name is\ndisplayed, whether or not it's got floating submodules.\n\n\t\tM.\n"},{"id":"183388","messageId":"CABURp0pDoS1wgJ+Fs3XFX=A_EuR4Gzi4mHLiQP+-icT_d3J+WQ@mail.gmail.com","threadId":"28896","inReplyTo":"4EE7BEF5.6050205@web.de","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2012-01-30T21:15:19Z","receivedAt":"2012-01-30T21:15:19Z","isPatch":true,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"I lost my grip on this thread over the holidays...\n\nOn Tue, Dec 13, 2011 at 4:09 PM, Jens Lehmann wrote:\n> Am 13.12.2011 15:17, schrieb Phil Hord:\n>> On Mon, Dec 12, 2011 at 2:36 PM, Junio C Hamano wrote:\n>> [...]\n>>> Distro package dependency tracking is a poor analogy for many reasons, but\n>>> I'll only touch a few.\n>> [...]\n>>> Naively, one might think that two branches, branch-1.0 and branch-2.0, can\n>>> be defined in the repository of L, tell somebody (like \"superproject that\n>>> covers all the packages in the distro\") that A wants branch-1.0 and B\n>>> wants branch-2.0 of L respectively, to emulate this, but if one thinks\n>>> further, one would realize that it is insufficient. For one thing, it is\n>>> unclear what should happen when both A and B are asked to be checked out,\n>>> but more importantly, in dependency requirements on real distro packaging,\n>>> the application C could say \"I want v1.0 API but v1.4 is broken and not\n>>> compatible with me\", which won't fit on the two-branches model. A\n>>> workaround to add more branches to L could be devised but any workaround\n>>> cannot be a good solution that allows a random application C among 47\n>>> others to dictate how the branch structure of L project should look like.\n>>>\n>>> Fortunately, the dependency management is a solved problem by distro\n>>> package management and build systems, and they do so without using\n>>> anything from submodules. There is no point reinventing these logic in git\n>>> submodules and emulating poorly.\n>>>\n>>> The only remotely plausible analogy around distro packaging would be a\n>>> superproject full of all the packages in a distro as its submodules, and\n>>> records exact versions of each and every package that goes on a release\n>>> CD (or DVD). In that case, you do want to have a central registry that\n>>> records what exact version of each package is used to cut the CD and the\n>>> mother of all modules superproject could be one way to implement it. But\n>>> that is not an example of floating, but is a direct opposite.\n>>>\n>>> This exchange convinced me further that anybody who wishes to use\n>>> \"floating\" is better off either by doing one or both of the following:\n>>>\n>>>  - using \"exact\" but not updating religiously, as the interdepency\n>>>   requirement in their project is not strict; or\n>>>\n>>>  - not using submodules at all, but merely keeping these unrelated A, B, C\n>>>   and L as standalone repositories next to each other in the directory\n>>>   structure.\n>>\n>> My interdependency requirements are not so cut-and-dry.  We use\n>> submodules to isolate controlled regions of code.  We may need to\n>> share our project with a contractor who is allowed to see code\n>> pertaining to \"vendorA\" but not that for \"vendorB\" or \"VendorN\".  But\n>> our in-house developers want to have all the vendor code in one place\n>> for convenient integration. Submodules do this nicely for us.  We can\n>> give the contractor just the main modules and the VendorA modules and\n>> he'll be fine.  In-house devs get all the submodules (using the\n>> vendor-ALL superproject).\n>>\n>> But this necessarily means there is too much coupling for comfort\n>> between our submodules.   For example, when an API changes in the main\n>> submodule, each of the vendor submodules is affected because they each\n>> implement that API in a custom method.  Some of those vendor modules\n>> belong to different people.  Submodule synchronization becomes a real\n>> chore.\n>\n> Hmm, maybe having vendor-specific branches in the superproject would\n> help here. But that is hard to tell without knowing more details about\n> your setup. But I suspect your vendor-ALL superproject is exactly the\n> right spot to deal with these kind of problems (and if that isn't easy\n> that might be a result of the difficulty of the problem you are trying\n> to solve here, keeping different vendors in sync with your API ;-).\n>\n>> Floating would help, I think.  Instead I do this:\n>>\n>>   git pull origin topic_foo && git submodule foreach 'git pull origin topic_foo'\n>>\n>>   git submodule foreach 'git push origin topic_foo' && git push origin topic_foo\n>\n> This sounds to me like you would need the \"--recurse-submodules\" option\n> implemented for \"git pull\" and \"git push\", no?\n\nOnly if I have nested submodules, but yes, we do use --recurs* in our scripts.\n\n> And I miss to see how\n> floating would help when the tips of some submodules are not ready to\n> work with other submodules tips ...\n\nBy project policy, for any branch, all submodules' tips of the\nsame-named branch should be interoperable. The CI server looks after\nthis, as much as he can.\n\nI think of branch names as sticky notes (extra-lightweight tags,\nsometimes). We have linear history in many of our vendor submodules,\nbut multiple \"branches\" indicate where each superproject branch has\npresumably finished integration.\n\n>> But not all my developers are git-gurus yet, and they sometimes mess\n>> up their ad hoc scripts or miss important changes they forgot to push\n>> in one submodule or another.\n>\n> Sure, even though current git should help you some by showing changes\n> in the submodules.\n\nReal newbies may not even remember to use 'git status' strategically.\n\n>>  Or worse, their pull or push fails and\n>> they can't see the problem for all the noise.  So they email it to me.\n>\n> We circumvent that by not pulling, but fetching and merging in the\n> submodule first and after that in the superproject. You have much more\n> control about what is going wrong where (and can have more\n> git-experienced people help with - or even do - the merges).\n\nI do that, too, and I wish I didn't have to. I wish I could safely\nand sanely recover from a conflicted \"git pull --recurse-submodules\"\npull from the superproject. That is, I wish doing so were as\nstraightforward as recovering from the same condition would be if all\nmy code were in one repository instead of in submodules.\n\nWhich is the gist -- I wish submodules did not make git more\ncomplicated than it already is.\n\nPhil\n"},{"id":"183432","messageId":"4F28554D.9090107@web.de","threadId":"28896","inReplyTo":"CABURp0pDoS1wgJ+Fs3XFX=A_EuR4Gzi4mHLiQP+-icT_d3J+WQ@mail.gmail.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-01-31T20:55:41Z","receivedAt":"2012-01-31T20:55:41Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 30.01.2012 22:15, schrieb Phil Hord:\n> I lost my grip on this thread over the holidays...\n> \n> On Tue, Dec 13, 2011 at 4:09 PM, Jens Lehmann wrote:\n>> Am 13.12.2011 15:17, schrieb Phil Hord:\n>>> git pull origin topic_foo && git submodule foreach 'git pull origin topic_foo'\n>>>\n>>> git submodule foreach 'git push origin topic_foo' && git push origin topic_foo\n>>\n>> This sounds to me like you would need the \"--recurse-submodules\" option\n>> implemented for \"git pull\" and \"git push\", no?\n> \n> Only if I have nested submodules, but yes, we do use --recurs* in our scripts.\n\nI'm confused, push doesn't know the \"--recurse-submodules\" option\nat all yet while pull only does a deep fetch when it is given, the\nsubmodule work trees are not updated to the merge result right now.\n\n>> And I miss to see how\n>> floating would help when the tips of some submodules are not ready to\n>> work with other submodules tips ...\n> \n> By project policy, for any branch, all submodules' tips of the\n> same-named branch should be interoperable. The CI server looks after\n> this, as much as he can.\n\nWe do the same thing on our CI server, but it can only test some\ncombinations (even though that tends to show most problems pretty\nearly). But in the end every superproject is responsible to use a\nworking set of submodule commits, and I would rather bet on a\ncombination the CI server tested than on what happens to be on the\ncurrent tips.\n\n> I think of branch names as sticky notes (extra-lightweight tags,\n> sometimes). We have linear history in many of our vendor submodules,\n> but multiple \"branches\" indicate where each superproject branch has\n> presumably finished integration.\n\nWe also add a branch in submodules every time a superproject needs\nto move away from the submodules master (so the commits won't get\nlost by accident).\n\n>>> But not all my developers are git-gurus yet, and they sometimes mess\n>>> up their ad hoc scripts or miss important changes they forgot to push\n>>> in one submodule or another.\n>>\n>> Sure, even though current git should help you some by showing changes\n>> in the submodules.\n> \n> Real newbies may not even remember to use 'git status' strategically.\n\nHmm, but then they will screw up things in the superproject too, no?\n\n>>> Or worse, their pull or push fails and\n>>> they can't see the problem for all the noise. So they email it to me.\n>>\n>> We circumvent that by not pulling, but fetching and merging in the\n>> submodule first and after that in the superproject. You have much more\n>> control about what is going wrong where (and can have more\n>> git-experienced people help with - or even do - the merges).\n> \n> I do that, too, and I wish I didn't have to. I wish I could safely\n> and sanely recover from a conflicted \"git pull --recurse-submodules\"\n> pull from the superproject.\n\nThat's what my recursive checkout work is aiming at. Me thinks after\nthat we will also need some good ideas on how to present and help\nsolving submodule merge conflicts.\n\n> That is, I wish doing so were as\n> straightforward as recovering from the same condition would be if all\n> my code were in one repository instead of in submodules.\n>\n> Which is the gist -- I wish submodules did not make git more\n> complicated than it already is.\n\nI think we can make working with submodule much easier than it is\nnow, the next step being updating all submodule work trees as git\nupdates the superproject's work tree. Even though I suspect that in\nthe long run submodules will always be a bit more complicated than\nhaving everything in one repository, I'm confident that will be by\nfar outweighed by the advantages they bring.\n"},{"id":"183440","messageId":"CABURp0pSGGT8eyzNad-dNNx49oioAxOPOf3dmqu7M3fnV+PzdA@mail.gmail.com","threadId":"28896","inReplyTo":"4F28554D.9090107@web.de","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2012-01-31T22:50:02Z","receivedAt":"2012-01-31T22:50:02Z","isPatch":true,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"On Tue, Jan 31, 2012 at 3:55 PM, Jens Lehmann wrote:\n> Am 30.01.2012 22:15, schrieb Phil Hord:\n>> I lost my grip on this thread over the holidays...\n>>\n>> On Tue, Dec 13, 2011 at 4:09 PM, Jens Lehmann wrote:\n>>> Am 13.12.2011 15:17, schrieb Phil Hord:\n>>>>   git pull origin topic_foo && git submodule foreach 'git pull origin topic_foo'\n>>>>\n>>>>   git submodule foreach 'git push origin topic_foo' && git push origin topic_foo\n>>>\n>>> This sounds to me like you would need the \"--recurse-submodules\" option\n>>> implemented for \"git pull\" and \"git push\", no?\n>>\n>> Only if I have nested submodules, but yes, we do use --recurs* in our scripts.\n>\n> I'm confused, push doesn't know the \"--recurse-submodules\" option\n> at all yet while pull only does a deep fetch when it is given, the\n> submodule work trees are not updated to the merge result right now.\n\nSorry. I type faster than I think sometimes.\n\nI meant that I do use something like this:\n   git submodule foreach --recursive 'git checkout master' && git\ncheckout master\n\nAnd I expect to use something like this:\n   git submodule foreach --recursive 'git push origin topic_foo' &&\ngit push origin topic_foo\n\n>>> And I miss to see how\n>>> floating would help when the tips of some submodules are not ready to\n>>> work with other submodules tips ...\n>>\n>> By project policy, for any branch, all submodules' tips of the\n>> same-named branch should be interoperable.  The CI server looks after\n>> this, as much as he can.\n>\n> We do the same thing on our CI server, but it can only test some\n> combinations (even though that tends to show most problems pretty\n> early). But in the end every superproject is responsible to use a\n> working set of submodule commits, and I would rather bet on a\n> combination the CI server tested than on what happens to be on the\n> current tips.\n\nYes, I see what you mean. In our case, what \"happens to be on the\ntips\" should also have been vetted by our Gerrit+Jenkins code review\ndance (for each branch * each superproject), and so it should always\nbe good. If it's not, we have the Jenkins-maintained gitlink history\nwe can use to bisect.\n\n>> I think of branch names as sticky notes (extra-lightweight tags,\n>> sometimes).  We have linear history in many of our vendor submodules,\n>> but multiple \"branches\" indicate where each superproject branch has\n>> presumably finished integration.\n>\n> We also add a branch in submodules every time a superproject needs\n> to move away from the submodules master (so the commits won't get\n> lost by accident).\n\nGood idea. Gerrit sees to this for us also, but we do share code on\nancillary git servers and we follow the same practice.\n\n>>>> But not all my developers are git-gurus yet, and they sometimes mess\n>>>> up their ad hoc scripts or miss important changes they forgot to push\n>>>> in one submodule or another.\n>>>\n>>> Sure, even though current git should help you some by showing changes\n>>> in the submodules.\n>>\n>> Real newbies may not even remember to use 'git status' strategically.\n>\n> Hmm, but then they will screw up things in the superproject too, no?\n\nWhat I mean is that a developer may be completely focused on one\nparticular submodule (his domain). He does his work in this module,\nand when it's ready he commits and pushes to the server. 'git status'\nshows him that his directory is clean. But this is only because he\ndoesn't really know where the submodules top-directories are, so he\ndoesn't realize that he has changes in another submodule that he has\nnot committed. He has to know to run 'git status' from somewhere in\nthe superproject (ostensibly in the root directory of that\nsuperproject). But he may forget since 'git status' already assured\nhim he was done.\n\nLike this:\n\n#-- Setup\nmkdir super && cd super && git init\nmkdir A && touch A/foo\ngit add .\ngit submodule add gerrit:iptv/iptv_scripts B\ngit commit -m \"Initial commit\"\n\n#-- Work flow\ncd A && echo changes > foo\ncd ../B && echo changes > foo\ngit add . && git commit -m \"Made some changes\"\ngit status\n# On branch master\n# Your branch is ahead of 'origin/master' by 1 commit.\n#\nnothing to commit (working directory clean)\n\nSo git just told my new-to-git developer that his workdir is clean\nbecause he happened to run it from super/B. But if he ran it from\nsuper/A (or super) it would tell him otherwise.\n\nI guess what would help here is something like the opposite of 'git\nstatus' showing the status of descendant submodules; it would help if\nit showed the status of sibling submodules and the superproject as\nwell.\n\n>>>>  Or worse, their pull or push fails and\n>>>> they can't see the problem for all the noise.  So they email it to me.\n>>>\n>>> We circumvent that by not pulling, but fetching and merging in the\n>>> submodule first and after that in the superproject. You have much more\n>>> control about what is going wrong where (and can have more\n>>> git-experienced people help with - or even do - the merges).\n>>\n>> I do that, too, and I wish I didn't have to.  I wish I could safely\n>> and sanely recover from a conflicted \"git pull --recurse-submodules\"\n>> pull from the superproject.\n>\n> That's what my recursive checkout work is aiming at. Me thinks after\n> that we will also need some good ideas on how to present and help\n> solving submodule merge conflicts.\n>\n>> That is, I wish doing so were as\n>> straightforward as recovering from the same condition would be if all\n>> my code were in one repository instead of in submodules.\n>>\n>> Which is the gist -- I wish submodules did not make git more\n>> complicated than it already is.\n>\n> I think we can make working with submodule much easier than it is\n> now, the next step being updating all submodule work trees as git\n> updates the superproject's work tree. Even though I suspect that in\n> the long run submodules will always be a bit more complicated than\n> having everything in one repository, I'm confident that will be by\n> far outweighed by the advantages they bring.\n\nI'm really looking forward to it.\n\nThanks,\nPhil\n"},{"id":"183509","messageId":"4F29BEB7.1080901@web.de","threadId":"28896","inReplyTo":"CABURp0pSGGT8eyzNad-dNNx49oioAxOPOf3dmqu7M3fnV+PzdA@mail.gmail.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-02-01T22:37:43Z","receivedAt":"2012-02-01T22:37:43Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 31.01.2012 23:50, schrieb Phil Hord:\n> What I mean is that a developer may be completely focused on one\n> particular submodule (his domain). He does his work in this module,\n> and when it's ready he commits and pushes to the server. 'git status'\n> shows him that his directory is clean. But this is only because he\n> doesn't really know where the submodules top-directories are, so he\n> doesn't realize that he has changes in another submodule that he has\n> not committed. He has to know to run 'git status' from somewhere in\n> the superproject (ostensibly in the root directory of that\n> superproject). But he may forget since 'git status' already assured\n> him he was done.\n\n> I guess what would help here is something like the opposite of 'git\n> status' showing the status of descendant submodules; it would help if\n> it showed the status of sibling submodules and the superproject as\n> well.\n\nHmm, I really think the fact that submodules are unaware that they\nare part of a superproject is a feature. I'd prefer seeing that kind\nof problem being tackled by the CI server and/or user education. Or\nmaybe a pre-commit hook which issues a warning in that case?\n"},{"id":"184037","messageId":"CABURp0rt=LcjMfDU61m0de-gLpX1a3x3vhb0zVxCbceSvD9jFw@mail.gmail.com","threadId":"28896","inReplyTo":"4F29BEB7.1080901@web.de","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2012-02-06T17:31:29Z","receivedAt":"2012-02-06T17:31:29Z","isPatch":true,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"On Wed, Feb 1, 2012 at 5:37 PM, Jens Lehmann wrote:\n> Am 31.01.2012 23:50, schrieb Phil Hord:\n>> What I mean is that a developer may be completely focused on one\n>> particular submodule (his domain).  He does his work in this module,\n>> and when it's ready he commits and pushes to the server.  'git status'\n>> shows him that his directory is clean.  But this is only because he\n>> doesn't really know where the submodules top-directories are, so he\n>> doesn't realize that he has changes in another submodule that he has\n>> not committed.  He has to know to run 'git status' from somewhere in\n>> the superproject (ostensibly in the root directory of that\n>> superproject).  But he may forget since 'git status' already assured\n>> him he was done.\n> \n>> I guess what would help here is something like the opposite of 'git\n>> status' showing the status of descendant submodules;  it would help if\n>> it showed the status of sibling submodules and the superproject as\n>> well.\n>\n> Hmm, I really think the fact that submodules are unaware that they\n> are part of a superproject is a feature. I'd prefer seeing that kind\n> of problem being tackled by the CI server and/or user education. Or\n> maybe a pre-commit hook which issues a warning in that case?\n\nI agree that submodule isolation is a feature essential to the\narchitecture of git and the submodules implementation. But it is also\na limitation, not just of this example. A pre-commit hook is a nice\nidea, but it doesn't help 'git status' (which is the standard go-to\nanswer point for \"where am I\").\n\nThis has me thinking more about recursing siblings now, though. I find\nmyself typing something like this quite a lot:\n git submodule foreach 'git grep \"someFunction\" || :'\n\nOr worse (in that the UI is more unwieldy):\n git submodule foreach 'git log --oneline \"-SsomeFunction\" || :'\n\nBut what I want is this:\n git --git-dir=${TOP}/../.git grep --recurse-submodules \"someFunction\"\n\nBut not really, because I am lazy and that is too much typing.\n git grep --include-siblings \"someFunction\"\n\nMaybe I can add a \"sib\" macro to get this:\n git sib grep \"someFunction\"\n\nBut now I've really wandered off-topic.\nPhil\n"},{"id":"184063","messageId":"4F3046D7.2010304@web.de","threadId":"28896","inReplyTo":"CABURp0rt=LcjMfDU61m0de-gLpX1a3x3vhb0zVxCbceSvD9jFw@mail.gmail.com","subject":"Re: [RFC/PATCH] add update to branch support for \"floating submodules\"","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-02-06T21:32:07Z","receivedAt":"2012-02-06T21:32:07Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 06.02.2012 18:31, schrieb Phil Hord:\n> On Wed, Feb 1, 2012 at 5:37 PM, Jens Lehmann wrote:\n>> Hmm, I really think the fact that submodules are unaware that they\n>> are part of a superproject is a feature. I'd prefer seeing that kind\n>> of problem being tackled by the CI server and/or user education. Or\n>> maybe a pre-commit hook which issues a warning in that case?\n> \n> I agree that submodule isolation is a feature essential to the\n> architecture of git and the submodules implementation. But it is also\n> a limitation, not just of this example. A pre-commit hook is a nice\n> idea, but it doesn't help 'git status' (which is the standard go-to\n> answer point for \"where am I\").\n\nYes, this feature also is a limitation. To put it in other words: I want\neach submodule to be a full fledged repo of its own. IMO it must always\nbe possible to just clone a submodule without any superproject and run\nany git command in it. And the other way around: each superproject must\nbe usable as a submodule in another superproject. That makes adding some\nkind of \"superproject awareness\" in a sane way rather difficult, as a\nrepo can't say \"I live in a superproject\" or \"I am the topmost project\".\n\n> This has me thinking more about recursing siblings now, though. I find\n> myself typing something like this quite a lot:\n> git submodule foreach 'git grep \"someFunction\" || :'\n\nThere was an attempt to teach git grep the --recurse-submodules option,\nbut unfortunately it looks like this didn't lead anywhere so far.\n\n> Or worse (in that the UI is more unwieldy):\n> git submodule foreach 'git log --oneline \"-SsomeFunction\" || :'\n\nThis could also be done by teaching git log the --recurse-submodules\noption. Me thinks in the long run a lot of git commands should learn\nthat option to make it easy to optionally include submodules in\nwhatever they are doing. But my focus is on recursive checkout for the\nnext time, so I have no idea when I find some time to do that.\n\n> But what I want is this:\n> git --git-dir=${TOP}/../.git grep --recurse-submodules \"someFunction\"\n> \n> But not really, because I am lazy and that is too much typing.\n> git grep --include-siblings \"someFunction\"\n> \n> Maybe I can add a \"sib\" macro to get this:\n> git sib grep \"someFunction\"\n\nAnd from what you where saying earlier a \"git sib status\" would be nice\ntoo? What about using alias commands for that functionality? They could\npoint to a script which searches the topmost repo and calls \"git submodule\nforeach 'git \"$@\" || :'\" from there ...\n"}]}