{"thread":{"id":"40569","subject":"Make \"git checkout\" automatically update submodules?","startedAt":"2015-10-15T22:50:05Z","lastAt":"2015-10-23T22:51:11Z","messageCount":10,"participants":["Kannan Goundan","Junio C Hamano","Christian Couder","Stefan Beller"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"271779","messageId":"loom.20151016T001449-848@post.gmane.org","threadId":"40569","inReplyTo":null,"subject":"Make \"git checkout\" automatically update submodules?","fromName":"Kannan Goundan","fromEmail":"kannan@cakoose.com","sentAt":"2015-10-15T22:50:05Z","receivedAt":"2015-10-15T22:50:05Z","isPatch":false,"sender":{"key":"kannan@cakoose.com","avatar":null},"body":"Git submodules seem to be a great fit for one of our repos.  The biggest\npain point is that it's too easy to forget to update submodules.\n\n1. I often forget since most repos don't need it.\n2. Infrequent users of our repo almost never know to update submodules and\nend up coming to us with strange build errors.\n3. Existing scripts that work with Git repos are usually not built to handle\nsubmodules.\n\nIn the common case of the submodule content having no local edits, it would\nbe nice if \"git checkout\" automatically updated submodules [1].  If there\nare local edits, it could error out (maybe override with\n\"--ignore-modified-submodules\" or something).\n\nI'm not a Git expert, though.  Is there a reason something like this isn't\nalready implemented?  Maybe there's an existing write-up or mailing list\nthread I can read to get some background information?\n\nThanks!\n\n[1] Our post-checkout procedure is:\n\n    git submodule sync\n    git submodule update --init\n    git submodule foreach --recursive \\\n      'git submodule sync ; git submodule update --init'\n\n(Not sure if this is correct.  Different articles/blogs suggest a slightly\ndifferent set of commands.)\n"},{"id":"271780","messageId":"xmqq7fmnhg4x.fsf@gitster.mtv.corp.google.com","threadId":"40569","inReplyTo":"loom.20151016T001449-848@post.gmane.org","subject":"Re: Make \"git checkout\" automatically update submodules?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-10-15T23:21:50Z","receivedAt":"2015-10-15T23:21:50Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kannan Goundan <kannan@cakoose.com> writes:\n\n> Git submodules seem to be a great fit for one of our repos.  The biggest\n> pain point is that it's too easy to forget to update submodules.\n> ...\n> In the common case of the submodule content having no local edits, it would\n> be nice if \"git checkout\" automatically updated submodules [1].  If there\n> are local edits, it could error out (maybe override with\n> \"--ignore-modified-submodules\" or something).\n>\n> I'm not a Git expert, though.  Is there a reason something like this isn't\n> already implemented?  Maybe there's an existing write-up or mailing list\n> thread I can read to get some background information?\n\nI think it is mostly because the area has a lot of corner cases and\ndifferent workflows.  For example ...\n\n> [1] Our post-checkout procedure is:\n>\n>     git submodule sync\n>     git submodule update --init\n>     git submodule foreach --recursive \\\n>       'git submodule sync ; git submodule update --init'\n\n... this will clearly not work well for everybody, as you do not\nwant to instantiate _all_ the submodules _unconditionally_.  Well,\n\"you\" (Kannan) and your project may want to, but not necessarily\nother large projects (e.g. imagine Android using submodules).\n\nSo you can view the current status being \"nothing is instantiated by\ndefault\", which _is_ far from satisfactory than the ideal case where\nperhaps the project can specify in .gitmodules which submodules are\nto be instantiated by default, add labels to modules in .gitmodules\nso that \"git clone\" of a top-level project with submodules can be\ntold \"git clone --init-submodules=<label>\" to instantiate the modules\nthat match the given label after the top-level is cloned, etc. etc.\n\nBut such a way to allow these more complicated situations to be\nhandled nicely has not been designed by anybody, so for now \"nothing\ninstantiated by default\" stands as the safest default.\n"},{"id":"272125","messageId":"loom.20151023T013752-72@post.gmane.org","threadId":"40569","inReplyTo":"xmqq7fmnhg4x.fsf@gitster.mtv.corp.google.com","subject":"Re: Make","fromName":"Kannan Goundan","fromEmail":"kannan@cakoose.com","sentAt":"2015-10-22T23:46:28Z","receivedAt":"2015-10-22T23:46:28Z","isPatch":false,"sender":{"key":"kannan@cakoose.com","avatar":null},"body":"Thanks for the explanation.  I didn't realize some projects don't want to\ninitialize all their submodules, but the explicit opt-in idea you described\nsounds nice.\n\nI've seen cases where people will financially \"sponsor\" feature development\nin open source projects.  Is there any precedent for this in the Git\nproject?  Is it ok to use this mailing list to ask about such things?\n"},{"id":"272127","messageId":"xmqq37x2qqzf.fsf@gitster.mtv.corp.google.com","threadId":"40569","inReplyTo":"loom.20151023T013752-72@post.gmane.org","subject":"Re: Make","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-10-23T02:05:24Z","receivedAt":"2015-10-23T02:05:24Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kannan Goundan <kannan@cakoose.com> writes:\n\n> Thanks for the explanation.  I didn't realize some projects don't want to\n> initialize all their submodules, but the explicit opt-in idea you described\n> sounds nice.\n>\n> I've seen cases where people will financially \"sponsor\" feature development\n> in open source projects.  Is there any precedent for this in the Git\n> project?  Is it ok to use this mailing list to ask about such things?\n\nWe are unfortunately not set up to handle money well.  For a\nbackground explanation, please go read [*1*], which I wrote my take\non \"money\" some time ago.  Note that it is an explanation and not a\njustification.  It explains why we are not set up to handle money\nwell and what the issues around money that are troublesome for the\nproject are.  It does not mean to say that it is a good thing that\nit is hard to buy feature with money from our project [*2*].\n\nI do not see (and back then in the discussion I do not think anybody\nsaw) how we can make \"We, a sponsoring company, pay this money to\nthe project to fund effort Y.\" work well.  But that of course does\nnot mean it is impossible to make it work--somebody with a fresh\nperspective may come up a way to do so, and that would be a very\nwelcome development.\n\n\n[Footnote/Reference]\n\n*1* http://thread.gmane.org/gmane.comp.version-control.git/264993/focus=265215\n\n*2* Just like my message that you are responding to was an\n    explanation of the reason why we do not recurse and and\n    initialize all submodules by default.\n"},{"id":"272128","messageId":"loom.20151023T044009-172@post.gmane.org","threadId":"40569","inReplyTo":"xmqq37x2qqzf.fsf@gitster.mtv.corp.google.com","subject":"Re: Make \"git checkout\" automatically update submodules?","fromName":"Kannan Goundan","fromEmail":"kannan@cakoose.com","sentAt":"2015-10-23T03:46:47Z","receivedAt":"2015-10-23T03:46:47Z","isPatch":false,"sender":{"key":"kannan@cakoose.com","avatar":null},"body":"Junio C Hamano <gitster <at> pobox.com> writes:\n\n> We are unfortunately not set up to handle money well.  For a\n> background explanation, please go read [*1*], which I wrote my take\n> on \"money\" some time ago.  Note that it is an explanation and not a\n> justification.  It explains why we are not set up to handle money\n> well and what the issues around money that are troublesome for the\n> project are.  It does not mean to say that it is a good thing that\n> it is hard to buy feature with money from our project [*2*].\n\nI think the way I described it (\"sponsoring a feature\") doesn't really\nreflect how I was imagining it.  In my head, it looked like this:\n\n1. Figure out whether the Git community and maintainers seem ok with the\noverall feature idea.  If not, give up.\n2. Come up with a plan for the UI/UX; see if the Git community and\nmaintainers seem ok with it.  If not, iterate or give up.\n3. Implement it, then go through the regular process of getting it merged\nupstream.  If it doesn't go well, might have to iterate or give up.\n\nI could try doing that myself, but someone familiar with the Git\ncodebase/community/history would be better at it (and probably be easier for\nyou guys to work with :-)\n\nI guess I'm just wondering if there are people who meet those qualifications\nand are interested in going through those steps for pay.  Or maybe there's a\ncompany that does this, like the old Cygnus Solutions?\n\nIn particular, I don't expect anything to change about the project's\ndevelopment process.\n\n(This part is not relevant to the Git project, but I understand that it's\nhard for anyone to guarantee a feature will make it into an open source\nproject.  I imagine these kinds of contracts are set up so that you're\nprimarily paying for the effort, not the outcome.  If it ends up not working\nout, you don't get your money back.)\n"},{"id":"272131","messageId":"CAP8UFD2cuZ=bonk49a39dDCrODH8uYUCWJ5-BT=gE1kkqcVA4Q@mail.gmail.com","threadId":"40569","inReplyTo":"loom.20151023T044009-172@post.gmane.org","subject":"Re: Make \"git checkout\" automatically update submodules?","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2015-10-23T06:19:36Z","receivedAt":"2015-10-23T06:19:36Z","isPatch":false,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Fri, Oct 23, 2015 at 5:46 AM, Kannan Goundan <kannan@cakoose.com> wrote:\n> Junio C Hamano <gitster <at> pobox.com> writes:\n>\n>> We are unfortunately not set up to handle money well.  For a\n>> background explanation, please go read [*1*], which I wrote my take\n>> on \"money\" some time ago.  Note that it is an explanation and not a\n>> justification.  It explains why we are not set up to handle money\n>> well and what the issues around money that are troublesome for the\n>> project are.  It does not mean to say that it is a good thing that\n>> it is hard to buy feature with money from our project [*2*].\n>\n> I think the way I described it (\"sponsoring a feature\") doesn't really\n> reflect how I was imagining it.  In my head, it looked like this:\n>\n> 1. Figure out whether the Git community and maintainers seem ok with the\n> overall feature idea.  If not, give up.\n> 2. Come up with a plan for the UI/UX; see if the Git community and\n> maintainers seem ok with it.  If not, iterate or give up.\n> 3. Implement it, then go through the regular process of getting it merged\n> upstream.  If it doesn't go well, might have to iterate or give up.\n>\n> I could try doing that myself, but someone familiar with the Git\n> codebase/community/history would be better at it (and probably be easier for\n> you guys to work with :-)\n>\n> I guess I'm just wondering if there are people who meet those qualifications\n> and are interested in going through those steps for pay.  Or maybe there's a\n> company that does this, like the old Cygnus Solutions?\n\nWell I am starting to do that for Booking.com. Not sure if it will\nalso be possible for me to work for you as I also work on IPFS\n(http://ipfs.io) for the company that develops it, but we can discuss\nit privately.\n\nThere was David Kastrup (dak at gnu.org) who previously said he could\nbe interested in such jobs. We wrote a very short article about it in\nthe first edition of Git Rev News last March:\n\nhttp://git.github.io/rev_news/2015/03/25/edition-1/\n\nWe also wrote a very short article \"Job Offer\" article about\nBooking.com looking for Git developers in Git Rev News in the third\nedition last May:\n\nhttp://git.github.io/rev_news/2015/05/13/edition-3/\n\nso if you want we can write a similar \"Job Offer\" article in the next\nGit Rev News edition.\n\nYou can even propose such an article yourself by editing the draft of\nthe next edition here:\n\nhttps://github.com/git/git.github.io/blob/master/rev_news/drafts/edition-9.md\n\nand then creating a pull request.\n\n> In particular, I don't expect anything to change about the project's\n> development process.\n>\n> (This part is not relevant to the Git project, but I understand that it's\n> hard for anyone to guarantee a feature will make it into an open source\n> project.  I imagine these kinds of contracts are set up so that you're\n> primarily paying for the effort, not the outcome.  If it ends up not working\n> out, you don't get your money back.)\n"},{"id":"272146","messageId":"xmqqk2qdpku3.fsf@gitster.mtv.corp.google.com","threadId":"40569","inReplyTo":"loom.20151023T044009-172@post.gmane.org","subject":"Re: Make \"git checkout\" automatically update submodules?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-10-23T17:15:48Z","receivedAt":"2015-10-23T17:15:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kannan Goundan <kannan@cakoose.com> writes:\n\n> I think the way I described it (\"sponsoring a feature\") doesn't really\n> reflect how I was imagining it.  In my head, it looked like this:\n> ...\n> I could try doing that myself, but someone familiar with the Git\n> codebase/community/history would be better at it (and probably be easier for\n> you guys to work with :-)\n>\n> I guess I'm just wondering if there are people who meet those qualifications\n> and are interested in going through those steps for pay.  Or maybe there's a\n> company that does this, like the old Cygnus Solutions?\n>\n> In particular, I don't expect anything to change about the project's\n> development process.\n\nIn other words, the sponsoring entity is paying for effort and not\nfor result--money does not buy inclusion.\n\nThat may be workable from the project's point of view; I however\nwonder if that is workable from the sponsor's point of view.  Things\nthat may be problematic inside the sponsoring entity (i.e. between\nthose with money and those who interact with the hired person)\ninclude:\n\n - Does the hired person really meet the right qualifications?\n\n - Did the hired person make a good faith effort?\n\n - Was it the right time to start the topic, or was the codebase too\n   much in flux at the moment to accept work in that area?\n\nBut these are problems between the sponsor and the hired person and\ndo not concern us ;-)\n"},{"id":"272148","messageId":"CAGZ79kYMyU5h8uQkNEpMU558FgLrNM52_aQsXud6CrBAUgFSNA@mail.gmail.com","threadId":"40569","inReplyTo":"loom.20151016T001449-848@post.gmane.org","subject":"Re: Make \"git checkout\" automatically update submodules?","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-23T17:20:15Z","receivedAt":"2015-10-23T17:20:15Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Oct 15, 2015 at 3:50 PM, Kannan Goundan <kannan@cakoose.com> wrote:\n> Git submodules seem to be a great fit for one of our repos.  The biggest\n> pain point is that it's too easy to forget to update submodules.\n>\n> 1. I often forget since most repos don't need it.\n> 2. Infrequent users of our repo almost never know to update submodules and\n> end up coming to us with strange build errors.\n> 3. Existing scripts that work with Git repos are usually not built to handle\n> submodules.\n>\n> In the common case of the submodule content having no local edits, it would\n> be nice if \"git checkout\" automatically updated submodules [1].  If there\n> are local edits, it could error out (maybe override with\n> \"--ignore-modified-submodules\" or something).\n>\n> I'm not a Git expert, though.  Is there a reason something like this isn't\n> already implemented?  Maybe there's an existing write-up or mailing list\n> thread I can read to get some background information?\n>\n> Thanks!\n>\n> [1] Our post-checkout procedure is:\n>\n>     git submodule sync\n>     git submodule update --init\n>     git submodule foreach --recursive \\\n>       'git submodule sync ; git submodule update --init'\n>\n> (Not sure if this is correct.  Different articles/blogs suggest a slightly\n> different set of commands.)\n>\n\nCheckout [1]. There are lots of good patches, but hard to find.\n(Including, but not limited to a recursive git checkout enhancement!)\n\nThat said I've recently started working on submodules, too.\nI am trying to push my work upstream as fast as possible\nas that works best for us.\n\n[1] https://github.com/jlehmann/git-submod-enhancements/wiki\n"},{"id":"272153","messageId":"xmqq37x1pfhl.fsf@gitster.mtv.corp.google.com","threadId":"40569","inReplyTo":"CAGZ79kYMyU5h8uQkNEpMU558FgLrNM52_aQsXud6CrBAUgFSNA@mail.gmail.com","subject":"Re: Make \"git checkout\" automatically update submodules?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-10-23T19:11:18Z","receivedAt":"2015-10-23T19:11:18Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> Checkout [1]. There are lots of good patches, but hard to find.\n> (Including, but not limited to a recursive git checkout enhancement!)\n> ...\n> [1] https://github.com/jlehmann/git-submod-enhancements/wiki\n\nYes, Jens is not just one of the people who have been working on\nharder, and thinking longer about, submodules than anybody else, but\nalso has demonstrated that he has good taste and balanced view on\nthe design of the subsystem over time, whose technical judgment we\ncan trust.\n\nNot all the changes listed on the page may necessarily be good as-is\n(e.g. some may help only a subset of users while hurting others,\nlike the \"recursively check-out everything unconditionally\" that\ntrigerred this thread), but the page has a good collection to remind\nanybody, who designs a coherent whole, of issues that need to be\ntaken into account.\n\nThanks for a pointer to an excellent starting page.\n"},{"id":"272158","messageId":"loom.20151024T003828-264@post.gmane.org","threadId":"40569","inReplyTo":"CAGZ79kYMyU5h8uQkNEpMU558FgLrNM52_aQsXud6CrBAUgFSNA@mail.gmail.com","subject":"Re: Make \"git checkout\" automatically update submodules?","fromName":"Kannan Goundan","fromEmail":"kannan@cakoose.com","sentAt":"2015-10-23T22:51:11Z","receivedAt":"2015-10-23T22:51:11Z","isPatch":false,"sender":{"key":"kannan@cakoose.com","avatar":null},"body":"Stefan Beller <sbeller <at> google.com> writes:\n\n> [1] https://github.com/jlehmann/git-submod-enhancements/wiki\n\nOh wow, Christmas came early!  I'll give this code a try.\n"}]}