{"thread":{"id":"12936","subject":"Migrating svn to git with heavy use of externals","startedAt":"2008-03-31T20:59:00Z","lastAt":"2008-04-18T18:30:03Z","messageCount":48,"participants":["D. Stuart Freeman","Avery Pennarun","Roman Shaposhnik","Junio C Hamano","Ping Yin","Roman V. Shaposhnik","Jeremy Maitin-Shepard","Linus Torvalds","Martin Langhoff","Sverre Rabbelier","Dmitry Potapov","Jakub Narebski"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"73457","messageId":"47F15094.5050808@et.gatech.edu","threadId":"12936","inReplyTo":null,"subject":"Migrating svn to git with heavy use of externals","fromName":"D. Stuart Freeman","fromEmail":"stuart.freeman@et.gatech.edu","sentAt":"2008-03-31T20:59:00Z","receivedAt":"2008-03-31T20:59:00Z","isPatch":false,"sender":{"key":"stuart.freeman@et.gatech.edu","avatar":"https://gravatar.com/avatar/d2c9483eaa2fa28008c84e68b765886b118c445b1ddf2ad0d0fe7ba814127a9a?d=mp&s=160"},"body":"I'm a developer on the Sakai project.  I think Sakai could benefit\ngreatly from use of git because we have a huge need to track local\nchanges while contributing back to a central codebase.  I've started\nlooking at git-svn and have managed to get a copy of our repository into\ngit, and looked at the stuff to do with submodules as a replacement for\nexternals.  The problem is we rely very heavily on externals, for\ninstance when we make a tag for release we tag all the modules at the\nsame time and use an externals file to build the release from those\ntags.  I realize that's probably not a best practice, but it's what we\ndo.  Our latest release is here:\nhttps://source.sakaiproject.org/svn/sakai/tags/sakai_2-5-0/ if you want\nto get an idea of the scope of the problem.  How would you convert this\nto a git repository?  I'm currently looking at\nhttp://blog.alieniloquent.com/2008/03/08/git-svn-with-svnexternals/ but\nthat doesn't look like it would leave all the old release tags intact.\n\n-- \nD. Stuart Freeman\nGeorgia Institute of Technology\n\n\nbegin:vcard\nfn:D. Stuart Freeman\nn:Freeman;Douglas\nemail;internet:stuart.freeman@et.gatech.edu\ntel;work:(404)385-1473\nx-mozilla-html:FALSE\nversion:2.1\nend:vcard\n\n"},{"id":"73859","messageId":"47FBB448.3060900@et.gatech.edu","threadId":"12936","inReplyTo":"47F15094.5050808@et.gatech.edu","subject":"Re: Migrating svn to git with heavy use of externals","fromName":"D. Stuart Freeman","fromEmail":"stuart.freeman@et.gatech.edu","sentAt":"2008-04-08T18:07:04Z","receivedAt":"2008-04-08T18:07:04Z","isPatch":false,"sender":{"key":"stuart.freeman@et.gatech.edu","avatar":"https://gravatar.com/avatar/d2c9483eaa2fa28008c84e68b765886b118c445b1ddf2ad0d0fe7ba814127a9a?d=mp&s=160"},"body":"Maybe I should clarify.\nI've imported an svn managed project into a git repository\nwith 71 submodules, what I don't understand though is if I\nhave a branch called 2-5-x and another called 2-4-x in each of\nthe submodules and the superproject, is there a way to\nassociate those?\n\nD. Stuart Freeman wrote:\n> I'm a developer on the Sakai project.  I think Sakai could benefit\n> greatly from use of git because we have a huge need to track local\n> changes while contributing back to a central codebase.  I've started\n> looking at git-svn and have managed to get a copy of our repository into\n> git, and looked at the stuff to do with submodules as a replacement for\n> externals.  The problem is we rely very heavily on externals, for\n> instance when we make a tag for release we tag all the modules at the\n> same time and use an externals file to build the release from those\n> tags.  I realize that's probably not a best practice, but it's what we\n> do.  Our latest release is here:\n> https://source.sakaiproject.org/svn/sakai/tags/sakai_2-5-0/ if you want\n> to get an idea of the scope of the problem.  How would you convert this\n> to a git repository?  I'm currently looking at\n> http://blog.alieniloquent.com/2008/03/08/git-svn-with-svnexternals/ but\n> that doesn't look like it would leave all the old release tags intact.\n> \n\n\n-- \nD. Stuart Freeman\nGeorgia Institute of Technology\n\n\nbegin:vcard\nfn:D. Stuart Freeman\nn:Freeman;Douglas\nemail;internet:stuart.freeman@et.gatech.edu\ntel;work:(404)385-1473\nx-mozilla-html:FALSE\nversion:2.1\nend:vcard\n\n"},{"id":"73869","messageId":"32541b130804081306q6e06af20u794357eba9d434e@mail.gmail.com","threadId":"12936","inReplyTo":"47FBB448.3060900@et.gatech.edu","subject":"Re: Migrating svn to git with heavy use of externals","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-08T20:06:37Z","receivedAt":"2008-04-08T20:06:37Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Tue, Apr 8, 2008 at 2:07 PM, D. Stuart Freeman\n<stuart.freeman@et.gatech.edu> wrote:\n> Maybe I should clarify.\n>  I've imported an svn managed project into a git repository\n>  with 71 submodules, what I don't understand though is if I\n>  have a branch called 2-5-x and another called 2-4-x in each of\n>  the submodules and the superproject, is there a way to\n>  associate those?\n\nI don't think git-svn currently knows how to import svn:externals\nproperly.  Basically you'd have to do it yourself, perhaps with the\nhelp of something like git-filter-branch and a shell script.\n\nThe equivalent of svn:externals in git is called git-submodule, and\nit's actually much more powerful than svn:externals, because you can\nlink to a *specific revision* and not just a branch.  In other words,\nI can set up my application to point at r2956 of a library, so even if\nthe library changes in the future, my application always gets exactly\nthat version.  (To have the app use the later version, you have to\n'git pull' in the submodule, then make a commit in the application\nmodule.)\n\nSee \"man git-submodule\" and \"man git-filter-branch\" for more information.\n\nIf I'm wrong and git-svn already supports svn:externals, I'm sure\nsomeone will correct me :)\n\nHave fun,\n\nAvery\n"},{"id":"73878","messageId":"47FBDA77.2050402@et.gatech.edu","threadId":"12936","inReplyTo":"32541b130804081306q6e06af20u794357eba9d434e@mail.gmail.com","subject":"Re: Migrating svn to git with heavy use of externals","fromName":"D. Stuart Freeman","fromEmail":"stuart.freeman@et.gatech.edu","sentAt":"2008-04-08T20:49:59Z","receivedAt":"2008-04-08T20:49:59Z","isPatch":false,"sender":{"key":"stuart.freeman@et.gatech.edu","avatar":"https://gravatar.com/avatar/d2c9483eaa2fa28008c84e68b765886b118c445b1ddf2ad0d0fe7ba814127a9a?d=mp&s=160"},"body":"Avery Pennarun wrote:\n> On Tue, Apr 8, 2008 at 2:07 PM, D. Stuart Freeman\n> <stuart.freeman@et.gatech.edu> wrote:\n>> Maybe I should clarify.\n>>  I've imported an svn managed project into a git repository\n>>  with 71 submodules, what I don't understand though is if I\n>>  have a branch called 2-5-x and another called 2-4-x in each of\n>>  the submodules and the superproject, is there a way to\n>>  associate those?\n> \n> I don't think git-svn currently knows how to import svn:externals\n> properly.  Basically you'd have to do it yourself, perhaps with the\n> help of something like git-filter-branch and a shell script.\n> \n> The equivalent of svn:externals in git is called git-submodule, and\n> it's actually much more powerful than svn:externals, because you can\n> link to a *specific revision* and not just a branch.  In other words,\n> I can set up my application to point at r2956 of a library, so even if\n> the library changes in the future, my application always gets exactly\n> that version.  (To have the app use the later version, you have to\n> 'git pull' in the submodule, then make a commit in the application\n> module.)\n> \n> See \"man git-submodule\" and \"man git-filter-branch\" for more information.\n> \n> If I'm wrong and git-svn already supports svn:externals, I'm sure\n> someone will correct me :)\n> \n> Have fun,\n> \n> Avery\n\nIt's possible to have svn:externals point at a specific revision, but\nthat's not the point.  I'm convinced that submodules are the answer, I'm\njust not sure how to make them work. Assume \"sakai\" is the superproject\nand \"access\" is a submodule, I've done:\n\ncd sakai\ngit checkout work\ngit submodule add ../access access\n\nAnd that's cool, but then I do:\n\ncd ../access\ngit checkout -b 2-5-x sakai_2-5-x  # sakai_2-5-x is an svn import\ncd ../sakai\ngit checkout -b 2-5-x sakai_2-5-x\ngit submodule add -b 2-5-x ../access access\n\nWhich gives me an error about access already existing.  I'm pretty sure\nI'm just not thinking about this the way git does, I blame svn for\ndamaging my brain.\n\n-- \nD. Stuart Freeman\nGeorgia Institute of Technology\n\n\nbegin:vcard\nfn:D. Stuart Freeman\nn:Freeman;Douglas\nemail;internet:stuart.freeman@et.gatech.edu\ntel;work:(404)385-1473\nx-mozilla-html:FALSE\nversion:2.1\nend:vcard\n\n"},{"id":"73879","messageId":"32541b130804081401n743f39c9o3f016da9dee2eb92@mail.gmail.com","threadId":"12936","inReplyTo":"47FBDA77.2050402@et.gatech.edu","subject":"Re: Migrating svn to git with heavy use of externals","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-08T21:01:05Z","receivedAt":"2008-04-08T21:01:05Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Tue, Apr 8, 2008 at 4:49 PM, D. Stuart Freeman\n<stuart.freeman@et.gatech.edu> wrote:\n>  It's possible to have svn:externals point at a specific revision, but\n>  that's not the point.\n\nRight, I forgot that they added that.\n\n>  cd ../access\n>  git checkout -b 2-5-x sakai_2-5-x  # sakai_2-5-x is an svn import\n>  cd ../sakai\n>  git checkout -b 2-5-x sakai_2-5-x\n>  git submodule add -b 2-5-x ../access access\n>\n>  Which gives me an error about access already existing.\n\nYou should only ever need to add a given submodule once.  As far as I\ncan tell, this is a bit of confusion in the way git-submodule works,\nbut I don't have any suggestions for what to do about it yet so I\ndon't complain :)\n\nThe way to understand git-submodule's operation is in terms of what it\nactually does.  Roughly speaking, git-submodule-add puts things into\n.gitmodules and .git/config; git-submodule-init copies that stuff from\n.gitmodules to .git/config (so if you're the guy who did the add, you\ncan skip this step).  Then git-commit actually checks the submodule\nreference into the parent tree, and someone who pulls your parent tree\nneeds to run git-submodule-update in order to actually retrieve the\nnew submodule pointer.\n\nIn other words, git-submodule is very powerful, but also very\ncomplicated, and at least one of the things I said up above is\nprobably wrong :)  By comparison, svn:externals is at least easy to\nunderstand.\n\nAnyway, in this case, what you need to know is that .git/config\nalready contains your submodule information.  Sadly, .gitmodules is\nprobably sitting somewhere on your original branch, so it probably\ndoesn't exist.  You could remove the entry from .git/config by hand\nand use git-submodule-add again (thus putting it in both places), or\ncopy the .gitmodules file from the original branch, or git-cherry-pick\nthe commit where you added it.\n\nYou should *also* cd into the access subdir and checkout the right\nrevision there; at that time, the next commit to the sakai repository\nwill make sure the submodule reference is to the right place.\n\nPhew, I hope that made things more clear instead of less clear. :)\n\nHave fun,\n\nAvery\n"},{"id":"73890","messageId":"47FBF5F7.1060105@et.gatech.edu","threadId":"12936","inReplyTo":"32541b130804081401n743f39c9o3f016da9dee2eb92@mail.gmail.com","subject":"Re: Migrating svn to git with heavy use of externals","fromName":"D. Stuart Freeman","fromEmail":"stuart.freeman@et.gatech.edu","sentAt":"2008-04-08T22:47:19Z","receivedAt":"2008-04-08T22:47:19Z","isPatch":false,"sender":{"key":"stuart.freeman@et.gatech.edu","avatar":"https://gravatar.com/avatar/d2c9483eaa2fa28008c84e68b765886b118c445b1ddf2ad0d0fe7ba814127a9a?d=mp&s=160"},"body":"Avery Pennarun wrote:\n> Anyway, in this case, what you need to know is that .git/config\n> already contains your submodule information.  Sadly, .gitmodules is\n> probably sitting somewhere on your original branch, so it probably\n> doesn't exist.  You could remove the entry from .git/config by hand\n> and use git-submodule-add again (thus putting it in both places), or\n> copy the .gitmodules file from the original branch, or git-cherry-pick\n> the commit where you added it.\n> \n> You should *also* cd into the access subdir and checkout the right\n> revision there; at that time, the next commit to the sakai repository\n> will make sure the submodule reference is to the right place.\n> \n> Phew, I hope that made things more clear instead of less clear. :)\n> \n> Have fun,\n> \n> Avery\n\nOK, this makes a lot more sense now.  Thanks.\n\n-- \nStuart\n"},{"id":"73915","messageId":"8FE3B7A7-4C2D-4202-A5FC-EBC4F4670273@sun.com","threadId":"12936","inReplyTo":"32541b130804081401n743f39c9o3f016da9dee2eb92@mail.gmail.com","subject":"Re: Migrating svn to git with heavy use of externals","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-09T03:03:58Z","receivedAt":"2008-04-09T03:03:58Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Apr 8, 2008, at 2:01 PM, Avery Pennarun wrote:\n> The way to understand git-submodule's operation is in terms of what it\n> actually does.  Roughly speaking, git-submodule-add puts things into\n> .gitmodules and .git/config;\n\nI could be mistaken, but I don't think \"git submodule add\" does anything\nto the .git/config. In fact, how settings migrate between .gitmodules\nand .git/config has been a long standing source of slight confusion\nfor me.\n\nPlease correct me if I'm wrong, but it seems that the only reason\nfor the file .gitmodules to be there at all is because it can be\nrevved through commits, just as any file under Git's control.\n.git/config doesn't have such a property. Other than that, it is not\nreally needed, right?\n\n> In other words, git-submodule is very powerful, but also very\n> complicated,\n\nSpeaking of complications, it took me awhile to realize that 90%\nof the Submodule magic seems to be based on the secret ability of\ntree objects to hold references not only to blobs and trees but\nalso to *commits*:\n     $ git init\n     $ mkdir foo ; cd foo\n     $ git init\n     $ touch file\n     $ git add file\n     $ git commit -mInit\n     Created initial commit 5a61c46: Init\n         0 files changed, 0 insertions(+), 0 deletions(-)\n         create mode 100644 file\nNow observe:\n     $ cd ..\n     $ git add foo\n     $ git ls-files --stage\n     160000 5a61c4698ca56004c2532ce02e6736cfd2e977d1 0\tfoo\nThe  '5a61c46' is simply a reference to the commit we've just created.\nOf course, it is  insufficient to know what commit \"foo\" refers\nto unless we also know what Git repo this commit resides in. And that's\nwhere the mapping between a path (\"foo\") and  a url pointing to\nthe submodule comes into play.\n\nThis is a cool property and it lets you build functionality like\n\"git submodules\" in a variety of different ways. I just wish it\nwas documented somewhere.\n\n> Anyway, in this case, what you need to know is that .git/config\n> already contains your submodule information.  Sadly, .gitmodules is\n> probably sitting somewhere on your original branch, so it probably\n> doesn't exist.  You could remove the entry from .git/config by hand\n> and use git-submodule-add again (thus putting it in both places), or\n> copy the .gitmodules file from the original branch, or git-cherry-pick\n> the commit where you added it.\n\nThat's exactly what makes me doubtful about .gitmodules being the\nbest place for storing the url, but then again, I don't have any\nbetter ideas. :-( Yet ;-)\n\nThanks,\nRoman.\n"},{"id":"73916","messageId":"32541b130804082033q55c795b5ieaa4e120956ff030@mail.gmail.com","threadId":"12936","inReplyTo":"8FE3B7A7-4C2D-4202-A5FC-EBC4F4670273@sun.com","subject":"Re: Migrating svn to git with heavy use of externals","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-09T03:33:49Z","receivedAt":"2008-04-09T03:33:49Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Tue, Apr 8, 2008 at 11:03 PM, Roman Shaposhnik <rvs@sun.com> wrote:\n> On Apr 8, 2008, at 2:01 PM, Avery Pennarun wrote:\n> > The way to understand git-submodule's operation is in terms of what it\n> > actually does.  Roughly speaking, git-submodule-add puts things into\n> > .gitmodules and .git/config;\n>\n>  I could be mistaken, but I don't think \"git submodule add\" does anything\n>  to the .git/config. In fact, how settings migrate between .gitmodules\n>  and .git/config has been a long standing source of slight confusion\n>  for me.\n>\n>  Please correct me if I'm wrong, but it seems that the only reason\n>  for the file .gitmodules to be there at all is because it can be\n>  revved through commits, just as any file under Git's control.\n>  .git/config doesn't have such a property. Other than that, it is not\n>  really needed, right?\n\nYou have the last paragraph right, but I think the first paragraph wrong :)\n\n.gitmodules doesn't do anything unless git-submodule reads it, which\nit does in git-submodule-init and git-submodule-add.  (You know\ngit-submodule-add is screwing with .git/config because you don't need\nto call git-submodule-init when you use it.)  git-submodule-update,\nAFAICT, just reads .git/config.\n\n>  Speaking of complications, it took me awhile to realize that 90%\n>  of the Submodule magic seems to be based on the secret ability of\n>  tree objects to hold references not only to blobs and trees but\n>  also to *commits*:\n\nIndeed, this is the majority of the coolness right there.  The rest of\nthe screwiness with .gitmodules and so on is really just to support\nfetching the objects for the submodules from repos than the primary\nsupermodule one.\n\nAlso, git-checkout seems to explicitly *not* checkout refs to commits\nby itself; you have to call git-submodule-update for that.  This is\nprobably because git-checkout wouldn't know what to do if the\nsubmodule were dirty (ie. the sub-checkout couldn't complete because\nfiles had been changed but not checked in).  This is useful in a\nnot-destroying-my-data way, but the behaviour isn't too obvious or\ncoherent.\n\n>  That's exactly what makes me doubtful about .gitmodules being the\n>  best place for storing the url, but then again, I don't have any\n>  better ideas. :-( Yet ;-)\n\nThere's definitely no better place; .git/config isn't versioned, and\nURLs don't belong in the tree objects themselves, which are otherwise\nlocation-neutral and transport-neutral.\n\nIn my own use case, I think having all the objects from the\nsupermodule *and* submodules all be in the same repo is what I want.\nThis kind of obviates the need for .gitmodules entirely, if\ngit-checkout and friends will do the right thing.  I think I'll submit\nsome patches eventually once I have this figured out properly.\n\nHave fun,\n\nAvery\n"},{"id":"73919","messageId":"49E9DCEC-8A9E-4AD7-BA58-5A40F475F2EA@sun.com","threadId":"12936","inReplyTo":"32541b130804082033q55c795b5ieaa4e120956ff030@mail.gmail.com","subject":"Re: Migrating svn to git with heavy use of externals","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-09T04:39:01Z","receivedAt":"2008-04-09T04:39:01Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Apr 8, 2008, at 8:33 PM, Avery Pennarun wrote:\n\n> On Tue, Apr 8, 2008 at 11:03 PM, Roman Shaposhnik <rvs@sun.com> wrote:\n>> On Apr 8, 2008, at 2:01 PM, Avery Pennarun wrote:\n>>> The way to understand git-submodule's operation is in terms of  \n>>> what it\n>>> actually does.  Roughly speaking, git-submodule-add puts things into\n>>> .gitmodules and .git/config;\n>>\n>> I could be mistaken, but I don't think \"git submodule add\" does  \n>> anything\n>> to the .git/config. In fact, how settings migrate between .gitmodules\n>> and .git/config has been a long standing source of slight confusion\n>> for me.\n>>\n>> Please correct me if I'm wrong, but it seems that the only reason\n>> for the file .gitmodules to be there at all is because it can be\n>> revved through commits, just as any file under Git's control.\n>> .git/config doesn't have such a property. Other than that, it is not\n>> really needed, right?\n>\n> You have the last paragraph right, but I think the first paragraph  \n> wrong :)\n\nWell, may be we are talking about slightly different things, or   \nthere's a version mismatch,\nbut here's what I get with 1.5.4.5:\n    $ git init\n    $ git submodule add /tmp/GIT/1\n    Initialized empty Git repository in /tmp/GIT/3/1/.git/\n     $ cat .git/config\n     [core]\n\trepositoryformatversion = 0\n\tfilemode = true\n\tbare = false\n\tlogallrefupdates = true\nNow, .gitmodules is there alright, so if I do:\n     $ git submodule init\nI get the migration of settings to .git/config:\n      $ cat .git/config\n      [core]\n\trepositoryformatversion = 0\n\tfilemode = true\n\tbare = false\n\tlogallrefupdates = true\n      [submodule \"1\"]\n\turl = /tmp/GIT/1/.git\n\n>> Speaking of complications, it took me awhile to realize that 90%\n>> of the Submodule magic seems to be based on the secret ability of\n>> tree objects to hold references not only to blobs and trees but\n>> also to *commits*:\n>\n> Indeed, this is the majority of the coolness right there.  The rest of\n> the screwiness with .gitmodules and so on is really just to support\n> fetching the objects for the submodules from repos than the primary\n> supermodule one.\n\nYeap. It all reminds me a bit of symbolic links in file systems. With  \nthe\nkey difference being that symbolic links can only point to an object,\nwhere we can actually reference a particular *state* of that object.\n\nI like this ability very much. It comes especially handy when a single\ncomponent participates in multiple superprojects (being referenced\nas a submodule) and every single one of them can reference a state\nof the component that they like.\n\nThat said, I still can't quite figure out how to do a very basic thing:\nhow can I change the SHA1 that a tree objects refers to without\nchecking out a corresponding submodule first? IOW, suppose I've\njust cloned ~/Superproject into /tmp/super-clone. All of the submodules\nare still empty (nothing has been cheked out into them yet) and all\nI want to do is to bump the version of one submodule\n     /tmp/super-clone/foo\nand then push the changes back to the ~/Superproject, so that everybody\nwho pulls from it get the newer foo. It seems that the procedure  \noutlined\nin Git's manual seems to be pretty heavyweight for such a simple thing.\n\n>> That's exactly what makes me doubtful about .gitmodules being the\n>> best place for storing the url, but then again, I don't have any\n>> better ideas. :-( Yet ;-)\n>\n> There's definitely no better place; .git/config isn't versioned, and\n> URLs don't belong in the tree objects themselves, which are otherwise\n> location-neutral and transport-neutral.\n\nAgreed. But I guess I'd be less confused if \"git submodule\" didn't muck\nwith .git/config at all. Or are there any other consumers of the  \ninformation\nthat it puts there (except itself)?\n\n> In my own use case, I think having all the objects from the\n> supermodule *and* submodules all be in the same repo is what I want.\n> This kind of obviates the need for .gitmodules entirely, if\n> git-checkout and friends will do the right thing.  I think I'll submit\n> some patches eventually once I have this figured out properly.\n\nHm. But what about those who might want to pull from you? .git/config\ndoesn't propagate, which means that they'll be kind of stuck, don't\nyou think?\n\nThanks,\nRoman.\n"},{"id":"73923","messageId":"32541b130804082334s604b62b0j82b510c331f48213@mail.gmail.com","threadId":"12936","inReplyTo":"49E9DCEC-8A9E-4AD7-BA58-5A40F475F2EA@sun.com","subject":"Re: Migrating svn to git with heavy use of externals","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-09T06:34:58Z","receivedAt":"2008-04-09T06:34:58Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Wed, Apr 9, 2008 at 12:39 AM, Roman Shaposhnik <rvs@sun.com> wrote:\n>  Agreed. But I guess I'd be less confused if \"git submodule\" didn't muck\n>  with .git/config at all. Or are there any other consumers of the\n> information\n>  that it puts there (except itself)?\n\nThat I don't know.  If there aren't any others, then I agree, I'm not\nsure what the whole .git/config messing is about.\n\n> > In my own use case, I think having all the objects from the\n> > supermodule *and* submodules all be in the same repo is what I want.\n> > This kind of obviates the need for .gitmodules entirely, if\n> > git-checkout and friends will do the right thing.  I think I'll submit\n> > some patches eventually once I have this figured out properly.\n>\n>  Hm. But what about those who might want to pull from you? .git/config\n>  doesn't propagate, which means that they'll be kind of stuck, don't\n>  you think?\n\nNot exactly.  The idea is that if the supermodule and submodules are\nall lumped into a single repo (and your refs are set up correctly),\nthen cloning the supermodule will also clone all the submodules.  So\neveryone will have all the necessary refs anyway; as long as\ngit-checkout checks them out, .gitmodules shouldn't have to exist at\nall, becaues there's nothing \"special\" for git-submodule to do.\n\nHave fun,\n\nAvery\n"},{"id":"73925","messageId":"7vhcebcyty.fsf@gitster.siamese.dyndns.org","threadId":"12936","inReplyTo":"32541b130804082334s604b62b0j82b510c331f48213@mail.gmail.com","subject":"Re: Migrating svn to git with heavy use of externals","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-09T06:43:53Z","receivedAt":"2008-04-09T06:43:53Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Avery Pennarun\" <apenwarr@gmail.com> writes:\n\n> On Wed, Apr 9, 2008 at 12:39 AM, Roman Shaposhnik <rvs@sun.com> wrote:\n>>  Agreed. But I guess I'd be less confused if \"git submodule\" didn't muck\n>>  with .git/config at all. Or are there any other consumers of the\n>> information\n>>  that it puts there (except itself)?\n>\n> That I don't know.  If there aren't any others, then I agree, I'm not\n> sure what the whole .git/config messing is about.\n\nIts actually the other way around.\n\nIn-tree .gitmodules is used to give hints to prime what is placed in\n.git/config, which after initialized should serve as the authoritative\ninformation on managed submodules as far as your repository is concerned.\n\"git submodule init\" may be a handy way to do this \"priming\", but you do\nnot necessarily have to use it but instead manually adjust .git/config\nyourself; this is so that you can configure remote url that is different\nfrom what .gitmodules suggests to suite your local needs.\n\nAlthough putting everything in a single repository could work, that does\nnot have to be the only way to work with submodules.  In fact, the basic\nsubmodule design is trying very hard not to force you to grab objects that\nare needed for all submodules when you are cloning the superproject, as\nnot cloning nor checking out any submodule is the default.\n"},{"id":"73974","messageId":"1207771054.13123.228.camel@work.sfbay.sun.com","threadId":"12936","inReplyTo":"32541b130804082334s604b62b0j82b510c331f48213@mail.gmail.com","subject":"Intricacies of submodules [was: Migrating svn to git with heavy use of externals]","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-09T19:57:34Z","receivedAt":"2008-04-09T19:57:34Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Wed, 2008-04-09 at 02:34 -0400, Avery Pennarun wrote:\n> On Wed, Apr 9, 2008 at 12:39 AM, Roman Shaposhnik <rvs@sun.com> wrote:\n> > > In my own use case, I think having all the objects from the\n> > > supermodule *and* submodules all be in the same repo is what I want.\n> > > This kind of obviates the need for .gitmodules entirely, if\n> > > git-checkout and friends will do the right thing.  I think I'll submit\n> > > some patches eventually once I have this figured out properly.\n> >\n> >  Hm. But what about those who might want to pull from you? .git/config\n> >  doesn't propagate, which means that they'll be kind of stuck, don't\n> >  you think?\n> \n> Not exactly.  The idea is that if the supermodule and submodules are\n> all lumped into a single repo (and your refs are set up correctly),\n> then cloning the supermodule will also clone all the submodules.  \n\nInteresting! How do you make it happen? Do you use git hooks or\nsomething? On my end, I can't really reproduce that behavior of clone\nbut I would very much like to:\n  $ alias mkrepo=\"git init; touch file; git add file; git commit -mInit\"\n  $ mkdir super ; cd super\n  $ mkrepo\n  $ mkdir submodule ; cd submodule\n  $ mkrepo\n  $ cd ..\n  $ git submodule add submodule\n  Adding existing repo at 'submodule' to the index\n  $ git commit -mSubmodule\n  Created commit 5921c87: Submodule\n     2 files changed, 4 insertions(+), 0 deletions(-) \n     create mode 100644 .gitmodules\n     create mode 160000 submodule\n\nNow, when I clone super I don't actually have submodule cloned:\n   $ git clone super super-clone\n   $ cd  super-clone\n   $ git submodule status\n   -7482d0433ed681aa243629f13cd97ca5be242393 submodule\n\nIn fact, it seems that I can't even do \"submodule update\", which\nseems like a bug to me, by the way:\n    $ git submodule init  \n    Submodule 'submodule' (submodule) registered for path 'submodule'\n\n    $ git submodule update\n    Initialized empty Git repository in /tmp/TEST/super-clone/submodule/.git/\n    fatal: no matching remote head\n    fetch-pack from 'submodule' failed.\n    Clone of 'submodule' into submodule path 'submodule' failed\n\nAny ideas on what's going on here? Or what am I doing wrong?\n\n> So everyone will have all the necessary refs anyway; as long as\n> git-checkout checks them out, .gitmodules shouldn't have to exist at\n> all, becaues there's nothing \"special\" for git-submodule to do.\n\nI would very much like to have that, yes. Please do provide additional\ndetails on how's your setup is different from mine.\n\nThanks,\nRoman.\n"},{"id":"73979","messageId":"32541b130804091327x475efc4cw978b336769f2ff9e@mail.gmail.com","threadId":"12936","inReplyTo":"1207771054.13123.228.camel@work.sfbay.sun.com","subject":"Re: Intricacies of submodules [was: Migrating svn to git with heavy use of externals]","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-09T20:27:58Z","receivedAt":"2008-04-09T20:27:58Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Wed, Apr 9, 2008 at 3:57 PM, Roman Shaposhnik <rvs@sun.com> wrote:\n> On Wed, 2008-04-09 at 02:34 -0400, Avery Pennarun wrote:\n>  > So everyone will have all the necessary refs anyway; as long as\n>  > git-checkout checks them out, .gitmodules shouldn't have to exist at\n>  > all, becaues there's nothing \"special\" for git-submodule to do.\n>\n>  I would very much like to have that, yes. Please do provide additional\n>  details on how's your setup is different from mine.\n\nSorry, I wasn't clear.  I meant that there's no fundamental reason\nthat this shouldn't be possible; as far as I know, there's no way to\nmake git do this in an obvious way (yet).\n\nIt's encouraging that other people seem to want the same behaviour as\nme, which means I might get to working on it sooner :)  Not that this\nshould discourage you from trying, of course: perhaps I'm just missing\nsomething too.\n\nNote: I think a big part of the secret is using \".\" as the location of\nthe submodule's repository in .gitmodules.  \"git submodule add\" seems\nto expand . to a full path, but you can change it by hand if you edit\nthe file.\n\nHave fun,\n\nAvery\n"},{"id":"74012","messageId":"6CFA8EC2-FEE0-4746-A4F6-45082734FEEC@sun.com","threadId":"12936","inReplyTo":"7vhcebcyty.fsf@gitster.siamese.dyndns.org","subject":"Intricacies of submodules [was: Migrating svn to git with heavy use of externals]","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-10T03:43:34Z","receivedAt":"2008-04-10T03:43:34Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"Hi Junio!\n\nOn Apr 8, 2008, at 11:43 PM, Junio C Hamano wrote:\n\n> \"Avery Pennarun\" <apenwarr@gmail.com> writes:\n>\n>> On Wed, Apr 9, 2008 at 12:39 AM, Roman Shaposhnik <rvs@sun.com>  \n>> wrote:\n>>> Agreed. But I guess I'd be less confused if \"git submodule\" didn't  \n>>> muck\n>>> with .git/config at all. Or are there any other consumers of the\n>>> information\n>>> that it puts there (except itself)?\n>>\n>> That I don't know.  If there aren't any others, then I agree, I'm not\n>> sure what the whole .git/config messing is about.\n>\n> Its actually the other way around.\n\nGot it. But if you don't mind, I still would like to ask you a few  \nquestions\nto clarify some things.\n\n> In-tree .gitmodules is used to give hints to prime what is placed in\n> .git/config, which after initialized should serve as the authoritative\n> information on managed submodules as far as your repository is  \n> concerned.\n> \"git submodule init\" may be a handy way to do this \"priming\", but  \n> you do\n> not necessarily have to use it but instead manually adjust .git/config\n> yourself; this is so that you can configure remote url that is  \n> different\n> from what .gitmodules suggests to suite your local needs.\n\nOk. Now I understand that .git/config is supposed to be the  \nauthoritative\nsource of information on submodules. Yet we also have .gitmodules\nto take care of. This leads to information duplication and makes me\nbelieve that .git/config should be as much as sync with .gitmodules as\npossible. Yet, even with the latest version of Git we don't have\n\"git submodule add\" updating .git/config. So here comes the first  \nquestion:\n     * Do you consider this behavior to be a bug or do you a have a  \nreasonable\n         explanation for it?\nContinuing in the same line of though as far as information  \nduplication goes,\nhere's my second question:\n     * Whenever .gitmodules and .git/config disagree on the URL for a  \nparticular\n        submodule do you expect .git/config to always take precedence?\nAnd finally, since from your explanation it appears that the only  \nreason for\n.gitmodules existence is to \"prime\" the .git/config it seems that what  \nwe're\ntrying to achieve is a way for Git settings that are usually part  \nof .git/config\nto be resident within the repository itself. That would give these  \nsetting\na benefit of percolating through clone/fetch/push operations, yet be\noverridden by individual .git/config settings. And so I have my final  \nquestion:\n      * Has an idea of having a regular file (subject to having  \nhistory, etc.)\n        called something like .gitconfig at the top level of Git's  \nrepository ever\n        been considered (implemented?). That way you a repository  \nmaintainer\n        would be able to force a particular set of settings on all of  \nits clones\n        yet clones will be able to override then in .git/config if  \nneeded.\n\n> Although putting everything in a single repository could work, that  \n> does\n> not have to be the only way to work with submodules.  In fact, the  \n> basic\n> submodule design is trying very hard not to force you to grab  \n> objects that\n> are needed for all submodules when you are cloning the superproject,  \n> as\n> not cloning nor checking out any submodule is the default.\n\n\nIndeed. This is a very beneficial setup for large projects. In fact,  \nwhat I'm working\non right now is a prototype of a build infrastructure that would be  \nsmart enough\nto import \"cached\" binaries of the build of a particular submodule if  \nthe submodule\nitself hasn't been checked out yet. SHA1 lets me do the versioning  \nproperly and\nonce developers do checkout sources of any submodule the build system\nwill stop importing \"cached\" binaries and start build it for real. All  \nwithout developers\nactually doing anything special.\n\nThanks,\nRoman.\n"},{"id":"74015","messageId":"7v63uqz265.fsf@gitster.siamese.dyndns.org","threadId":"12936","inReplyTo":"6CFA8EC2-FEE0-4746-A4F6-45082734FEEC@sun.com","subject":"Re: Intricacies of submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-10T05:53:06Z","receivedAt":"2008-04-10T05:53:06Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Roman Shaposhnik <rvs@sun.com> writes:\n\n> ... Yet, even with the latest version of Git we don't have\n> \"git submodule add\" updating .git/config. So here comes the first  \n> question:\n>      * Do you consider this behavior to be a bug or do you a have a  \n> reasonable\n>          explanation for it?\n\nI would say that the part is simply underdeveloped.  I do not think many\npeople from the core circle of the git community are heavy users of\nsubmodules.\n\nThe \"submodule add\" command was done primarily by and for people who\nwanted to initially register a commit as a submodule from a subdirectory\nrepository, back when there was not much actual propagation support (that\nis, \"what should happen when somebody cloned such a toplevel project with\nsubmodule?\") designed yet.  The command simply records the global hint to\nthe .gitmodules file, so that people who get such a commit that records a\nsubmodule in its tree can also learn where to turn to when they do want to\nget to the submodule by looking at in-tree .gitmodules file.\n\nHowever, the way others will obtain a copy of the submodule repository\nwill be quite different from the way you access it (you already have it,\nso you do not need to clone it from elsewhere to initialize it).  It may\nnot make much sense to record the URL that you tell others to use in your\nown .git/config in the repository of the originator of such a superproject\nvs submodule combination.  So in that sense, I am not sure if not mucking\nwith .git/config is even a bad thing.\n\nThe side that registers data in .git/config using what is in .gitmodules\nas a hint, which is what \"git submodule init\" is about, is not very much\ndeveloped either (yet).  It does not have enough user interaction to allow\nthe user to tailor the URL for his own needs, for example.  There is no\nduplication per-se; .gitmodules may give you git:// URL but you might need\nto rewrite it to corresponding http:// URL because of your networking\nsituation.  And that is the reason why .git/config should be the\nauthoritative copy.  There may be bugs in the implementation -- I dunno --\nbut at least that is the intent.  IOW, at runtime, .gitmodules should not\nbe consulted for purposes other than to update .git/config entries.\n\nThe original discussion that led to the current implementation dates back\nin May-June timeframe of 2007.  I would not be surprised if not all of the\ngood ideas were incorporated in the current implementation.  For example,\none thing that we may want to do is to record what contents we've seen in\nthe .gitmodules file in order to prime each entry in .git/config, so that\nwe can give users a chance to adjust what is in .git/config when we notice\nthe entry in .gitmodules has changed.\n\nFor example, consider that .gitmodules said the submodule should be taken\nfrom repository URL git://A.xz/project.git when you cloned.  You may have\nused the given URL as-is to prime your .git/config, or you may have chosen\nto use http://A.xz/project.git/ for networking reasons.\n\nAfter working with the project for a while (i.e. you pull and perhaps push\nback or send patches upstream), .gitmodules file changes and it now says\nthe repository resides at host B.xz because the project relocated.  You\nwould want the next \"git submodule update\" to notice that your .git/config\nrecords a URL you derived from git://A.xz/project.git/, and that you have\nnot seen this new URL git://B.xz/project.git/, and give you a chance to\nmake adjustments if needed.\n\nAfter that happens, if you seeked to an old version (perhaps you wanted to\nwork on an old bug), .gitmodules file that is checked out of that old\nversion may say the \"upstream\" is at A.xz, but the entry in .git/config\nmay already be based on B.xz.  But because you have already seen this old\nURL in .gitmodules, you may not want to get asked about adjusting the\nentry in .git/config merely because you checked out an old version.  What\nthis means is that it is not enough to just record \"What the current URL\nyou chose to use is\" in .git/config (which is obvious), and it is also not\nenough to record \"what URL .gitmodules had when you made that choice\", but\nyou would also need to record \"What URLs you have _seen_ when making that\nchoice\".\n\nThe above is one thing I remember seeing in the original discussion but I\ndo not think implemented in the current code.  I strongly suspect there\nare other good design bits left unimplemented in the discussion.  There\ndefinitely are other things people who are interested in submodules may\nwant to improve.\n\n>       * Has an idea of having a regular file (subject to having  \n> history, etc.)\n>         called something like .gitconfig at the top level of Git's  \n> repository ever\n>         been considered (implemented?). That way you a repository  \n> maintainer\n>         would be able to force a particular set of settings on all of  \n> its clones\n>         yet clones will be able to override then in .git/config if  \n> needed.\n\nConsidered, yes, implemented, no.  Not because nobody bothered to, but\nbecause it is unclear if it is a good thing to do in general to begin\nwith.  What's recorded in .git/config is pretty much personal (e.g. \"who\nyou are known as to this project?\", \"what's the SMTP host, user and\npassword when sending out patches from here?\", \"do you want to use color\nin diff?\"), dependent on local needs (e.g. \"what protocol a particular\nremote repository should be reached via\"), or what the repository (as\nopposed to \"project\") is about (e.g. \"is this a bare, shared distribution\npoint, or is this a developer repository with a work tree?\").\n\nProject policies do not belong to .git/config and should be propagated\nin-tree.  For example, \"indenting with more than 8 spaces is a whitespace\nerror for *.c files\" is described in .gitattributes and given to all\ncloners.\n\nThere may be some behaviour that is currently controlled by what is\nrecorded in .git/config but should be enforced project-wide.  If there are\nsuch things, we may want to have a mechanism that reads from in-tree data,\njust like the attributes code does.\n"},{"id":"74082","messageId":"46dff0320804100907w5240b79bga3acad9fb8ecb353@mail.gmail.com","threadId":"12936","inReplyTo":"6CFA8EC2-FEE0-4746-A4F6-45082734FEEC@sun.com","subject":"Re: Intricacies of submodules [was: Migrating svn to git with heavy use of externals]","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-10T16:07:51Z","receivedAt":"2008-04-10T16:07:51Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Thu, Apr 10, 2008 at 11:43 AM, Roman Shaposhnik <rvs@sun.com> wrote:\n> Hi Junio!\n>\n\n>      * Has an idea of having a regular file (subject to having history,\n> etc.)\n>        called something like .gitconfig at the top level of Git's repository\n> ever\n>        been considered (implemented?). That way you a repository maintainer\n>        would be able to force a particular set of settings on all of its\n> clones\n>        yet clones will be able to override then in .git/config if needed.\n>\n\nI like this idea, it's another common/special requirement just like\n.gitignore vs. $GIT_DIR/info/exclude.\n\n\n\n-- \nPing Yin\n"},{"id":"74090","messageId":"1207855653.13123.257.camel@work.sfbay.sun.com","threadId":"12936","inReplyTo":"46dff0320804100907w5240b79bga3acad9fb8ecb353@mail.gmail.com","subject":"Re: Intricacies of submodules [was: Migrating svn to git with heavy use of externals]","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-10T19:27:33Z","receivedAt":"2008-04-10T19:27:33Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Fri, 2008-04-11 at 00:07 +0800, Ping Yin wrote:\n> On Thu, Apr 10, 2008 at 11:43 AM, Roman Shaposhnik <rvs@sun.com> wrote:\n> > Hi Junio!\n> >\n> \n> >      * Has an idea of having a regular file (subject to having history,\n> > etc.)\n> >        called something like .gitconfig at the top level of Git's repository\n> > ever\n> >        been considered (implemented?). That way you a repository maintainer\n> >        would be able to force a particular set of settings on all of its\n> > clones\n> >        yet clones will be able to override then in .git/config if needed.\n> >\n> \n> I like this idea, it's another common/special requirement just like\n> .gitignore vs. $GIT_DIR/info/exclude.\n\nWell, I guess if enough of us like it there's a chance it can be\nimplemented, right? ;-)\n\nTo some extent it seems that you've solved this particular issue for \nsubmodules with your PATCH/RFC 3/7. Now, in a general case, if \ngit-config(1) can be patched to take into account one extra place\nfor retrieving options from (.gitconfig) it seems that\nretiring .gitmodules completely would be just one benefit of many.\nOther benefits would include propagating setting like most of the\ncore.* and quite a few other things I see listed in git-config(1)\nman page.\n\nIt seems that the only downside here would be a need for a bit\nof special handling when a setting needs to be recorded. Otherwise\nit looks like a pretty clean and general idea.\n\nThanks,\nRoman.\n"},{"id":"74096","messageId":"1207859579.13123.306.camel@work.sfbay.sun.com","threadId":"12936","inReplyTo":"7v63uqz265.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-10T20:32:58Z","receivedAt":"2008-04-10T20:32:58Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Wed, 2008-04-09 at 22:53 -0700, Junio C Hamano wrote:\n> Roman Shaposhnik <rvs@sun.com> writes:\n> \n> > ... Yet, even with the latest version of Git we don't have\n> > \"git submodule add\" updating .git/config. So here comes the first  \n> > question:\n> >      * Do you consider this behavior to be a bug or do you a have a  \n> > reasonable\n> >          explanation for it?\n> \n> I would say that the part is simply underdeveloped.\n\nFair enough. Which is good news for me, since I can not really imagine\nhow the repositories I need to establish can survive without solid\nSubmodule support. I'm very interested in getting this functionality\nright with git-submodule. And I can be either your guinea pig or\na frenetic hamster. After all, you don't mind complete newcomers\nto the development process sending you code, do you? ;-)\n\n> I do not think many people from the core circle of the git community are \n> heavy users of submodules.\n\nI wonder why. Most of the software projects that I have to deal with\nseem to be a pretty hefty collections of loosely coupled things. In that\nother thread I had on git-repack there was a typical example of what\nhappens if you put something like that into a single repo (700\nsubdirectories at the top level -- that's what :-(). Does it all mean\nthat the core Git community mostly works on projects like Linux kernel\nand not things like OpenOffice or Mozilla, etc?\n\n> However, the way others will obtain a copy of the submodule repository\n> will be quite different from the way you access it (you already have it,\n> so you do not need to clone it from elsewhere to initialize it).  It may\n> not make much sense to record the URL that you tell others to use in your\n> own .git/config in the repository of the originator of such a superproject\n> vs submodule combination.  So in that sense, I am not sure if not mucking\n> with .git/config is even a bad thing.\n\nIt is all about consistency as far as I see it. One huge advantage of\nGit is that it is a DSCM. It makes things totally symmetric. The\nparagraph that I quoted above hints at a possibility of treating the\ninitial repo somewhat differently from its copies. That would break\na nice symmetry. And it would do that unnecessarily.\n\n> After working with the project for a while (i.e. you pull and perhaps push\n> back or send patches upstream), .gitmodules file changes and it now says\n> the repository resides at host B.xz because the project relocated.  You\n> would want the next \"git submodule update\" to notice that your .git/config\n> records a URL you derived from git://A.xz/project.git/, and that you have\n> not seen this new URL git://B.xz/project.git/, and give you a chance to\n> make adjustments if needed.\n\nI guess something like that could be implemented via Git hooks, right?\n\n> After that happens, if you seeked to an old version (perhaps you wanted to\n> work on an old bug), .gitmodules file that is checked out of that old\n> version may say the \"upstream\" is at A.xz, but the entry in .git/config\n> may already be based on B.xz.  But because you have already seen this old\n> URL in .gitmodules, you may not want to get asked about adjusting the\n> entry in .git/config merely because you checked out an old version.  What\n> this means is that it is not enough to just record \"What the current URL\n> you chose to use is\" in .git/config (which is obvious), and it is also not\n> enough to record \"what URL .gitmodules had when you made that choice\", but\n> you would also need to record \"What URLs you have _seen_ when making that\n> choice\".\n\nGood point.\n\n> >       * Has an idea of having a regular file (subject to having  \n> > history, etc.)\n> >         called something like .gitconfig at the top level of Git's  \n> > repository ever\n> >         been considered (implemented?). That way you a repository  \n> > maintainer\n> >         would be able to force a particular set of settings on all of  \n> > its clones\n> >         yet clones will be able to override then in .git/config if  \n> > needed.\n> \n> Considered, yes, implemented, no.  Not because nobody bothered to, but\n> because it is unclear if it is a good thing to do in general to begin\n> with.  What's recorded in .git/config is pretty much personal (e.g. \"who\n> you are known as to this project?\", \"what's the SMTP host, user and\n> password when sending out patches from here?\", \"do you want to use color\n> in diff?\"), dependent on local needs (e.g. \"what protocol a particular\n> remote repository should be reached via\"), or what the repository (as\n> opposed to \"project\") is about (e.g. \"is this a bare, shared distribution\n> point, or is this a developer repository with a work tree?\").\n\nSome of it is personal, yes. But sometimes those personal preferences\nneed to be enforced on a project level (of course, giving everybody\na way to override the setting if they really want to). For a big\nsoftware organization with a mix of senior and junior engineers I need\na way to set up *my* workspace in such a way that everybody who\nclones/pulls from it get not only the source code, but also \"Git best\npractices\". That would simplify things a great deal for me, because\nI can always say: \"just pull my latest .gitconfig, make sure you\ndon't have any extra stuff in your .git/confing and everything \nin Git will work for you\". That would also simplify things for junior\nguys as well -- they can be sure that whatever needs to be done\nwith Git's setup I can do for them and all they have to do is pull.\n\nPerhaps, this model is different from what the majority of Git\ndevelopers uses (especially working on OpenSource projects) but I'm\npretty sure it is quite widespread within corporate firewalls.\n\n> Project policies do not belong to .git/config and should be propagated\n> in-tree.  For example, \"indenting with more than 8 spaces is a whitespace\n> error for *.c files\" is described in .gitattributes and given to all\n> cloners.\n\nAgreed. But what I have in mind is more of: in this project everybody\nshall use super-duper-GUI-merge by default. I can't really propagate \nmerge.tool in any way, can I? But I really need to. Since the health\nof my project really depends on junior engineers not being confused\nwhen doing the merges.\n\nWhat is also not clear to me is the difference between what is\nconsidered to be an attribute (part of .gitattributes) and\nwhat it considered to be a setting (part of .git/config). It seems\nthat the line gets quite blurry (at least for me it does) and thus\nI'd totally appreciate any help understanding this difference\nbetter.\n\n> There may be some behaviour that is currently controlled by what is\n> recorded in .git/config but should be enforced project-wide.  If there are\n> such things, we may want to have a mechanism that reads from in-tree data,\n> just like the attributes code does.\n\nThat's exactly what I have in mind. \n\nThanks,\nRoman.\n"},{"id":"74111","messageId":"7vd4oxufwf.fsf@gitster.siamese.dyndns.org","threadId":"12936","inReplyTo":"1207859579.13123.306.camel@work.sfbay.sun.com","subject":"Re: Intricacies of submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-11T05:20:00Z","receivedAt":"2008-04-11T05:20:00Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Roman Shaposhnik <rvs@sun.com> writes:\n\n> ... I'm very interested in getting this functionality\n> right with git-submodule. And I can be either your guinea pig or\n> a frenetic hamster. After all, you don't mind complete newcomers\n> to the development process sending you code, do you? ;-)\n\nEverybody starts out as a total stranger.  Linus has never worked with me\nwhen I started, and many people who are the core members of git community\nhave never worked with me before either.\n\n>> However, the way others will obtain a copy of the submodule repository\n>> will be quite different from the way you access it (you already have it,\n>> so you do not need to clone it from elsewhere to initialize it).  It may\n>> not make much sense to record the URL that you tell others to use in your\n>> own .git/config in the repository of the originator of such a superproject\n>> vs submodule combination.  So in that sense, I am not sure if not mucking\n>> with .git/config is even a bad thing.\n>\n> It is all about consistency as far as I see it. One huge advantage of\n> Git is that it is a DSCM. It makes things totally symmetric. The\n> paragraph that I quoted above hints at a possibility of treating the\n> initial repo somewhat differently from its copies. That would break\n> a nice symmetry. And it would do that unnecessarily.\n\nI do not think being distributed is about such symmetry.\n\nBeing distributed is more about each repository being able to serve its\nown purpose, being able to get configured suitably and individually,\nwithout disturbing others, and allowing a workflow around it that\n_potentially_ treats everybody as equals.\n\nNot having the kind of symmetry you talk about is not anything new about\nsubmodules, nor is it necessarily a bad thing.  You create a history here,\nyou push it into there.  Somebody else clones your history from there and\nstarts hacking.\n\nThe way that somebody's clone interacts with the intermediary and the way\nyour original repository interacts with the intermediary _are_ different,\nand they ought to stay different if that intermediary is _your_ owned\npublishing repository.  You can push into it, but that somebody else\nshould not be able to.  There should be no symmetry about that repository.\n\nThat somebody else may have his own publishing repository where he pushes\nthe result of his work into and you fetch from.  Taken together, each of\nyou and that somebody else having his own repository to allow others to\nfetch from, makes you two the equals in the global picture.\n\nYou would only need the symmetry of your kind if there is a single\nintermediary that is _the central location_, a shared repository where\neverybody meets.  Only in that case, you _may_, after priming the process\nby initially creating the superproject - submodule combination in the\noriginating repository and pushing it to the shared repository, want to\nclone it back to a new work tree you will use as your usual working place\n(and nuke the originating one, which is not needed anymore as the process\nhas been primed now).  At that point, your usual working place and\neverybody else's working place would look symmetrical, as everybody\nincluding you cloned from a single shared location.\n\nI am not saying that is necessarily a bad thing to wish for.  I am only\nsaying that the kind of symmetry you talk about does not have much to do\nwith being distributed.  If anything, that symmetry is more closely tied\nto using a centralized work flow, not distributed.\n\n>> After working with the project for a while (i.e. you pull and perhaps push\n>> back or send patches upstream), .gitmodules file changes and it now says\n>> the repository resides at host B.xz because the project relocated.  You\n>> would want the next \"git submodule update\" to notice that your .git/config\n>> records a URL you derived from git://A.xz/project.git/, and that you have\n>> not seen this new URL git://B.xz/project.git/, and give you a chance to\n>> make adjustments if needed.\n>\n> I guess something like that could be implemented via Git hooks, right?\n\nI do not see a reason to bring in hooks here.  To answer \"Yes\" to your\n\"right?\" question, \"git submodule update\" ought to call out to a hook in\nsuch a situation, which it doesn't right now.  So the answer for the\ncurrent implementation would be \"no\".  To make it \"Yes\", the command needs\nto be modified to call out a hook, but should it be implemented as a hook,\nwhen it is already so clearly specified what needs to happen?\n\n>> Considered, yes, implemented, no.  Not because nobody bothered to, but\n>> because it is unclear if it is a good thing to do in general to begin\n>> with.  What's recorded in .git/config is pretty much personal (e.g. \"who\n>> you are known as to this project?\", \"what's the SMTP host, user and\n>> password when sending out patches from here?\", \"do you want to use color\n>> in diff?\"), dependent on local needs (e.g. \"what protocol a particular\n>> remote repository should be reached via\"), or what the repository (as\n>> opposed to \"project\") is about (e.g. \"is this a bare, shared distribution\n>> point, or is this a developer repository with a work tree?\").\n>\n> Some of it is personal, yes. But sometimes those personal preferences\n> need to be enforced on a project level (of course, giving everybody\n> a way to override the setting if they really want to). For a big\n> software organization with a mix of senior and junior engineers I need\n> a way to set up *my* workspace in such a way that everybody who\n> clones/pulls from it get not only the source code, but also \"Git best\n> practices\". That would simplify things a great deal for me, because\n> I can always say: \"just pull my latest .gitconfig, make sure you\n> don't have any extra stuff in your .git/confing and everything \n> in Git will work for you\".\n\nI think the way you stated the above speaks for itself.  The issue you are\nsolving is mostly human (social), and solution is majorly instruction with\nslight help from mechanism.  The instruction \"Use this latest thing, do\nnot have anything in .git/config\" can be substituted with \"Use this latest\nupdate-git-config.sh which mucks with your .git/config to conform to our\nproject standard\", without losing simplicity and with much enhanced\nrobustness, as you can now enforce that the users do not have anything\nthat would interfere with and countermand your policy you would want to\nimplement.\n"},{"id":"74126","messageId":"46dff0320804110904w531035f4w79c1889bc90c09ee@mail.gmail.com","threadId":"12936","inReplyTo":"7vd4oxufwf.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-11T16:04:13Z","receivedAt":"2008-04-11T16:04:13Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Fri, Apr 11, 2008 at 1:20 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>  > Some of it is personal, yes. But sometimes those personal preferences\n>  > need to be enforced on a project level (of course, giving everybody\n>  > a way to override the setting if they really want to). For a big\n>  > software organization with a mix of senior and junior engineers I need\n>  > a way to set up *my* workspace in such a way that everybody who\n>  > clones/pulls from it get not only the source code, but also \"Git best\n>  > practices\". That would simplify things a great deal for me, because\n>  > I can always say: \"just pull my latest .gitconfig, make sure you\n>  > don't have any extra stuff in your .git/confing and everything\n>  > in Git will work for you\".\n>\n>  I think the way you stated the above speaks for itself.  The issue you are\n>  solving is mostly human (social), and solution is majorly instruction with\n>  slight help from mechanism.  The instruction \"Use this latest thing, do\n>  not have anything in .git/config\" can be substituted with \"Use this latest\n>  update-git-config.sh which mucks with your .git/config to conform to our\n>  project standard\", without losing simplicity and with much enhanced\n>  robustness, as you can now enforce that the users do not have anything\n>  that would interfere with and countermand your policy you would want to\n>  implement.\n>\nBut, how  to handle the case that  there are more than one policies\nfor different projects?\n\n\n\n-- \nPing Yin\n"},{"id":"74151","messageId":"7vmyo0owep.fsf@gitster.siamese.dyndns.org","threadId":"12936","inReplyTo":"46dff0320804110904w531035f4w79c1889bc90c09ee@mail.gmail.com","subject":"Re: Intricacies of submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-11T22:32:14Z","receivedAt":"2008-04-11T22:32:14Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Ping Yin\" <pkufranky@gmail.com> writes:\n\n> On Fri, Apr 11, 2008 at 1:20 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>>  > Some of it is personal, yes. But sometimes those personal preferences\n>>  > need to be enforced on a project level (of course, giving everybody\n>>  > a way to override the setting if they really want to). For a big\n>>  > software organization with a mix of senior and junior engineers I need\n>>  > a way to set up *my* workspace in such a way that everybody who\n>>  > clones/pulls from it get not only the source code, but also \"Git best\n>>  > practices\". That would simplify things a great deal for me, because\n>>  > I can always say: \"just pull my latest .gitconfig, make sure you\n>>  > don't have any extra stuff in your .git/confing and everything\n>>  > in Git will work for you\".\n>>\n>>  I think the way you stated the above speaks for itself.  The issue you are\n>>  solving is mostly human (social), and solution is majorly instruction with\n>>  slight help from mechanism.  The instruction \"Use this latest thing, do\n>>  not have anything in .git/config\" can be substituted with \"Use this latest\n>>  update-git-config.sh which mucks with your .git/config to conform to our\n>>  project standard\", without losing simplicity and with much enhanced\n>>  robustness, as you can now enforce that the users do not have anything\n>>  that would interfere with and countermand your policy you would want to\n>>  implement.\n>>\n> But, how  to handle the case that  there are more than one policies\n> for different projects?\n\n\"How to\"?  You would handle the case just like either of us suggested\nabove.\n\nAre you talking about a single project with more than one policies A, B,\nC, ... that conflict with each other?  Or are you talking about more than\none projects, each of which has a single project-wide policy?\n\nI do not think the former makes sense and won't be helped with in-tree\nfile that overrides .git/config Roman discussed either.\n\nThe latter would be helped equally well whether that in-tree polic file is\ncalled .gitconfig or update-git-config.sh.\n"},{"id":"74173","messageId":"1207970038.10408.8.camel@ginkgo","threadId":"12936","inReplyTo":"7vmyo0owep.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-12T03:13:58Z","receivedAt":"2008-04-12T03:13:58Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Fri, 2008-04-11 at 15:32 -0700, Junio C Hamano wrote:\n> > But, how  to handle the case that  there are more than one policies\n> > for different projects?\n> \n> \"How to\"?  You would handle the case just like either of us suggested\n> above.\n> \n> Are you talking about a single project with more than one policies A, B,\n> C, ... that conflict with each other?  Or are you talking about more than\n> one projects, each of which has a single project-wide policy?\n> \n> I do not think the former makes sense and won't be helped with in-tree\n> file that overrides .git/config Roman discussed either.\n> \n> The latter would be helped equally well whether that in-tree polic file is\n> called .gitconfig or update-git-config.sh.\n\nI believe Fedor addressed the social aspects of this issue quite well,\nso I'm just going to focus on a technical aspect here: there is a \ndifference between .gitconfig and update-git-config.sh approaches \nthat I would like you to acknowledge. With update-git-config.sh you\nare allowing for a repository to be in a state that is inconsistent\nwith the policies that need to be enforced, without novice users even\nrealizing that. Contrast this with .gitconfig where policies get\nenforced right from the minute your clone operation finishes and there's\nmuch less opportunity for the user to shoot himself in the foot. In \nfact \"shooting in the foot\" (senselessly overriding default policies\nvia .git/config) becomes an *explicit* action on user's part. He is\nthe one to blame.\n\nThanks,\nRoman.\n"},{"id":"74174","messageId":"46dff0320804112020v5488b5bbg903deba840ef468a@mail.gmail.com","threadId":"12936","inReplyTo":"7vmyo0owep.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-12T03:20:06Z","receivedAt":"2008-04-12T03:20:06Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Sat, Apr 12, 2008 at 6:32 AM, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> \"Ping Yin\" <pkufranky@gmail.com> writes:\n>\n>  > On Fri, Apr 11, 2008 at 1:20 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>  >>  > Some of it is personal, yes. But sometimes those personal preferences\n>  >>  > need to be enforced on a project level (of course, giving everybody\n>  >>  > a way to override the setting if they really want to). For a big\n>  >>  > software organization with a mix of senior and junior engineers I need\n>  >>  > a way to set up *my* workspace in such a way that everybody who\n>  >>  > clones/pulls from it get not only the source code, but also \"Git best\n>  >>  > practices\". That would simplify things a great deal for me, because\n>  >>  > I can always say: \"just pull my latest .gitconfig, make sure you\n>  >>  > don't have any extra stuff in your .git/confing and everything\n>  >>  > in Git will work for you\".\n>  >>\n>  >>  I think the way you stated the above speaks for itself.  The issue you are\n>  >>  solving is mostly human (social), and solution is majorly instruction with\n>  >>  slight help from mechanism.  The instruction \"Use this latest thing, do\n>  >>  not have anything in .git/config\" can be substituted with \"Use this latest\n>  >>  update-git-config.sh which mucks with your .git/config to conform to our\n>  >>  project standard\", without losing simplicity and with much enhanced\n>  >>  robustness, as you can now enforce that the users do not have anything\n>  >>  that would interfere with and countermand your policy you would want to\n>  >>  implement.\n>  >>\n>  > But, how  to handle the case that  there are more than one policies\n>  > for different projects?\n>\n>  \"How to\"?  You would handle the case just like either of us suggested\n>  above.\n>\n>  Are you talking about a single project with more than one policies A, B,\n>  C, ... that conflict with each other?  Or are you talking about more than\n>  one projects, each of which has a single project-wide policy?\n>\n>  I do not think the former makes sense and won't be helped with in-tree\n>  file that overrides .git/config Roman discussed either.\n>\n>  The latter would be helped equally well whether that in-tree polic file is\n>  called .gitconfig or update-git-config.sh.\n\nI meant more than one projects, each of  which has a different\nproject-wide policy.  I originally thought update-git-config.sh can't\nhelp, but i'm wrong since it can update $GIT_DIR/config instead of\n$HOME/.gitconfig.\n\nHowever, i think .gitconfig is better since it's more consistent with\nother analogies.\n\n\n\n-- \nPing Yin\n"},{"id":"74175","messageId":"46dff0320804112102t52a60072rc97c772a1e74f597@mail.gmail.com","threadId":"12936","inReplyTo":"7v63uqz265.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-12T04:02:25Z","receivedAt":"2008-04-12T04:02:25Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Thu, Apr 10, 2008 at 1:53 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>  The original discussion that led to the current implementation dates back\n>  in May-June timeframe of 2007.  I would not be surprised if not all of the\n>  good ideas were incorporated in the current implementation.  For example,\n>  one thing that we may want to do is to record what contents we've seen in\n>  the .gitmodules file in order to prime each entry in .git/config, so that\n>  we can give users a chance to adjust what is in .git/config when we notice\n>  the entry in .gitmodules has changed.\n>\n>  For example, consider that .gitmodules said the submodule should be taken\n>  from repository URL git://A.xz/project.git when you cloned.  You may have\n>  used the given URL as-is to prime your .git/config, or you may have chosen\n>  to use http://A.xz/project.git/ for networking reasons.\n>\n>  After working with the project for a while (i.e. you pull and perhaps push\n>  back or send patches upstream), .gitmodules file changes and it now says\n>  the repository resides at host B.xz because the project relocated.  You\n>  would want the next \"git submodule update\" to notice that your .git/config\n>  records a URL you derived from git://A.xz/project.git/, and that you have\n>  not seen this new URL git://B.xz/project.git/, and give you a chance to\n>  make adjustments if needed.\n\nI think this should be done if \"git submodule update\" fails. The\nreason it fails may be different, such as newest commits not pushed\nout and the subproject relocated etc. So it can only given some hints\nwith \"maybe\".\n\nHowever, how to detect the url has changed in .gitmodules? Compare the\nlatest two version of .gitmodules?\n\nAnd if only the protocol or domain changes of the submodule between\n$GIT_CONFIG/config and .gitmodules, i think the\n\"url.<usethis>.insteadof = <otherurl>\" form introduced in v1.5.5 is\nmore helpful.\n\n>\n>  After that happens, if you seeked to an old version (perhaps you wanted to\n>  work on an old bug), .gitmodules file that is checked out of that old\n>  version may say the \"upstream\" is at A.xz, but the entry in .git/config\n>  may already be based on B.xz.  But because you have already seen this old\n>  URL in .gitmodules, you may not want to get asked about adjusting the\n>  entry in .git/config merely because you checked out an old version.  What\n>  this means is that it is not enough to just record \"What the current URL\n>  you chose to use is\" in .git/config (which is obvious), and it is also not\n>  enough to record \"what URL .gitmodules had when you made that choice\", but\n>  you would also need to record \"What URLs you have _seen_ when making that\n>  choice\".\n>\n\nWhen bug happens, i only care the commit in the index of submodule and\nwheter i can check out the old submodule commit. However, does it\nreally matter that what the url of the submodule is?\n\n\n-- \nPing Yin\n"},{"id":"74179","messageId":"7vlk3jlkrr.fsf@gitster.siamese.dyndns.org","threadId":"12936","inReplyTo":"1207970038.10408.8.camel@ginkgo","subject":"Re: Intricacies of submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-12T05:11:52Z","receivedAt":"2008-04-12T05:11:52Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Roman Shaposhnik <rvs@sun.com> writes:\n\n> ... Contrast this with .gitconfig where policies get\n> enforced right from the minute your clone operation finishes and there's\n> much less opportunity for the user to shoot himself in the foot.\n\nWhy?  Even if you expect .git/config in a new repository would be vanilla\n(which you can't really, as crazy sysadmin can have /etc/gitconfig or\ntemplate to override what you do), $HOME/.gitconfig would be in effect the\nminute you clone.\n\nAs you cannot reasonably expect that your project is the _only_ project\nyour cloners would use, you cannot dictate what $HOME/.gitconfig has.\n\nA policy issue needs to be addressed at the human level anyway, so I do\nnot really see major difference either way.  You need to trust your users\nto follow the guideline at some point, and all you can do is to make it\neasy for them to do so, and (optionally) verify that they are actually\nfollowing the guideline.  We need to suggest an easy-to-use and robust\nmechanism to allow you to do so as the BCP.\n\nConvenience and robustness need to be considered at the same time.  In\nthat area, I would say a custom \"sane environment setup script\" would be\nthe more flexible, as it rolls the customization and verification into one\nstep.\n\nTrust goes mutual and your users need to be able to trust you, too.  If\nthe config mechanism blindly starts reading from in-tree .gitconfig, you\ncan do nasty things with aliases for example.  So the \"sane environment\nsetup script\" would also be a good idea in that sense, too --- the users,\nperhaps only the most suspicious and untrusting kind, have a way to verify\nit does not mean any harm before running it.\n\nDon't get me wrong.  I am not saying that everybody should start rolling\ntheir own \"sane environment setup script\" and ship their project with it.\nI am only suggesting it as a possible way to do your \"policy enforcement\"\nwithout having to introduce in-tree .gitconfig, which I unfortunately see\nno fundamental upsides but definite downsides (security included).\n"},{"id":"74180","messageId":"7vej9blk4j.fsf@gitster.siamese.dyndns.org","threadId":"12936","inReplyTo":"46dff0320804112102t52a60072rc97c772a1e74f597@mail.gmail.com","subject":"Re: Intricacies of submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-12T05:25:48Z","receivedAt":"2008-04-12T05:25:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Ping Yin\" <pkufranky@gmail.com> writes:\n\n>>  After working with the project for a while (i.e. you pull and perhaps push\n>>  back or send patches upstream), .gitmodules file changes and it now says\n>>  the repository resides at host B.xz because the project relocated.  You\n>>  would want the next \"git submodule update\" to notice that your .git/config\n>>  records a URL you derived from git://A.xz/project.git/, and that you have\n>>  not seen this new URL git://B.xz/project.git/, and give you a chance to\n>>  make adjustments if needed.\n>\n> I think this should be done if \"git submodule update\" fails. The\n> reason it fails may be different, such as newest commits not pushed\n> out and the subproject relocated etc. So it can only given some hints\n> with \"maybe\".\n>\n> However, how to detect the url has changed in .gitmodules? Compare the\n> latest two version of .gitmodules?\n\nThat's why I suggested (and Roman seems to have got it, so I do not think\nwhat I wrote was too confusing to be understood) you should record the set\nof _all_ URLs you have _seen_ in .git/config.  If the URL in .gitmodules\nchecked out is included in that set, you do not do anything.  Otherwise\nyou ask.\n\nI think \"git submodule update\" is a good place to do that check, but I'd\nprefer it be done _before_ it actually goes to the network to start\naccessing potentially stale URL.  The old URL may not be defunct but the\nproject decided not to advertise it to be used for some non-technical\nreason (e.g. the site owner asked them not to point at it and instead use\nsome other mirrors).\n\n> When bug happens, i only care the commit in the index of submodule and\n> wheter i can check out the old submodule commit. However, does it\n> really matter that what the url of the submodule is?\n\nNo.  The discussion was what should _not_ happen when you run \"git\nsubmodule update\" from that state.  Usually in a steadily advancing\nhistory, you _want_ \"git submodule update\" to notice that the suggested\nremote URL has changed in .gitmodules and give the user a chance to adjust\nthe URL _before_ it hits the network, but you obviously do not want it to\nhappen only because you happened to be at a seeked back commit when you\ninitiated \"git submodule update\".  In other words, you are agreeing with\nme without really reading what I wrote ;-)  It does not matter, and\nrecording the URLs you have _seen_ (not \"the last one you saw\", or \"the\none you initialized .git/config with\") is a way to make sure that the\nfixed \"git submodule update\" agrees with us on that point.\n"},{"id":"74183","messageId":"46dff0320804112326v7beccd1dp3dc9fdb5c81bb25d@mail.gmail.com","threadId":"12936","inReplyTo":"7vej9blk4j.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-12T06:26:31Z","receivedAt":"2008-04-12T06:26:31Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Sat, Apr 12, 2008 at 1:25 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> \"Ping Yin\" <pkufranky@gmail.com> writes:\n>\n>\n> >>  After working with the project for a while (i.e. you pull and perhaps push\n>  >>  back or send patches upstream), .gitmodules file changes and it now says\n>  >>  the repository resides at host B.xz because the project relocated.  You\n>  >>  would want the next \"git submodule update\" to notice that your .git/config\n>  >>  records a URL you derived from git://A.xz/project.git/, and that you have\n>  >>  not seen this new URL git://B.xz/project.git/, and give you a chance to\n>  >>  make adjustments if needed.\n>  >\n>  > I think this should be done if \"git submodule update\" fails. The\n>  > reason it fails may be different, such as newest commits not pushed\n>  > out and the subproject relocated etc. So it can only given some hints\n>  > with \"maybe\".\n>  >\n>  > However, how to detect the url has changed in .gitmodules? Compare the\n>  > latest two version of .gitmodules?\n>\n>  That's why I suggested (and Roman seems to have got it, so I do not think\n>  what I wrote was too confusing to be understood) you should record the set\n>  of _all_ URLs you have _seen_ in .git/config.  If the URL in .gitmodules\n>  checked out is included in that set, you do not do anything.  Otherwise\n>  you ask.\n\nI don't think it deserves such a change (say recoding history urls to\n$GIT_DIR/config) to just ask just the user whether to change url in\n$GIT_CONFIG/config when the url in .gitmodules changes to a new one.\n\nActually, i think this is an ugly solution :-)\n\n>\n>  I think \"git submodule update\" is a good place to do that check, but I'd\n>  prefer it be done _before_ it actually goes to the network to start\n>  accessing potentially stale URL.  The old URL may not be defunct but the\n>  project decided not to advertise it to be used for some non-technical\n>  reason (e.g. the site owner asked them not to point at it and instead use\n>  some other mirrors).\n\nIf only the protocol (such as http://->git://) is different between\nurls in $GIT_DIR/config and .gitmodules, i think use\n\"url.base.insteadOf = newbase\" is simpler.\n\nIf the urls are totally different, when url in .gitmodules changes,\nthere is little chance that the url in $GIT_DIR/config will also\nchange.\n\n\n-- \nPing Yin\n"},{"id":"74375","messageId":"1208202740.25663.69.camel@work.sfbay.sun.com","threadId":"12936","inReplyTo":"7vlk3jlkrr.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-14T19:52:20Z","receivedAt":"2008-04-14T19:52:20Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Fri, 2008-04-11 at 22:11 -0700, Junio C Hamano wrote:\n> Roman Shaposhnik <rvs@sun.com> writes:\n> \n> > ... Contrast this with .gitconfig where policies get\n> > enforced right from the minute your clone operation finishes and there's\n> > much less opportunity for the user to shoot himself in the foot.\n> \n> Why?  Even if you expect .git/config in a new repository would be vanilla\n> (which you can't really, as crazy sysadmin can have /etc/gitconfig or\n> template to override what you do), $HOME/.gitconfig would be in effect the\n> minute you clone.\n\nI think I understand where you are going with this. Although, truth be\ntold, to me ~/.gitconfig is much less of a concern. Why? Well, because\nby definition if the user is smart enough to edit ~/.gitconfig I'm\nnot concerned about him. As I pointed out my main concern is about\njunior developers for whom the only way to screw things up would\nbe to have a global /etc/gitconfig, which is still quite rare.\n\n> As you cannot reasonably expect that your project is the _only_ project\n> your cloners would use, you cannot dictate what $HOME/.gitconfig has.\n\nSee, that's exactly why I would love to have in-tree .gitconfig ;-) \n~/.gitconfig is not flexible enough to have settings for multiple\nprojects and .git/config needs to be managed by scritps. In-tree\n.gitconfig just works.\n\n> A policy issue needs to be addressed at the human level anyway, so I do\n> not really see major difference either way.  You need to trust your users\n> to follow the guideline at some point, and all you can do is to make it\n> easy for them to do so, and (optionally) verify that they are actually\n> following the guideline.  We need to suggest an easy-to-use and robust\n> mechanism to allow you to do so as the BCP.\n\nAnd that's where it becomes a matter of preference. I can now see your \npoint very clearly and I tend to slightly disagree with it. But! This\nis definitely not a technical issue anymore (in-tree .gitconfig and\nin-tree shell script for managing .git/config are technically \nequivalent). So, I think I don't have any more arguments to add to the\ndiscussion. I do have one question left (see bellow) and one comment \nto make: my experience has been that it is much easier to trust\nvolunteer and open source developers compared to corporate ones. \nI do get it 100% that Git is \"for the kernel folks; by the kernel folks\"\nand I actually think that it is a healthy environment for an SCM to\ngrow in. But!\n\nAll I'm saying is that if the needs of the corporate folks can be taken\ninto account without doing Git's architecture any harm I think they\nshould be.\n\n> Don't get me wrong.  I am not saying that everybody should start rolling\n> their own \"sane environment setup script\" and ship their project with it.\n> I am only suggesting it as a possible way to do your \"policy enforcement\"\n> without having to introduce in-tree .gitconfig, which I unfortunately see\n> no fundamental upsides but definite downsides (security included).\n\nAnd here comes my question: could you, please, elaborate on *technical*\ndrawbacks of in-tree .gitconfig (such as security that you've\nmentioned).\n\nThanks,\nRoman.\n"},{"id":"74376","messageId":"1208202986.25663.73.camel@work.sfbay.sun.com","threadId":"12936","inReplyTo":"7vd4oxufwf.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-14T19:56:26Z","receivedAt":"2008-04-14T19:56:26Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Thu, 2008-04-10 at 22:20 -0700, Junio C Hamano wrote:\n> Roman Shaposhnik <rvs@sun.com> writes:\n> \n> > ... I'm very interested in getting this functionality\n> > right with git-submodule. And I can be either your guinea pig or\n> > a frenetic hamster. After all, you don't mind complete newcomers\n> > to the development process sending you code, do you? ;-)\n> \n> Everybody starts out as a total stranger.  Linus has never worked with me\n> when I started, and many people who are the core members of git community\n> have never worked with me before either.\n\nCool! I do have a couple of questions on the development etiquette,\nbut I think I'll ask them off-line unless somebody can point me\nto an FAQ on how Git's development is setup. The section \n\"Community and Development\" doesn't seem to answer much\nof my questions.\n\nThanks,\nRoman.\n"},{"id":"74398","messageId":"7vd4or7wdt.fsf@gitster.siamese.dyndns.org","threadId":"12936","inReplyTo":"1208202740.25663.69.camel@work.sfbay.sun.com","subject":"Re: Intricacies of submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-15T01:13:50Z","receivedAt":"2008-04-15T01:13:50Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Roman Shaposhnik <rvs@sun.com> writes:\n\n>> Don't get me wrong.  I am not saying that everybody should start rolling\n>> their own \"sane environment setup script\" and ship their project with it.\n>> I am only suggesting it as a possible way to do your \"policy enforcement\"\n>> without having to introduce in-tree .gitconfig, which I unfortunately see\n>> no fundamental upsides but definite downsides (security included).\n>\n> And here comes my question: could you, please, elaborate on *technical*\n> drawbacks of in-tree .gitconfig (such as security that you've\n> mentioned).\n\nJust to name a few, as I do not see a point in spending time elaborating\nin detail when there is an alternative without such security downsides.\n\nOne of your examples was about a forced use of custom merge tool.\nConsider in-tree .gitconfig that is always read for everybody that\ndescribes such a tool.  A malicious script named there is a security risk\nfor people who clone such a project.  A smudge filter is even worse, as it\nkicks in the minute you try to check out the project.\n\nThese executable (not just merge tool or attribute filters) are designed\nto be named by .git/config exactly because .git/config is designed to be\npersonal (i.e. \"that _particular repository only_\") and you can afford to\nbe environment and platform specific there.  If you start describing them\nin in-tree .gitconfig, they must be cross platform and (worse yet)\nyou have to make sure they are installed everywhere.\n\nThere are states recorded by git-submodule whether the particular\nrepository has seen and is interested in which submodule (i.e. \"submodule\ninit\" has been run).\n\nI'm too lazy to make a laundary list of what you can have in .git/config\nwith the current system (see Documentation/config.txt), but that part of\nthe system is built around the design that the configuration is specific\nto the repository (and sharing what the user records in ~/.gitconfig\nacross repositories is in line with it).\n\nUnless you are willing to sift through all of them, mark which ones can be\noverriden by in-tree .gitconfig and which ones cannot, and implement an\neasy to use (by both the developers and the users) mechanism to enforce\nthe distinction, just changing the git_config() function to read from one\nnew place (i.e. in-tree .gitconfig) would not be a sufficient solution for\nwhat you seem to want to do.\n"},{"id":"74405","messageId":"46dff0320804141913r5585ed9cr25fa3bdae3c2d5a@mail.gmail.com","threadId":"12936","inReplyTo":"7vd4or7wdt.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-15T02:13:51Z","receivedAt":"2008-04-15T02:13:51Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Tue, Apr 15, 2008 at 9:13 AM, Junio C Hamano <gitster@pobox.com> wrote:\n> Roman Shaposhnik <rvs@sun.com> writes:\n>\n>\n>  I'm too lazy to make a laundary list of what you can have in .git/config\n>  with the current system (see Documentation/config.txt), but that part of\n>  the system is built around the design that the configuration is specific\n>  to the repository (and sharing what the user records in ~/.gitconfig\n>  across repositories is in line with it).\n>\nI can give some which can be enforced in a project level\n\nmerge.summary\nstatus.submodulesummary\ncore.whitespace\n\n\n-- \nPing Yin\n"},{"id":"74500","messageId":"1208317795.26863.91.camel@goose.sun.com","threadId":"12936","inReplyTo":"7vd4or7wdt.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Roman V. Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-16T03:49:55Z","receivedAt":"2008-04-16T03:49:55Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Mon, 2008-04-14 at 18:13 -0700, Junio C Hamano wrote:\n> Roman Shaposhnik <rvs@sun.com> writes:\n> \n> >> Don't get me wrong.  I am not saying that everybody should start rolling\n> >> their own \"sane environment setup script\" and ship their project with it.\n> >> I am only suggesting it as a possible way to do your \"policy enforcement\"\n> >> without having to introduce in-tree .gitconfig, which I unfortunately see\n> >> no fundamental upsides but definite downsides (security included).\n> >\n> > And here comes my question: could you, please, elaborate on *technical*\n> > drawbacks of in-tree .gitconfig (such as security that you've\n> > mentioned).\n> \n> Just to name a few, as I do not see a point in spending time elaborating\n> in detail when there is an alternative without such security downsides.\n> \n> One of your examples was about a forced use of custom merge tool.\n> Consider in-tree .gitconfig that is always read for everybody that\n> describes such a tool.  A malicious script named there is a security risk\n> for people who clone such a project.  A smudge filter is even worse, as it\n> kicks in the minute you try to check out the project.\n\nI'm sorry, but I don't buy this argument. If you have a malicious user\ngaining access to the repository all bets are off. To single out \nin-tree .gitconfig as the only place which could be hacked seems to\nbe a bit shortsighted and unfair. Any \"executable\" portion of your\nproject that rarely gets eyeballed (such as Makefile infrastrucutre)\ncould be used. In fact, under your scenario in-tree .gitconfig is\nlikely to be the least of your worries. \n\nAnd here's one more thing: in-tree .gitconfig and in-tree \nupdate-my-git-settings.sh are absolutely identical as far\nas their security ramifications are concerned. If you really paranoid\nyou have to eyeball either of them.\n\n> These executable (not just merge tool or attribute filters) are designed\n> to be named by .git/config exactly because .git/config is designed to be\n> personal (i.e. \"that _particular repository only_\") and you can afford to\n> be environment and platform specific there.  If you start describing them\n> in in-tree .gitconfig, they must be cross platform and (worse yet)\n> you have to make sure they are installed everywhere.\n\nI don't buy this argument either. First of all, there's a $PATH. On top\nof that even automounters learned how to deal with heterogeneous\nhosts efficiently ($HOST, $CPU, etc.) so I really don't think Git should\nhave any problems. But the most obvious counterargument to your\nstatement would be that quite a few developers (myself included) don't\nhave a luxury of developing on a single architecture. Thus in-tree\n.gitconfig doesn't change anything -- *my* single Git repository has to\nprovide settings that work on: [sparc|intel]-[Solaris|Linux]. I do\nhave .git/config that accomplished that. I see no reason for in-tree\n.gitconfig to not be able to.\n\n> I'm too lazy to make a laundary list of what you can have in .git/config\n> with the current system (see Documentation/config.txt), but that part of\n> the system is built around the design that the configuration is specific\n> to the repository (and sharing what the user records in ~/.gitconfig\n> across repositories is in line with it).\n> \n> Unless you are willing to sift through all of them, mark which ones can be\n> overriden by in-tree .gitconfig and which ones cannot, and implement an\n> easy to use (by both the developers and the users) mechanism to enforce\n> the distinction, just changing the git_config() function to read from one\n> new place (i.e. in-tree .gitconfig) would not be a sufficient solution for\n> what you seem to want to do.\n\nWhy? I'm really confused here. Unless I'm given a clear example of at\nleast one setting that somehow becomes dangerous when stored inside\nin-tree .gitconfig, I really do consider such an enforcement to be\nas meaningful as enforcing that Git MUST manage source code and nothing\nelse. You seemed to mention the trust issue. Well, why don't you trust\nthe user to place whatever he wants in in-tree .gitconfig? And yes,\nwe are talking about trustworthy users here and repositories that\nhaven't been compromised.\n\nThanks,\nRoman.\n\nP.S. Junio, I really don't want to waste your time especially since\nI get a feeling that our discussion has clearly moved into a domain\nof taste and preferences. But I had to refute your security and \nheterogeneity arguments simply because they don't seem to have any\nsubstance to them.\n"},{"id":"74655","messageId":"87lk3c4ali.fsf@jeremyms.com","threadId":"12936","inReplyTo":"1208317795.26863.91.camel@goose.sun.com","subject":"Re: Intricacies of submodules","fromName":"Jeremy Maitin-Shepard","fromEmail":"jbms@cmu.edu","sentAt":"2008-04-17T18:09:29Z","receivedAt":"2008-04-17T18:09:29Z","isPatch":false,"sender":{"key":"jbms@cmu.edu","avatar":null},"body":"\"Roman V. Shaposhnik\" <rvs@sun.com> writes:\n\n[snip]\n\n> I'm sorry, but I don't buy this argument. If you have a malicious user\n> gaining access to the repository all bets are off. To single out \n> in-tree .gitconfig as the only place which could be hacked seems to\n> be a bit shortsighted and unfair. Any \"executable\" portion of your\n> project that rarely gets eyeballed (such as Makefile infrastrucutre)\n> could be used. In fact, under your scenario in-tree .gitconfig is\n> likely to be the least of your worries. \n\n> And here's one more thing: in-tree .gitconfig and in-tree \n> update-my-git-settings.sh are absolutely identical as far\n> as their security ramifications are concerned. If you really paranoid\n> you have to eyeball either of them.\n\nThere is a huge difference: if you allow in-tree .gitconfig by default,\nthen git clone <some-repository> becomes an unsafe operation.  I can't\neven inspect some arbitrary repository to _see_ if I like the code and\nthink it is safe very easily, since I'd normally do that by cloning the\nrepository.\n\nObviously actually executing untrusted code is unsafe regardless of\nwhether you type \"git clone\" or \"make\" to do it, but not everyone\nintends to type \"make\" after checking out an unknown repository, and the\nuser is explicitly invoking make with the knowledge that it is running\nwhatever code is in the repository.  Similarly, if the user explicitly\ncalls some shell script in order to set things up, he is conscious that\nhe is performing a potentially unsafe operation.\n\nAs a silly analogy, it is currently perfectly safe to clone a repository\nthat has a text document containing instructions about committing\nsuicide, because there is the assumption that the instructions are not\nautomatically executed simply because they are on the user's hard drive.\n\n[snip]\n\n> Why? I'm really confused here. Unless I'm given a clear example of at\n> least one setting that somehow becomes dangerous when stored inside\n> in-tree .gitconfig, I really do consider such an enforcement to be\n> as meaningful as enforcing that Git MUST manage source code and nothing\n> else. You seemed to mention the trust issue. Well, why don't you trust\n> the user to place whatever he wants in in-tree .gitconfig? And yes,\n> we are talking about trustworthy users here and repositories that\n> haven't been compromised.\n\nObviously any configuration option that specifies a shell command to run\nis unsafe to specify in an in-tree .gitconfig.  As Junio noted,\nsmudge/clean commands are especially unsafe because they will be\nexecuted even if the user only uses the clone command.\n\nYou actually seem to be the one assuming that a Git repository must\nstore source code (in particular source code that is then blindly\nexecuted by anyone that clones the repository), as that is the only case\nin which an in-tree .gitconfig can introduce no additional security\nrisk, since your security is then already completely dependent on\ntrusting the contents of the repository.\n\n-- \nJeremy Maitin-Shepard\n"},{"id":"74659","messageId":"alpine.LFD.1.00.0804171158360.2879@woody.linux-foundation.org","threadId":"12936","inReplyTo":"87lk3c4ali.fsf@jeremyms.com","subject":"Re: Intricacies of submodules","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-04-17T19:06:06Z","receivedAt":"2008-04-17T19:06:06Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 17 Apr 2008, Jeremy Maitin-Shepard wrote:\n> \n> There is a huge difference: if you allow in-tree .gitconfig by default,\n> then git clone <some-repository> becomes an unsafe operation.\n\nI have to agree.\n\nThe git config file is rather powerful, with things like aliases etc that \ncan be used to run external programs (and with the external diff \nfunctionality that includes it for very basic and default operations), and \nways of subtly (and not so subtly) rewriting repository information etc \netc.\n\nSo if we do end up doing a \"tracked config file\", I'd personally very much \nprefer it be limited in some way. For example, we obviously track the \n.gitignore and .gitattributes files, but they are much more limited in \ntheir effects. Maybe we could have a \"limited config file\" that allows for \n*some* config options to be set?\n\n\t\tLinus\n"},{"id":"74662","messageId":"1208461808.26863.129.camel@goose.sun.com","threadId":"12936","inReplyTo":"87lk3c4ali.fsf@jeremyms.com","subject":"Re: Intricacies of submodules","fromName":"Roman V. Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-17T19:50:08Z","receivedAt":"2008-04-17T19:50:08Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Thu, 2008-04-17 at 14:09 -0400, Jeremy Maitin-Shepard wrote:\n> > And here's one more thing: in-tree .gitconfig and in-tree \n> > update-my-git-settings.sh are absolutely identical as far\n> > as their security ramifications are concerned. If you really paranoid\n> > you have to eyeball either of them.\n> \n> There is a huge difference: if you allow in-tree .gitconfig by default,\n> then git clone <some-repository> becomes an unsafe operation.  I can't\n> even inspect some arbitrary repository to _see_ if I like the code and\n> think it is safe very easily, since I'd normally do that by cloning the\n> repository.\n\nAre you saying that a *remote* in-tree .gitconfig would be capable of\naffecting *local* system before the end of the clone operation? I find\nit very hard to believe. And if it is so, I'd love to be educated on the\nsubject matter. What I (and to some extent Ping Yin) have been proposing\nis a completely different semantics -- the in-tree .gitconfig would only\nbe able to affect your *local* operations. Doing clone of the *remote*\nrepository is a safe operation under such assumptions. Once you cloned\nit, you might need to eyeball the content of .gitconfig if you're really\nparanoid.\n\n> As a silly analogy, it is currently perfectly safe to clone a repository\n> that has a text document containing instructions about committing\n> suicide, because there is the assumption that the instructions are not\n> automatically executed simply because they are on the user's hard drive.\n\nSame holds true for the semantics being proposed. The intsructions are\n*not* executed until you actually try to do something with your \nrepository. There's a window of opportunity in which inspecting the\ncontent of .gitconfig is absolutely possible.\n\n> > Why? I'm really confused here. Unless I'm given a clear example of at\n> > least one setting that somehow becomes dangerous when stored inside\n> > in-tree .gitconfig, I really do consider such an enforcement to be\n> > as meaningful as enforcing that Git MUST manage source code and nothing\n> > else. You seemed to mention the trust issue. Well, why don't you trust\n> > the user to place whatever he wants in in-tree .gitconfig? And yes,\n> > we are talking about trustworthy users here and repositories that\n> > haven't been compromised.\n> \n> Obviously any configuration option that specifies a shell command to run\n> is unsafe to specify in an in-tree .gitconfig.  As Junio noted,\n> smudge/clean commands are especially unsafe because they will be\n> executed even if the user only uses the clone command.\n\nI'm sorry but I guess that went over my head. Is this the example of\nsomething that can affect local repository (and host!) during the\nclone operation? I tried to find documentation on the subject but\ngoogling for \"git smudge\" returns very few useful hits and the bits\nof documentation in gitattributes(5) don't really explain much.\n\n> You actually seem to be the one assuming that a Git repository must\n> store source code (in particular source code that is then blindly\n> executed by anyone that clones the repository), as that is the only case\n> in which an in-tree .gitconfig can introduce no additional security\n> risk, since your security is then already completely dependent on\n> trusting the contents of the repository.\n\nThere are two things at play: first of all, I usually *do* trust the\ncontent of the repository. Call it matter of personal preference,\nbut *for me* if you start with distrust -- there's very little you\ncan do with that repository to begin with. To me it is a bit of \nred herring. On the other hand I understand where you're coming from\nand I definitely appreciate the need for a clone operation to be\nsafe. So far, the only example of an unsafe setting that I have been\ngiven is smudge/clean filters. May be this is enough to shoot the\nvery idea of an in-tree .gitconfig down, but I still don't really\nunderstand the *complete* semantics of these things. Can somebody\nexplain, please?\n\nI hope this is not too much to ask.\n\nThanks,\nRoman.\n"},{"id":"74663","messageId":"7vmynsnt7x.fsf@gitster.siamese.dyndns.org","threadId":"12936","inReplyTo":"alpine.LFD.1.00.0804171158360.2879@woody.linux-foundation.org","subject":"Re: Intricacies of submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-17T20:04:34Z","receivedAt":"2008-04-17T20:04:34Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@linux-foundation.org> writes:\n\n> So if we do end up doing a \"tracked config file\", I'd personally very much \n> prefer it be limited in some way. For example, we obviously track the \n> .gitignore and .gitattributes files, but they are much more limited in \n> their effects. Maybe we could have a \"limited config file\" that allows for \n> *some* config options to be set?\n\nYes, that's all what I have been trying to say ;-)\n"},{"id":"74665","messageId":"46a038f90804171306t22491685p87d7445d44f00879@mail.gmail.com","threadId":"12936","inReplyTo":"1208461808.26863.129.camel@goose.sun.com","subject":"Re: Intricacies of submodules","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2008-04-17T20:06:27Z","receivedAt":"2008-04-17T20:06:27Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Thu, Apr 17, 2008 at 4:50 PM, Roman V. Shaposhnik <rvs@sun.com> wrote:\n>  There are two things at play: first of all, I usually *do* trust the\n>  content of the repository. Call it matter of personal preference,\n\nI think most people here split the trust into \"before or after\ncompilation\". I must trust that I can clone/checkout code safely so I\ncan review it.\n\nRunning the code contained in the repo we are discussing a completely\ndifferent matter.  Even before compilation, Makefiles and configure\nscripts may shoot you in the foot or in the face. But you had at least\na chance to review it.\n\ncheers,\n\n\nn\n-- \n martin.langhoff@gmail.com\n martin@laptop.org -- School Server Architect\n - ask interesting questions\n - don't get distracted with shiny stuff - working code first\n - http://wiki.laptop.org/go/User:Martinlanghoff\n"},{"id":"74669","messageId":"7vabjsnrda.fsf@gitster.siamese.dyndns.org","threadId":"12936","inReplyTo":"46a038f90804171306t22491685p87d7445d44f00879@mail.gmail.com","subject":"Re: Intricacies of submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-17T20:44:33Z","receivedAt":"2008-04-17T20:44:33Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Martin Langhoff\" <martin.langhoff@gmail.com> writes:\n\n> On Thu, Apr 17, 2008 at 4:50 PM, Roman V. Shaposhnik <rvs@sun.com> wrote:\n>>  There are two things at play: first of all, I usually *do* trust the\n>>  content of the repository. Call it matter of personal preference,\n>\n> I think most people here split the trust into \"before or after\n> compilation\". I must trust that I can clone/checkout code safely so I\n> can review it.\n\nI think that summarizes the arguments so far pretty well.\n\nHaving said that, the current \"clone\" implementation may happen to ignore\nin-tree anything, e.g. ident filter defined in .gitattributes may not be\napplied due to chicken-and-egg issue of not having .gitattributes\ninitially in the work tree when you check everything out to an empty work\ntree for the first time.\n\nBut I consider that is not by design, but is a limitation of the current\nimplementation that can be improved.\n"},{"id":"74671","messageId":"bd6139dc0804171400x515b3c8br3cb1501cca8a6d0a@mail.gmail.com","threadId":"12936","inReplyTo":"7vabjsnrda.fsf@gitster.siamese.dyndns.org","subject":"Re: Intricacies of submodules","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-04-17T21:00:54Z","receivedAt":"2008-04-17T21:00:54Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Thu, Apr 17, 2008 at 10:44 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>  Having said that, the current \"clone\" implementation may happen to ignore\n>  in-tree anything,\n\n<snip>\n\n>  But I consider that is not by design, but is a limitation of the current\n>  implementation that can be improved.\n\nI think it -should- be by design that it ignores everything unless we\nare certain that it is safe to do so. So as long as an in-tree doesn't\nprovide any hooks to execute things (which of course includes changing\nthe environment) it should be fine, but if it is, it should be ignored\ntill after clone has finished.\n\nBecause of that an in-tree '.gitconfig' would have no security risks\nas long as it is not 'used' until after the clone. This would be easy\nto make sure of by not syncing it with the real '.gitconfig' until\nafter cloning. (That is assuming there will be some sort of syncing to\nthe real 'gitconfig' from the in-tree '.gitconfig', if a fall-back\ntype of mechanism is chosen that might be more difficult)\n\nCheers,\n\nSverre Rabbelier\n"},{"id":"74681","messageId":"46a038f90804171425q1cc4cff4m6b783252040a3b26@mail.gmail.com","threadId":"12936","inReplyTo":"bd6139dc0804171400x515b3c8br3cb1501cca8a6d0a@mail.gmail.com","subject":"Re: Intricacies of submodules","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2008-04-17T21:25:01Z","receivedAt":"2008-04-17T21:25:01Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Thu, Apr 17, 2008 at 6:00 PM, Sverre Rabbelier <alturin@gmail.com> wrote:\n>  provide any hooks to execute things (which of course includes changing\n>  the environment) it should be fine, but if it is, it should be ignored\n>  till after clone has finished.\n\nIt should not be allowed at all. After the clone is the review, and\nthat has to be safe too.\n\n>  Because of that an in-tree '.gitconfig' would have no security risks\n>  as long as it is not 'used' until after the clone.\n\nThis is not true. A pre-commit hook or pre-checkout hook could be destructive.\n\ncheers,\n\n\nm\n-- \n martin.langhoff@gmail.com\n martin@laptop.org -- School Server Architect\n - ask interesting questions\n - don't get distracted with shiny stuff - working code first\n - http://wiki.laptop.org/go/User:Martinlanghoff\n"},{"id":"74682","messageId":"bd6139dc0804171427i6bf2813at719c8dec13bc225c@mail.gmail.com","threadId":"12936","inReplyTo":"46a038f90804171425q1cc4cff4m6b783252040a3b26@mail.gmail.com","subject":"Re: Intricacies of submodules","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-04-17T21:27:32Z","receivedAt":"2008-04-17T21:27:32Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Thu, Apr 17, 2008 at 11:25 PM, Martin Langhoff\n<martin.langhoff@gmail.com> wrote:\n> On Thu, Apr 17, 2008 at 6:00 PM, Sverre Rabbelier <alturin@gmail.com> wrote:\n>  >  provide any hooks to execute things (which of course includes changing\n>  >  the environment) it should be fine, but if it is, it should be ignored\n>  >  till after clone has finished.\n>\n>  It should not be allowed at all. After the clone is the review, and\n>  that has to be safe too.\n\nI reckon review is done without using git, I don't see how it would\npose a security risk.\n\n>  >  Because of that an in-tree '.gitconfig' would have no security risks\n>  >  as long as it is not 'used' until after the clone.\n>\n>  This is not true. A pre-commit hook or pre-checkout hook could be destructive.\n\nBut, those won't be executed till after the review, so everything\nwould be good still, wouldn't it?\n\nCheers,\n\nSverre Rabbelier\n"},{"id":"74684","messageId":"46a038f90804171431q51215be8od41792293712ca9@mail.gmail.com","threadId":"12936","inReplyTo":"bd6139dc0804171427i6bf2813at719c8dec13bc225c@mail.gmail.com","subject":"Re: Intricacies of submodules","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2008-04-17T21:31:53Z","receivedAt":"2008-04-17T21:31:53Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Thu, Apr 17, 2008 at 6:27 PM, Sverre Rabbelier <alturin@gmail.com> wrote:\n>  >  >  Because of that an in-tree '.gitconfig' would have no security risks\n>  >  >  as long as it is not 'used' until after the clone.\n>  >\n>  >  This is not true. A pre-commit hook or pre-checkout hook could be destructive.\n>\n>  But, those won't be executed till after the review, so everything\n>  would be good still, wouldn't it?\n\nNo. A local review can be quite \"active\", involving changing branches,\nmoving patches around, and fixing sh*t up. The hooks available offer\nplenty of danger if the repo can set them and make them active:\n\n$ ls .git/hooks/\napplypatch-msg  post-commit   post-update     pre-commit  update\ncommit-msg      post-receive  pre-applypatch  pre-rebase\n\ncheers,\n\n\n\n\nm\n-- \n martin.langhoff@gmail.com\n martin@laptop.org -- School Server Architect\n - ask interesting questions\n - don't get distracted with shiny stuff - working code first\n - http://wiki.laptop.org/go/User:Martinlanghoff\n"},{"id":"74687","messageId":"20080417222907.GP3133@dpotapov.dyndns.org","threadId":"12936","inReplyTo":"1208461808.26863.129.camel@goose.sun.com","subject":"Re: Intricacies of submodules","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-04-17T22:29:07Z","receivedAt":"2008-04-17T22:29:07Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Thu, Apr 17, 2008 at 12:50:08PM -0700, Roman V. Shaposhnik wrote:\n> Doing clone of the *remote*\n> repository is a safe operation under such assumptions. Once you cloned\n> it, you might need to eyeball the content of .gitconfig if you're really\n> paranoid.\n\nNo, I don't think it is right. It is absolutely unacceptable to expect\nall users to be aware of some hidden file and to eyeball it just to be\nsure that the next 'git log' (or some other normal git operation) will\nnot remove all their files from the disk.\n\nPerhaps, I have not followed this discussion carefully, so I am not sure\nwhat .gitconfig is intended to solve. But if you think that _blindly_\nadding some options to other people configurations is a good idea, I\nhave to disagree with you. Some options may be useful in some cases or\nfor some platforms, but not for others. So, having a single .gitconfig\nis going to be a bad fit for some users. Thus a more flexible and more\nsecure solution is needed, and it already exists.\n\nYou can put git-configure at the top of your repository and tell people\nto run it after cloning. In this way, anyone can inspect this script and\nif they trust they will run it. This script can check on what system git\nis running on, and maybe ask questions, etc, so it can be really helpful\nfor wide category of users.\n\nDmitry\n"},{"id":"74688","messageId":"alpine.LFD.1.00.0804171530460.2879@woody.linux-foundation.org","threadId":"12936","inReplyTo":"1208461808.26863.129.camel@goose.sun.com","subject":"Re: Intricacies of submodules","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-04-17T22:32:21Z","receivedAt":"2008-04-17T22:32:21Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 17 Apr 2008, Roman V. Shaposhnik wrote:\n> \n> Are you saying that a *remote* in-tree .gitconfig would be capable of\n> affecting *local* system before the end of the clone operation?\n\nNo. But what do you do after a \"git clone\".\n\nDo you, for example, do something like \"git log -p\" to actually see the \ncommits?\n\nAnd what happens if that runs an external diff viewer script that just \nhappens to do a \"rm -rf $HOME\"?\n\nSee? The .git/config file allows you to set those kinds of things. They \nshould *not* be things you download!\n\n\t\tLinus\n"},{"id":"74707","messageId":"46dff0320804171841n7124525akeed0c680089770f1@mail.gmail.com","threadId":"12936","inReplyTo":"46a038f90804171431q51215be8od41792293712ca9@mail.gmail.com","subject":"Re: Intricacies of submodules","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-18T01:41:34Z","receivedAt":"2008-04-18T01:41:34Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Fri, Apr 18, 2008 at 5:31 AM, Martin Langhoff\n<martin.langhoff@gmail.com> wrote:\n> On Thu, Apr 17, 2008 at 6:27 PM, Sverre Rabbelier <alturin@gmail.com> wrote:\n>  >  >  >  Because of that an in-tree '.gitconfig' would have no security risks\n>  >  >  >  as long as it is not 'used' until after the clone.\n>  >  >\n>  >  >  This is not true. A pre-commit hook or pre-checkout hook could be destructive.\n>  >\n>  >  But, those won't be executed till after the review, so everything\n>  >  would be good still, wouldn't it?\n>\n>  No. A local review can be quite \"active\", involving changing branches,\n>  moving patches around, and fixing sh*t up. The hooks available offer\n>  plenty of danger if the repo can set them and make them active:\n>\n>  $ ls .git/hooks/\n>  applypatch-msg  post-commit   post-update     pre-commit  update\n>  commit-msg      post-receive  pre-applypatch  pre-rebase\n>\nAFAIK, hooks are not cloned automatically. So where do the destructive\nhooks come from?\n\n-- \nPing Yin\n"},{"id":"74708","messageId":"46dff0320804171848v1f48735dt9930dc023bdb0946@mail.gmail.com","threadId":"12936","inReplyTo":"alpine.LFD.1.00.0804171530460.2879@woody.linux-foundation.org","subject":"Re: Intricacies of submodules","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-18T01:48:24Z","receivedAt":"2008-04-18T01:48:24Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Fri, Apr 18, 2008 at 6:32 AM, Linus Torvalds\n<torvalds@linux-foundation.org> wrote:\n>\n>\n>  On Thu, 17 Apr 2008, Roman V. Shaposhnik wrote:\n>  >\n>  > Are you saying that a *remote* in-tree .gitconfig would be capable of\n>  > affecting *local* system before the end of the clone operation?\n>\n>  No. But what do you do after a \"git clone\".\n>\n>  Do you, for example, do something like \"git log -p\" to actually see the\n>  commits?\n>\n>  And what happens if that runs an external diff viewer script that just\n>  happens to do a \"rm -rf $HOME\"?\n>\nGood point. This is the best example (maybe the only one till now) i\nhave seen that demostrates the bad thing of in-tree .gitconfig. So i\nvote for the limited in-tree .gitconfig point of Linus.\n\n\n\n-- \nPing Yin\n"},{"id":"74738","messageId":"fua9lm$qts$1@ger.gmane.org","threadId":"12936","inReplyTo":"1208461808.26863.129.camel@goose.sun.com","subject":"Re: Intricacies of submodules","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-04-18T14:02:31Z","receivedAt":"2008-04-18T14:02:31Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Roman V. Shaposhnik wrote:\n> On Thu, 2008-04-17 at 14:09 -0400, Jeremy Maitin-Shepard wrote:\n\n>>> And here's one more thing: in-tree .gitconfig and in-tree \n>>> update-my-git-settings.sh are absolutely identical as far\n>>> as their security ramifications are concerned. If you really paranoid\n>>> you have to eyeball either of them.\n>> \n>> There is a huge difference: if you allow in-tree .gitconfig by default,\n>> then git clone <some-repository> becomes an unsafe operation.  I can't\n>> even inspect some arbitrary repository to _see_ if I like the code and\n>> think it is safe very easily, since I'd normally do that by cloning the\n>> repository.\n\n[...]\n>> Obviously any configuration option that specifies a shell command to run\n>> is unsafe to specify in an in-tree .gitconfig.  As Junio noted,\n>> smudge/clean commands are especially unsafe because they will be\n>> executed even if the user only uses the clone command.\n> \n> Are you saying that a *remote* in-tree .gitconfig would be capable of\n> affecting *local* system before the end of the clone operation?\n\nAt the end of clone operation you usually do a checkout. clean/smudge\ncommands could wipe out your disk at the end of clone.  And one usually\ndoes checkout to view contents of repository (alternative is to use\nplumbing git-cat-file, which does not use .gitattributes).\n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n"},{"id":"74725","messageId":"32541b130804181130s21e6ea5bm38ea0b35cd009d3e@mail.gmail.com","threadId":"12936","inReplyTo":"32541b130804181128j57d76edcsbbd5fb8d4c782ae7@mail.gmail.com","subject":"Re: Intricacies of submodules","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-18T18:30:03Z","receivedAt":"2008-04-18T18:30:03Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/17/08, Junio C Hamano <gitster@pobox.com> wrote:\n> Linus Torvalds <torvalds@linux-foundation.org> writes:\n>  > So if we do end up doing a \"tracked config file\", I'd personally very much\n>  > prefer it be limited in some way. For example, we obviously track the\n>  > .gitignore and .gitattributes files, but they are much more limited in\n>  > their effects. Maybe we could have a \"limited config file\" that allows for\n>  > *some* config options to be set?\n>\n> Yes, that's all what I have been trying to say ;-)\n\nHow about this: we know that *most* options are harmless, at least\nfrom a security point of view.  AFAIK it's really just the ones where\nyou specify shell commands that are unsafe.\n\nWhy not have a list of \"safe\" config options in git, and when reading\n.gitconfig, error out if any of the options in that file are unsafe.\n(Alternatively: silently ignore the unsafe ones, or warn and then\nignore the unsafe ones.)  A more advanced variation of the same would\nbe to have .git/config options that list specific exceptions to the\nsafe list, so if .gitconfig causes an error, you can *explicitly* git\nconfig set to let .gitconfig override them.\n\nAnother possibility would be to have an \"unsafe\" list instead of a\n\"safe\" list, but that sounds rather error-prone to me.\n\nHave fun,\n\nAvery\n"}]}