{"thread":{"id":"37371","subject":"Relative submodule URLs","startedAt":"2014-08-18T18:22:12Z","lastAt":"2014-08-29T15:09:20Z","messageCount":25,"participants":["Robert Dailey","Jonathan Nieder","Heiko Voigt","Junio C Hamano","Marc Branchaud"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"247856","messageId":"CAHd499CRNjp-UzXiTt=xgDJWGOEqew+AuPFmrF3-VsEGefXiuA@mail.gmail.com","threadId":"37371","inReplyTo":null,"subject":"Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-18T18:22:12Z","receivedAt":"2014-08-18T18:22:12Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"The documentation wasn't 100% clear on this, but I'm assuming by\n\"remote origin\", it says that the relative URL is relative to the\nactual remote *named* origin (and it is not using origin as just a\ngeneral terminology).\n\nIs there a way to specify (on a per-clone basis) which named remote\nwill be used to calculate the URL for submodules?\n\nVarious co-workers use the remote named \"central\" instead of\n\"upstream\" and \"fork\" instead of \"origin\" (because that just makes\nmore sense to them and it's perfectly valid).\n\nHowever if relative submodules require 'origin' to exist AND also\nrepresent the upstream repository (in triangle workflow), then this\nbreaks on several levels. There is also the common issue of upstream\nsubmodules needing to be forked as well.\n\nWould like to see if there is a way to workaround these issues with Git 2.1.0\n\nThanks in advance.\n"},{"id":"247858","messageId":"20140818205505.GA20185@google.com","threadId":"37371","inReplyTo":"CAHd499CRNjp-UzXiTt=xgDJWGOEqew+AuPFmrF3-VsEGefXiuA@mail.gmail.com","subject":"Re: Relative submodule URLs","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2014-08-18T20:55:05Z","receivedAt":"2014-08-18T20:55:05Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nRobert Dailey wrote:\n\n> The documentation wasn't 100% clear on this, but I'm assuming by\n> \"remote origin\", it says that the relative URL is relative to the\n> actual remote *named* origin (and it is not using origin as just a\n> general terminology).\n\nThanks for reporting. The remote used is the default remote that \"git\nfetch\" without further arguments would use:\n\n\tget_default_remote () {\n\t\tcurr_branch=$(git symbolic-ref -q HEAD)\n\t\tcurr_branch=\"${curr_branch#refs/heads/}\"\n\t\torigin=$(git config --get \"branch.$curr_branch.remote\")\n\t\techo ${origin:-origin}\n\t}\n\nThe documentation is wrong. git-fetch(1) doesn't provide a name for\nthis thing. Any ideas for wording?\n\n> Is there a way to specify (on a per-clone basis) which named remote\n> will be used to calculate the URL for submodules?\n\nCurrently there isn't, short of reconfiguring the remote used by\ndefault by \"git fetch\".\n\n> Various co-workers use the remote named \"central\" instead of\n> \"upstream\" and \"fork\" instead of \"origin\" (because that just makes\n> more sense to them and it's perfectly valid).\n>\n> However if relative submodules require 'origin' to exist AND also\n> represent the upstream repository (in triangle workflow), then this\n> breaks on several levels.\n\nCan you explain further? In a triangle workflow, \"git fetch\" will\npull from the 'origin' remote by default and will push to the remote\nnamed in the '[remote] pushdefault' setting (see \"remote.pushdefault\"\nin git-config(1)). So you can do\n\n\t[remote]\n\t\tpushDefault = whereishouldpush\n\nand then 'git fetch' and 'git fetch --recurse-submodules' will fetch\nfrom \"origin\" and 'git push' will push to the whereishouldpush remote.\n\nIt might make sense to introduce a new\n\n\t[remote]\n\t\tdefault = whereishouldfetch\n\nsetting to allow the name \"origin\" above to be replaced, too. Is that\nwhat you mean?\n\nMeanwhile it is hard to fork a project that uses relative submodule\nURLs without also forking the submodules (or, conversely, to fork some\nof the submodules of a project that uses absolute submodule URLs).\nThat's a real and serious problem but I'm not sure how it relates to\nthe names of remotes. My preferred fix involves teaching git to read\na refs/meta/git (or similarly named) ref when cloning a project with\nsubmodules and let settings from .gitmodules in that ref override\n.gitmodules in other branches. Is that what you were referring to?\n\nCurious,\nJonathan\n"},{"id":"247901","messageId":"20140819102421.GA5012@book.hvoigt.net","threadId":"37371","inReplyTo":"20140818205505.GA20185@google.com","subject":"Re: Relative submodule URLs","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-08-19T10:24:27Z","receivedAt":"2014-08-19T10:24:27Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Mon, Aug 18, 2014 at 01:55:05PM -0700, Jonathan Nieder wrote:\n> Robert Dailey wrote:\n> \n> > The documentation wasn't 100% clear on this, but I'm assuming by\n> > \"remote origin\", it says that the relative URL is relative to the\n> > actual remote *named* origin (and it is not using origin as just a\n> > general terminology).\n> \n> Thanks for reporting. The remote used is the default remote that \"git\n> fetch\" without further arguments would use:\n> \n> \tget_default_remote () {\n> \t\tcurr_branch=$(git symbolic-ref -q HEAD)\n> \t\tcurr_branch=\"${curr_branch#refs/heads/}\"\n> \t\torigin=$(git config --get \"branch.$curr_branch.remote\")\n> \t\techo ${origin:-origin}\n> \t}\n> \n> The documentation is wrong. git-fetch(1) doesn't provide a name for\n> this thing. Any ideas for wording?\n\nHow about 'upstream'? Like this[1]?\n\nLets step back a little is this really what we want in such situation? Is one\nremote really the answer here? I suppose you have relative urls in your\n.gitmodules file and two remotes in you superproject right?\n\nWhat you want is that the remote names in the superproject are reflected in the\nsubmodules when you initialise and update them?\n\nBecause at the moment what you get is always a remote 'origin' in the\nsubmodule. Even if that remote was called 'fork' in the superproject.\n\nMaybe in the relative URLs case we should teach the clone in submodule update\nto use all remotes with their names from the superproject?\n\nWould that solve your issue?\n\n> > Is there a way to specify (on a per-clone basis) which named remote\n> > will be used to calculate the URL for submodules?\n> \n> Currently there isn't, short of reconfiguring the remote used by\n> default by \"git fetch\".\n\nWell currently it is either the tracked remote by the currently checked out\nbranch or if the branch has no tracked remote configured: 'origin'.\n\nSo by configuring (or checking out a branch with) a different remote you can\nchoose from remote submodule are cloned. No?\n\n> > Various co-workers use the remote named \"central\" instead of\n> > \"upstream\" and \"fork\" instead of \"origin\" (because that just makes\n> > more sense to them and it's perfectly valid).\n> >\n> > However if relative submodules require 'origin' to exist AND also\n> > represent the upstream repository (in triangle workflow), then this\n> > breaks on several levels.\n> \n> Can you explain further? In a triangle workflow, \"git fetch\" will\n> pull from the 'origin' remote by default and will push to the remote\n> named in the '[remote] pushdefault' setting (see \"remote.pushdefault\"\n> in git-config(1)). So you can do\n> \n> \t[remote]\n> \t\tpushDefault = whereishouldpush\n> \n> and then 'git fetch' and 'git fetch --recurse-submodules' will fetch\n> from \"origin\" and 'git push' will push to the whereishouldpush remote.\n> \n> It might make sense to introduce a new\n> \n> \t[remote]\n> \t\tdefault = whereishouldfetch\n> \n> setting to allow the name \"origin\" above to be replaced, too. Is that\n> what you mean?\n\nI think the OP problem stems from him having a branch that does not have a\nremote configured. Since they do not have 'origin' as a remote and\n\n\tgit submodule update --init --recursive path/to/submodule\n\nfails. Right?\n\nCheers Heiko\n\n[1]\nFrom: Heiko Voigt \nSubject: [PATCH] submodule: use 'upstream' instead of 'origin' in\n documentation\n\nWhen talking about relative URL's it is ambiguous to use the term\n'origin', since that might denote the default remote name 'origin'. Lets\nuse 'upstream' to make it more clear that the upstream repository is\nmeant.\n\nSigned-off-by: Heiko Voigt \n---\n Documentation/git-submodule.txt | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt\nindex 8e6af65..c6f82e6 100644\n--- a/Documentation/git-submodule.txt\n+++ b/Documentation/git-submodule.txt\n@@ -80,15 +80,15 @@ to exist in the superproject. If is not given, the\n The is also used as the submodule's logical name in its\n configuration entries unless `--name` is used to specify a logical name.\n +\n- is the URL of the new submodule's origin repository.\n+ is the URL of the new submodule's upstream repository.\n This may be either an absolute URL, or (if it begins with ./\n-or ../), the location relative to the superproject's origin\n+or ../), the location relative to the superproject's upstream\n repository (Please note that to specify a repository 'foo.git'\n which is located right next to a superproject 'bar.git', you'll\n have to use '../foo.git' instead of './foo.git' - as one might expect\n when following the rules for relative URLs - because the evaluation\n of relative URLs in Git is identical to that of relative directories).\n-If the superproject doesn't have an origin configured\n+If the superproject doesn't have any remote configured\n the superproject is its own authoritative upstream and the current\n working directory is used instead.\n +\n-- \n2.1.0.rc0.52.gaa544bf\n"},{"id":"247911","messageId":"CAHd499DVf4N3Y6m5qoiy-WQGX4K54umefRzehZMsrxiWbaiZ=g@mail.gmail.com","threadId":"37371","inReplyTo":"20140818205505.GA20185@google.com","subject":"Re: Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-19T16:07:08Z","receivedAt":"2014-08-19T16:07:08Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Mon, Aug 18, 2014 at 3:55 PM, Jonathan Nieder wrote:\n> Thanks for reporting. The remote used is the default remote that \"git\n> fetch\" without further arguments would use:\n>\n> get_default_remote () {\n> curr_branch=$(git symbolic-ref -q HEAD)\n> curr_branch=\"${curr_branch#refs/heads/}\"\n> origin=$(git config --get \"branch.$curr_branch.remote\")\n> echo ${origin:-origin}\n> }\n>\n> The documentation is wrong. git-fetch(1) doesn't provide a name for\n> this thing. Any ideas for wording?\n\nI guess a good start would be to call it the \"tracked remote\" instead\nof \"remote origin\". The word \"tracked\" here makes it very obvious that\nif I have a remote tracking branch setup, it will use the remote\nportion of that configuration.\n\nThe real question is, how will `git submodule update` function if a\ntracking remote is not configured? Will it fail with some useful error\nmessage? I don't like the idea of it defaulting to \"self remote\" mode,\nwhere it will be relative to my repo directory. That seems like it\nwould fail is subtle ways in a worst-case scenario (if I did by\nhappenstance have a bare repository cloned up one directory level for\nother reasons).\n\n> Currently there isn't, short of reconfiguring the remote used by\n> default by \"git fetch\".\n\nI wish there was a way to specify the remote on a per-branch basis\nseparately from the tracking branch. I read a while back that someone\nproposed some changes to git to support decentralized tracking\n(concept of an upstream tracking branch and a separate one for origin,\ni think). If that were implemented, then relative submodules could\nutilize the 'upstream' remote by default for each branch, which would\nprovide more predictable default behavior. Right now most people on my\nteam would not be aware that if they tracked a branch on their fork,\nthey would subsequently need to fork the submodules to that same\nremote.\n\n>> Various co-workers use the remote named \"central\" instead of\n>> \"upstream\" and \"fork\" instead of \"origin\" (because that just makes\n>> more sense to them and it's perfectly valid).\n>>\n>> However if relative submodules require 'origin' to exist AND also\n>> represent the upstream repository (in triangle workflow), then this\n>> breaks on several levels.\n>\n> Can you explain further? In a triangle workflow, \"git fetch\" will\n> pull from the 'origin' remote by default and will push to the remote\n> named in the '[remote] pushdefault' setting (see \"remote.pushdefault\"\n> in git-config(1)). So you can do\n>\n> [remote]\n> pushDefault = whereishouldpush\n>\n> and then 'git fetch' and 'git fetch --recurse-submodules' will fetch\n> from \"origin\" and 'git push' will push to the whereishouldpush remote.\n\nI didn't know about this option, seems useful.\n\nA common workflow that we use on my team is to set the tracking branch\nto 'upstream' for convenient pulls with rebase. This means a feature\nbranch of mine can track its parent branch on 'upstream', so that when\nother pull requests get merged in on the remote repo branch, I can\njust do `git pull` and my feature branch rebases onto the latest of\nthat parent branch.\n\nCases like these would work with relative submodules because\n'upstream' is the tracked remote (and most of the time we don't want\nto fork submodules). However sometimes I like to track the same pushed\nbranch on origin (my fork), especially when it is up for pull request.\nIn these cases, my submodule update will fail because I didn't fork my\nsubmodules when I changed my tracking branch. Is this correct?\n\n\"breaks on several levels\" was basically my way of saying that various\nworkflow choices will break when you introduce submodules. One of the\nbeautiful things about Git is that it allows everyone to choose their\nown workflow. But submodules seem to prevent that to some degree. I\nthink addressing relative submodule usability issues is the best\napproach for the long term as they feel more sustainable and scalable.\nIt's an absolute pain to move a submodule URL, I think we've all\nexperienced it. It's even harder for me because I'm the go-to at work\nfor help with Git. Most people that aren't advanced with Git will not\nknow what to do without a ton of reading & such.\n\n> It might make sense to introduce a new\n>\n> [remote]\n> default = whereishouldfetch\n>\n> setting to allow the name \"origin\" above to be replaced, too. Is that\n> what you mean?\n\nI think you're on the right path. However I'd suggest something like\nthe following:\n\n[submodule]\n remote = (e.g. `upstream`)\n\n[branch.]\n submoduleRemote = \n\nAbove, `submodule.remote` is the 'default' remote used by all relative\nsubmodules on all branches. If unspecified, it defaults to\n`branch..remote` as it currently behaves.\n\n`branch..submoduleRemote` is an override for `submodule.remote`.\n\nBasically if you consider this scenario:\n\n[branch.myfoo]\n remote = origin\n submoduleRemote = upstream\n\nI can track an origin branch, but the submodule will refer to 'upstream'.\n\nI can optionally set `submodule.remote` in my global config so I don't\nneed to set the submodule remote on each branch. This is useful if\nmost branches I track are on origin instead of upstream.\n\nI'm sure I'm missing other important aspects of Git that these options\nconflict with, and perhaps 2 additional options may be too redundant.\nIt's just some food for thought.\n\n> Meanwhile it is hard to fork a project that uses relative submodule\n> URLs without also forking the submodules (or, conversely, to fork some\n> of the submodules of a project that uses absolute submodule URLs).\n> That's a real and serious problem but I'm not sure how it relates to\n> the names of remotes. My preferred fix involves teaching git to read\n> a refs/meta/git (or similarly named) ref when cloning a project with\n> submodules and let settings from .gitmodules in that ref override\n> .gitmodules in other branches. Is that what you were referring to?\n\nCould you explain this a bit more? What is refs/meta/git? Never heard\nof that one. Does that have to be done while cloning or can an\nexisting repository be configured? I'm interested in your idea but it\nsounds confusing to me.\n\nThanks for taking the time to brainstorm with me.\n"},{"id":"247912","messageId":"CAHd499CJfX_n_KnQScTFueCSkj6i0x0ozwwD8Oe_2a-VH2oq1w@mail.gmail.com","threadId":"37371","inReplyTo":"20140819102421.GA5012@book.hvoigt.net","subject":"Re: Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-19T16:15:24Z","receivedAt":"2014-08-19T16:15:24Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Tue, Aug 19, 2014 at 5:24 AM, Heiko Voigt wrote:\n> I think the OP problem stems from him having a branch that does not have a\n> remote configured. Since they do not have 'origin' as a remote and\n>\n> git submodule update --init --recursive path/to/submodule\n>\n> fails. Right?\n\nNot exactly. The issue is that there is a tug of war going on between\nthree specific commands (all of which utilize the tracked remote):\n\ngit fetch\ngit pull\ngit submodule update (for relative submodules)\n\nThe way I set up my remote tracking branch will be different for each\nof these commands:\n\n- git pull :: If I want convenient pulls (with rebase), I will track\nmy upstream branch. My pushes have to be more explicit as a tradeoff.\n- git push :: If I want convenient pushes, track my origin branch.\nPulls become less convenient. My relative submodules will now need to\nbe forked.\n- git submodule update :: I track upstream to avoid forking my\nsubmodules. But pushes become more inconvenient.\n\nAs you can see, I feel like we're overusing the single remote setting.\nSure, we've added some global settings to set default push/pull\nremotes and such, but I don't think that is a sustainable long term\nsolution. I like the idea of possibly introducing multiple tracking\nremotes for various purposes. This adds some additional configuration\noverhead (slightly), but git is already very config heavy so it might\nbe worth exploring. At least, this feels like a better thing for the\nlong term as I won't be constantly switching my tracking remote for\nvarious purposes.\n\nCould also explore the possibility of creating \"const remotes\". If we\nspecify a remote MUST exist for relative submodules, git can create it\nfor us, and fail to operate without it. It's up to the user to map\n\"fork\" to \"origin\" if needed (perhaps add a `git remote clone \n` to assist with this)?\n\nVarious approaches we can take, but I don't do development on Git so\nI'm not sure what makes the most sense.\n"},{"id":"247948","messageId":"xmqqiolowi1f.fsf@gitster.dls.corp.google.com","threadId":"37371","inReplyTo":"CAHd499CJfX_n_KnQScTFueCSkj6i0x0ozwwD8Oe_2a-VH2oq1w@mail.gmail.com","subject":"Re: Relative submodule URLs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-08-19T16:39:08Z","receivedAt":"2014-08-19T16:39:08Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Robert Dailey writes:\n\n> The way I set up my remote tracking branch will be different for each\n> of these commands:\n>\n> - git pull :: If I want convenient pulls (with rebase), I will track\n> my upstream branch. My pushes have to be more explicit as a tradeoff.\n\nKeeping 'origin' pointing at the repository where you cloned from,\nwithout doing anything funky (i.e. \"set up my remote\") would give\nyou convenient pulls.\n\n> - git push :: If I want convenient pushes, track my origin branch.\n> Pulls become less convenient. My relative submodules will now need to\n> be forked.\n\nYou need to configure your pushes to go to a different place, if you\nwant them to go to a different place ;-).\n\nLong time ago, it used to be that you have to affect the URL used in\nboth direction, making pulls less conveninent, but hasn't this been\nmade an non-issue for triangular workflows with the introduction of\nremote.pushdefault long time ago?\n\n> - git submodule update :: I track upstream to avoid forking my\n> submodules. But pushes become more inconvenient.\n\nIf 'submodule update' follows the same place as 'pull' goes by\ndefault, I would imagine that there is no issue here, no? Am I\noversimplifying the issue by guessing that the root cause of is that\nyou are not using remote.pushdefault from your configuration\ntoolchest and instead setting the 'origin' to a wrong (i.e. where\npush goes) place?\n"},{"id":"247949","messageId":"CAHd499B9Wa=Y6P+OD8Ea-6dA4yZSkGZZSR9CwZAM45evDL_Qiw@mail.gmail.com","threadId":"37371","inReplyTo":"xmqqiolowi1f.fsf@gitster.dls.corp.google.com","subject":"Re: Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-19T16:50:08Z","receivedAt":"2014-08-19T16:50:08Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Tue, Aug 19, 2014 at 11:39 AM, Junio C Hamano wrote:\n> Robert Dailey writes:\n>\n>> The way I set up my remote tracking branch will be different for each\n>> of these commands:\n>>\n>> - git pull :: If I want convenient pulls (with rebase), I will track\n>> my upstream branch. My pushes have to be more explicit as a tradeoff.\n>\n> Keeping 'origin' pointing at the repository where you cloned from,\n> without doing anything funky (i.e. \"set up my remote\") would give\n> you convenient pulls.\n>\n>> - git push :: If I want convenient pushes, track my origin branch.\n>> Pulls become less convenient. My relative submodules will now need to\n>> be forked.\n>\n> You need to configure your pushes to go to a different place, if you\n> want them to go to a different place ;-).\n>\n> Long time ago, it used to be that you have to affect the URL used in\n> both direction, making pulls less conveninent, but hasn't this been\n> made an non-issue for triangular workflows with the introduction of\n> remote.pushdefault long time ago?\n>\n>> - git submodule update :: I track upstream to avoid forking my\n>> submodules. But pushes become more inconvenient.\n>\n> If 'submodule update' follows the same place as 'pull' goes by\n> default, I would imagine that there is no issue here, no? Am I\n> oversimplifying the issue by guessing that the root cause of is that\n> you are not using remote.pushdefault from your configuration\n> toolchest and instead setting the 'origin' to a wrong (i.e. where\n> push goes) place?\n\n\nMaybe I'm misunderstanding something here and you can help me out.\n\nAll the reading I've done (mostly github) says that 'upstream' points\nto the authoritative repository that you forked from but do not have\npermissions to write to. 'origin' points to the place you push your\nchanges for pull requests (the fork).\n\nBasically the workflow I use is:\n\n- Use 'upstream' to PULL changes (latest code is obtained from the\nauthoritative repository)\n- Use 'origin' to push your branches. Since I never modify the\nbranches that exist in 'upstream' on my 'origin' (I do everything\nthrough separate personal branches).\n\nThat means if I have a branch off of 'master' named 'topic', I will\ntrack 'upstream/master' and get latest with 'git pull'. When I'm ready\nfor a pull request, I do 'git push origin' (I use push.default =\nsimple).\n\nAccording to my understanding, relative submodules work here. But not\neveryone on my team uses this workflow. Sometimes they track\n\"origin/topic\" (if we use my example again). Won't the submodule try\nto find itself on the fork now?\n\nBasically it seems like what you're advocating is that I need to\nenforce a policy of \"always track upstream\" and \"never track origin\"\nand \"always set remote.pushdefault\". Seems a bit error prone...\n"},{"id":"247961","messageId":"xmqqoavguw26.fsf@gitster.dls.corp.google.com","threadId":"37371","inReplyTo":"CAHd499B9Wa=Y6P+OD8Ea-6dA4yZSkGZZSR9CwZAM45evDL_Qiw@mail.gmail.com","subject":"Re: Relative submodule URLs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-08-19T19:19:13Z","receivedAt":"2014-08-19T19:19:13Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Robert Dailey writes:\n\n> Maybe I'm misunderstanding something here and you can help me out.\n>\n> All the reading I've done (mostly github) says that 'upstream' points\n> to the authoritative repository that you forked from but do not have\n> permissions to write to. 'origin' points to the place you push your\n> changes for pull requests (the fork).\n\nI do not know if that is how GitHub teaches people, but I would have\nto say that these are strange phrasing. I suspect that part of\ntheir documentation was written long time ago, back when nobody on\nthe GitHub side were involved in design (let alone implementation)\nof Git, and I would take it with a grain of salt.\n\nHaving said that, here is a summary of the current support for\nreferring to different repositories in Git:\n\n The word 'origin' refers to where things originate from; a place\n you push to is where things go, so it makes no sense to use that\n word to refer to the repository where you publish your work\n result. The 'origin' may or may not be where you can push (or\n you would want to push) to. It is where you 'pull' from to\n synchronize with the 'upstream'.\n\n The 'upstream' in SCM context refers to those who control a\n logically more authoritative history than you, whose work you\n derive your work from, i.e. synonymous to 'origin'.\n\n For people like Linus (i.e. he may pull from others but that is\n to take in changes made as derived work; he does not pull to grab\n more authoritative work), there is no need to say 'upstream'; or\n you can consider he is his own 'upstream'.\n\n For those who use CVS-style central repository model (i.e. they\n would pull from that single central shared repository, and push\n their work back to the same repository), 'origin' are writable to\n them and they push to them. For people with CVS-style central\n shared repository model, their central repository is their\n 'upstream' with respect to their local copy.\n\n Since these two classes of people need just one other repository\n to refer to, we just used 'origin' when we did the very initial\n version of \"git clone\", and these users can keep using that name\n to refer to that single other repository they interact with.\n\n The support for the triangular workflow in which you pull from\n one place and push the result of work to another, which the\n configuration variable 'remote.pushdefault' is a part of, is\n relatively a more recent development in Git. I am not sure we\n have added an official term to our glossary to refer to the\n repository you push your work result to, but in the discussions\n we have seen phrases like 'publishing repository' used, I think.\n It must be writable by you, of course, and it may or may not be\n the same as the 'origin' repository.\n"},{"id":"247962","messageId":"20140819193010.GA64203@book.hvoigt.net","threadId":"37371","inReplyTo":"CAHd499B9Wa=Y6P+OD8Ea-6dA4yZSkGZZSR9CwZAM45evDL_Qiw@mail.gmail.com","subject":"Re: Re: Relative submodule URLs","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-08-19T19:30:10Z","receivedAt":"2014-08-19T19:30:10Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Tue, Aug 19, 2014 at 11:50:08AM -0500, Robert Dailey wrote:\n> Maybe I'm misunderstanding something here and you can help me out.\n> \n> All the reading I've done (mostly github) says that 'upstream' points\n> to the authoritative repository that you forked from but do not have\n> permissions to write to. 'origin' points to the place you push your\n> changes for pull requests (the fork).\n> \n> Basically the workflow I use is:\n> \n> - Use 'upstream' to PULL changes (latest code is obtained from the\n> authoritative repository)\n> - Use 'origin' to push your branches. Since I never modify the\n> branches that exist in 'upstream' on my 'origin' (I do everything\n> through separate personal branches).\n> \n> That means if I have a branch off of 'master' named 'topic', I will\n> track 'upstream/master' and get latest with 'git pull'. When I'm ready\n> for a pull request, I do 'git push origin' (I use push.default =\n> simple).\n> \n> According to my understanding, relative submodules work here. But not\n> everyone on my team uses this workflow. Sometimes they track\n> \"origin/topic\" (if we use my example again). Won't the submodule try\n> to find itself on the fork now?\n\nWell the remote for the submodule is currently only calculated once,\nwhen you do the initial\n\n\tgit submodule update --init\n\nthat clones the submodule. Afterwards the fixed url is configured under\nthe name 'origin' in the submodule like in a normal git repository that\nyou have freshly cloned. Which remote is used for cloning depends on the\nconfigured remote for the current branch or 'origin'.\n\nWhen you do a fetch or push with --recurse-submodules it only executes a\n'git fetch' or 'git push' without any specific remote. For fetch the\nsame commandline options (but only the options) are passed on.\n\nHere it might make sense to guess the remote in the submodule somehow\nand not do what fetch without remotes would do.\n\nFor the triangular workflow not much work has been done in regards to\nsubmodule support.\n\nBut since a submodule behaves like a normal git repository maybe there\nis not much work needed and we can just point to the workflow without\nsubmodules most times. We still have to figure that out properly.\n\nCheers Heiko\n"},{"id":"247964","messageId":"CAHd499Ad3T1sHQWwE9NNySTbfaL_UvcdX4qrWkhH2PYhet=hMg@mail.gmail.com","threadId":"37371","inReplyTo":"xmqqoavguw26.fsf@gitster.dls.corp.google.com","subject":"Re: Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-19T20:18:17Z","receivedAt":"2014-08-19T20:18:17Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Tue, Aug 19, 2014 at 2:19 PM, Junio C Hamano wrote:\n>\n> I do not know if that is how GitHub teaches people, but I would have\n> to say that these are strange phrasing. I suspect that part of\n> their documentation was written long time ago, back when nobody on\n> the GitHub side were involved in design (let alone implementation)\n> of Git, and I would take it with a grain of salt.\n>\n> Having said that, here is a summary of the current support for\n> referring to different repositories in Git:\n>\n> \n\nWow, that was a very good read. Thank you for that. I definitely have\nbeen using the wrong terms. upstream & origin are interchangeable, yet\nI was using them to represent two distinct repositories.\n\nI think going forward my central repository will be named 'origin' and\nfor the name of the second, nothing has really jumped out at me yet\nbut it'll either be \"fork\" or \"proxy\"... \"surrogate\" would be nice too\nif it wasn't such a long word in comparison.\n\nI'm sure you guys will find a name for it in good time. I wonder what\nLinus would suggest.\n"},{"id":"247965","messageId":"CAHd499BvBBymACfHVZyuSXuNSFbT+M8my4uATOsn30w90Zb0QQ@mail.gmail.com","threadId":"37371","inReplyTo":"20140819193010.GA64203@book.hvoigt.net","subject":"Re: Re: Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-19T20:23:36Z","receivedAt":"2014-08-19T20:23:36Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Tue, Aug 19, 2014 at 2:30 PM, Heiko Voigt wrote:\n> Well the remote for the submodule is currently only calculated once,\n> when you do the initial\n>\n> git submodule update --init\n>\n> that clones the submodule. Afterwards the fixed url is configured under\n> the name 'origin' in the submodule like in a normal git repository that\n> you have freshly cloned. Which remote is used for cloning depends on the\n> configured remote for the current branch or 'origin'.\n>\n> When you do a fetch or push with --recurse-submodules it only executes a\n> 'git fetch' or 'git push' without any specific remote. For fetch the\n> same commandline options (but only the options) are passed on.\n>\n> Here it might make sense to guess the remote in the submodule somehow\n> and not do what fetch without remotes would do.\n>\n> For the triangular workflow not much work has been done in regards to\n> submodule support.\n>\n> But since a submodule behaves like a normal git repository maybe there\n> is not much work needed and we can just point to the workflow without\n> submodules most times. We still have to figure that out properly.\n\nMaybe then the only thing we need is a --with-remote option for git\nsubmodule? ::\n\ngit submodule update --init --with-remote myremote\n\nThe --with-remote option would be a NOOP if it's already initialized,\nas you say. But I could create an alias for this as needed to make\nsure it is always specified.\n\nThat way, just in case someone cloned with their fork (in which case\n'origin' would not be pointing in the right place), they could tell it\nto use `myremote`. This is really the only strange case to handle\nright now (people that clone their forks instead of the actual\nupstream/central repository).\n"},{"id":"247967","messageId":"20140819205747.GB64203@book.hvoigt.net","threadId":"37371","inReplyTo":"CAHd499BvBBymACfHVZyuSXuNSFbT+M8my4uATOsn30w90Zb0QQ@mail.gmail.com","subject":"Re: Re: Re: Relative submodule URLs","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-08-19T20:57:47Z","receivedAt":"2014-08-19T20:57:47Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Tue, Aug 19, 2014 at 03:23:36PM -0500, Robert Dailey wrote:\n> On Tue, Aug 19, 2014 at 2:30 PM, Heiko Voigt wrote:\n> > Well the remote for the submodule is currently only calculated once,\n> > when you do the initial\n> >\n> > git submodule update --init\n> >\n> > that clones the submodule. Afterwards the fixed url is configured under\n> > the name 'origin' in the submodule like in a normal git repository that\n> > you have freshly cloned. Which remote is used for cloning depends on the\n> > configured remote for the current branch or 'origin'.\n> >\n> > When you do a fetch or push with --recurse-submodules it only executes a\n> > 'git fetch' or 'git push' without any specific remote. For fetch the\n> > same commandline options (but only the options) are passed on.\n> >\n> > Here it might make sense to guess the remote in the submodule somehow\n> > and not do what fetch without remotes would do.\n> >\n> > For the triangular workflow not much work has been done in regards to\n> > submodule support.\n> >\n> > But since a submodule behaves like a normal git repository maybe there\n> > is not much work needed and we can just point to the workflow without\n> > submodules most times. We still have to figure that out properly.\n> \n> Maybe then the only thing we need is a --with-remote option for git\n> submodule? ::\n> \n> git submodule update --init --with-remote myremote\n> \n> The --with-remote option would be a NOOP if it's already initialized,\n> as you say. But I could create an alias for this as needed to make\n> sure it is always specified.\n\nI would actually error out when specified in already cloned state.\nBecause otherwise the user might expect the remote to be updated.\n\nSince we are currently busy implementing recursive fetch and checkout I have\nadded that to our ideas list[1] so we do not forget about it.\n\nIn the meantime you can either use the branch..remote\nconfiguration to define a remote to use or just use 'origin'.\n\nCheers Heiko\n\n[1] https://github.com/jlehmann/git-submod-enhancements/wiki#add-with-remote--switch-to-submodule-update\n"},{"id":"248017","messageId":"CAHd499C2F2s51qmfwS5VxLvP-O2Wjdb6-yfj+T9tMkZz7ohVXQ@mail.gmail.com","threadId":"37371","inReplyTo":"20140819205747.GB64203@book.hvoigt.net","subject":"Re: Re: Re: Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-20T13:18:12Z","receivedAt":"2014-08-20T13:18:12Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Tue, Aug 19, 2014 at 3:57 PM, Heiko Voigt wrote:\n> I would actually error out when specified in already cloned state.\n> Because otherwise the user might expect the remote to be updated.\n>\n> Since we are currently busy implementing recursive fetch and checkout I have\n> added that to our ideas list[1] so we do not forget about it.\n>\n> In the meantime you can either use the branch..remote\n> configuration to define a remote to use or just use 'origin'.\n>\n> Cheers Heiko\n>\n> [1] https://github.com/jlehmann/git-submod-enhancements/wiki#add-with-remote--switch-to-submodule-update\n\nThanks Heiko.\n\nI would offer to help implement this for you, if you find it to be a\ngood idea, but I've never done git development before and based on\nwhat I've seen it seems like you need to know at least 2-3 languages\nto contribute: bash, perl, C++. I know C++ & Python but I don't know\nperl or bash scripting language.\n\nWhat would it take to help you guys out? It's easy to complain & file\nbugs but as a developer I feel like I should offer more, if it suits\nyou.\n\nLet me know I'm happy to help with anything. Thanks again!!\n"},{"id":"248053","messageId":"20140821123705.GA68456@book.hvoigt.net","threadId":"37371","inReplyTo":"CAHd499C2F2s51qmfwS5VxLvP-O2Wjdb6-yfj+T9tMkZz7ohVXQ@mail.gmail.com","subject":"Re: Re: Re: Relative submodule URLs","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-08-21T12:37:05Z","receivedAt":"2014-08-21T12:37:05Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Wed, Aug 20, 2014 at 08:18:12AM -0500, Robert Dailey wrote:\n> On Tue, Aug 19, 2014 at 3:57 PM, Heiko Voigt wrote:\n> > I would actually error out when specified in already cloned state.\n> > Because otherwise the user might expect the remote to be updated.\n> >\n> > Since we are currently busy implementing recursive fetch and checkout I have\n> > added that to our ideas list[1] so we do not forget about it.\n> >\n> > In the meantime you can either use the branch..remote\n> > configuration to define a remote to use or just use 'origin'.\n> >\n> > Cheers Heiko\n> >\n> > [1] https://github.com/jlehmann/git-submod-enhancements/wiki#add-with-remote--switch-to-submodule-update\n> \n> Thanks Heiko.\n> \n> I would offer to help implement this for you, if you find it to be a\n> good idea, but I've never done git development before and based on\n> what I've seen it seems like you need to know at least 2-3 languages\n> to contribute: bash, perl, C++. I know C++ & Python but I don't know\n> perl or bash scripting language.\n> \n> What would it take to help you guys out? It's easy to complain & file\n> bugs but as a developer I feel like I should offer more, if it suits\n> you.\n\nFor this particular case shell scripting should be sufficient. And it\nshould not take too much time. Have a look at the git-submodule.sh\nscript in the repository. That is the one implementing the git submodule\ncommand.\n\nAdditionally you need to extend the documentation and write a test or\ntwo. Writing a test is also done in shell script. The documentation[1] is\nin asciidoc which is pretty self explanatory.\n\nThe test should probably go into t/t7406-submodule-update.sh and, as\nPhil pointed out, in t7403-submodule-sync.sh).\n\nAlso make sure to read the shell scripting part in\nDocumentation/CodingGuidelines and as a general rule: Keep close to the\nstyle you find in the file. And when you are ready to send a patch:\nDocumentation/SubmittingPatches.\n\nIf you are happy but unsure about anything just send a patch with your\nimplementation (CC me and everyone involved) and we will discuss it here\non the list.\n\nCheers Heiko\n\n[1] Documentation/git-submodule.txt\n"},{"id":"248125","messageId":"53F76907.1090904@xiplink.com","threadId":"37371","inReplyTo":"CAHd499DVf4N3Y6m5qoiy-WQGX4K54umefRzehZMsrxiWbaiZ=g@mail.gmail.com","subject":"Re: Relative submodule URLs","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2014-08-22T16:00:07Z","receivedAt":"2014-08-22T16:00:07Z","isPatch":false,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 14-08-19 12:07 PM, Robert Dailey wrote:\n> On Mon, Aug 18, 2014 at 3:55 PM, Jonathan Nieder wrote:\n>> Thanks for reporting. The remote used is the default remote that \"git\n>> fetch\" without further arguments would use:\n>>\n>> get_default_remote () {\n>> curr_branch=$(git symbolic-ref -q HEAD)\n>> curr_branch=\"${curr_branch#refs/heads/}\"\n>> origin=$(git config --get \"branch.$curr_branch.remote\")\n>> echo ${origin:-origin}\n>> }\n>>\n>> The documentation is wrong. git-fetch(1) doesn't provide a name for\n>> this thing. Any ideas for wording?\n> \n> I guess a good start would be to call it the \"tracked remote\" instead\n> of \"remote origin\". The word \"tracked\" here makes it very obvious that\n> if I have a remote tracking branch setup, it will use the remote\n> portion of that configuration.\n> \n> The real question is, how will `git submodule update` function if a\n> tracking remote is not configured? Will it fail with some useful error\n> message? I don't like the idea of it defaulting to \"self remote\" mode,\n> where it will be relative to my repo directory. That seems like it\n> would fail is subtle ways in a worst-case scenario (if I did by\n> happenstance have a bare repository cloned up one directory level for\n> other reasons).\n> \n>> Currently there isn't, short of reconfiguring the remote used by\n>> default by \"git fetch\".\n> \n> I wish there was a way to specify the remote on a per-branch basis\n> separately from the tracking branch. I read a while back that someone\n> proposed some changes to git to support decentralized tracking\n> (concept of an upstream tracking branch and a separate one for origin,\n> i think). If that were implemented, then relative submodules could\n> utilize the 'upstream' remote by default for each branch, which would\n> provide more predictable default behavior. Right now most people on my\n> team would not be aware that if they tracked a branch on their fork,\n> they would subsequently need to fork the submodules to that same\n> remote.\n> \n>>> Various co-workers use the remote named \"central\" instead of\n>>> \"upstream\" and \"fork\" instead of \"origin\" (because that just makes\n>>> more sense to them and it's perfectly valid).\n>>>\n>>> However if relative submodules require 'origin' to exist AND also\n>>> represent the upstream repository (in triangle workflow), then this\n>>> breaks on several levels.\n>>\n>> Can you explain further? In a triangle workflow, \"git fetch\" will\n>> pull from the 'origin' remote by default and will push to the remote\n>> named in the '[remote] pushdefault' setting (see \"remote.pushdefault\"\n>> in git-config(1)). So you can do\n>>\n>> [remote]\n>> pushDefault = whereishouldpush\n>>\n>> and then 'git fetch' and 'git fetch --recurse-submodules' will fetch\n>> from \"origin\" and 'git push' will push to the whereishouldpush remote.\n> \n> I didn't know about this option, seems useful.\n> \n> A common workflow that we use on my team is to set the tracking branch\n> to 'upstream' for convenient pulls with rebase. This means a feature\n> branch of mine can track its parent branch on 'upstream', so that when\n> other pull requests get merged in on the remote repo branch, I can\n> just do `git pull` and my feature branch rebases onto the latest of\n> that parent branch.\n> \n> Cases like these would work with relative submodules because\n> 'upstream' is the tracked remote (and most of the time we don't want\n> to fork submodules). However sometimes I like to track the same pushed\n> branch on origin (my fork), especially when it is up for pull request.\n> In these cases, my submodule update will fail because I didn't fork my\n> submodules when I changed my tracking branch. Is this correct?\n> \n> \"breaks on several levels\" was basically my way of saying that various\n> workflow choices will break when you introduce submodules. One of the\n> beautiful things about Git is that it allows everyone to choose their\n> own workflow. But submodules seem to prevent that to some degree. I\n> think addressing relative submodule usability issues is the best\n> approach for the long term as they feel more sustainable and scalable.\n> It's an absolute pain to move a submodule URL, I think we've all\n> experienced it. It's even harder for me because I'm the go-to at work\n> for help with Git. Most people that aren't advanced with Git will not\n> know what to do without a ton of reading & such.\n> \n>> It might make sense to introduce a new\n>>\n>> [remote]\n>> default = whereishouldfetch\n>>\n>> setting to allow the name \"origin\" above to be replaced, too. Is that\n>> what you mean?\n\nA couple of years ago I started to work on such a thing ([1] [2] [3]), mainly\nbecause when we tried to change to relative submodules we got bitten when\nsomeone used clone's -o option so that his super-repo had no \"origin\" remote\n*and* his was checked out on a detached HEAD. So get_default_remote() failed\nfor him.\n\nI didn't have time to complete the work -- it ended up being quite involved.\n But Junio did come up with an excellent transition plan [4] for adopting a\ndefault remote setting.\n\n[1] (v0) http://thread.gmane.org/gmane.comp.version-control.git/200145\n[2] (v1) http://thread.gmane.org/gmane.comp.version-control.git/201065\n[3] (v2) http://thread.gmane.org/gmane.comp.version-control.git/201306\n[4] http://article.gmane.org/gmane.comp.version-control.git/201332\n\n> I think you're on the right path. However I'd suggest something like\n> the following:\n> \n> [submodule]\n> remote = (e.g. `upstream`)\n\nI think remote.default would be more generally useful, especially when\nworking with detached checkouts.\n\n(For the record, I would also be happy if clone got rid of its -o option and\n\"origin\" became the sacred, reserved remote name (perhaps translated into\nother languages as needed) that clone always uses no matter what.)\n\n> [branch.]\n> submoduleRemote = \n\nIf I understand correctly, you want this so that your branch can be a fork of\nonly the super-repo while the submodules are not forked and so they should\nkeep tracking their original repo.\n\nTo me this seems to be going in the opposite direction of having branches\nrecursively apply to submodules, which I think most of us want.\n\nA branch should fork the entire repo, including its submodules. The\nimplication is that if you want to push that branch somewhere, that somewhere\nneeds to be able to accept the forks of the submodules *even if those\nsubmodules aren't changed in your branch* because at the very least the\nbranch ref has to exist in the submodules' repositories.\n\nWith absolute-path submodules, the push is a simple as creating the branch\nref in the submodules' \"home\" repositories -- even if the main \"somewhere\"\nyou're pushing to isn't one of those repositories.\n\nWith relative-path submodules, the push's target repo *must* also have the\nsubmodules in their proper places, so that they can get updated.\nFurthermore, if you clone a repo that has relative-path submodules you *must*\nalso clone the submodules.\n\nRobert, I think what you'll say to this is that you still want your branch to\ntrack the latest submodules updates from their \"home\" repository. (BTW, I'm\nconfused with how you're using the terms \"upstream\" and \"origin\". I'll use\n\"home\" to refer to the repository where everything starts from, and \"fork\"\nfor the repository that your branch tracks.) Well, you get the updates you\nwant when your branch tracks a ref in the \"home\" repository. But when your\nbranch starts tracking a ref in another \"fork\" repository then you'll get the\nsubmodule updates in that ref's history from that \"fork\" repository.\n\nOnce your branch is tracking the \"fork\" repository, if you do a pull you\nwon't get any submodule updates because the fork's branch hasn't changed.\nYou need to fetch (recursively) from the \"home\" repo to get the submodule\nupdates (assuming one of the \"home\" repo's branches has updated its\nsubmodules). Then, with your branch checked out in the super-repo, if you\ncheck out the latest refs in your submodules git will tell you that you have\nuncommitted changes in your branch. The correct way to get submodule updates\ninto your branch is to commit them. Even though you're doing a pull/rebase,\nthere's nothing to rebase onto in the \"fork\" repository that has the updated\nsubmodules.\n\n\t\tM.\n"},{"id":"248200","messageId":"20140824133428.GA2839@book.hvoigt.net","threadId":"37371","inReplyTo":"53F76907.1090904@xiplink.com","subject":"Re: Re: Relative submodule URLs","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-08-24T13:34:28Z","receivedAt":"2014-08-24T13:34:28Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"Hi,\n\nsince the mail got quite long. To avoid 'tl;dr', I talk about two topics\nin this mail:\n\n * Submodule settings for default remote (complex, future)\n * New --with--remote parameter for 'git submodule' (simple, now)\n\nDepending on your interest you might want to skip the first part of the\nemail.\n\nI think they are two separate topics. Please only answer to either one\nand remove the other. That way we split the thread here and not mix the\ntwo together anymore.\n\nOn Fri, Aug 22, 2014 at 12:00:07PM -0400, Marc Branchaud wrote:\n> > I think you're on the right path. However I'd suggest something like\n> > the following:\n> > \n> > [submodule]\n> > remote = (e.g. `upstream`)\n> \n> I think remote.default would be more generally useful, especially when\n> working with detached checkouts.\n\nDepends what workflow you have. Especially for submodules where the\ndefault remote might change from branch to branch this is not\nnecessarily true. The following drawbacks in relation to submodules come\nto my mind:\n\n * You can not transport such configuration to the server. In case\n you are developing on a branch which has changes in a forked\n submodule that would be useful.\n\n * When your development in superproject and submodule gets merged to a\n stable branch (i.e. master) you also may not want that other remote\n anymore. So a setting, that can be per branch, might be preferred.\n\n * When your development gets pushed to a different remote the settings\n do not change. I.e. once part of the upstream repository the\n settings should possibly disappear.\n\n * You might only want to fork a certain submodule (since thats the\n only one you need to make changes in) in your branch. Then you need\n this setting to be per submodule.\n\nSo to sum up a default remote setting which would be generally useful\nfor submodules needs the following properties (IMO):\n\n * pushable\n * per branch\n * per remote\n * per submodule\n\nAll of these being optional, so in case you have a local mirror,\nincluding submodules, of some project in which you develop with your\nteam you might just want to set the default remote once for all\nsubmodules.\n\nI have not completely thought that through but the special ref idea[3]\ndescribed by Jonathan seems to make it possible to implement all these\nproperties.\n\n> > [branch.]\n> > submoduleRemote = \n> \n> If I understand correctly, you want this so that your branch can be a fork of\n> only the super-repo while the submodules are not forked and so they should\n> keep tracking their original repo.\n> \n> To me this seems to be going in the opposite direction of having branches\n> recursively apply to submodules, which I think most of us want.\n\nI disagree. While recursive branches might make sense in some\nsituations in most it does not. Consider a project in which you use a\nlibrary which is separately maintained. You develop on featureA in your\nproject and discover a bug in the submodule which you fix on a branch\n(which is then tracked in the submodule). Here it does not make sense\nto call your branch in the submodule featureA, since the submodule has no\nknowledge at all (and should not) about this featureA.\n\nWhile having said that, for a simple workflow while developing a certain\nfeature recursive branches make sense. Lets say as a temporary local\nbranch you could have that featureA branch in your submodule and just\ncommit any changes you need in the submodule on that branch (including\nextensions and stuff). Later in the process you divide up that branch in\nthe submodule into cleanups, bugfixes, extensions, ... to push it\nupstream for review and integration.\n\n> A branch should fork the entire repo, including its submodules. The\n> implication is that if you want to push that branch somewhere, that somewhere\n> needs to be able to accept the forks of the submodules *even if those\n> submodules aren't changed in your branch* because at the very least the\n> branch ref has to exist in the submodules' repositories.\n\nI disagree here as well. As the distributed nature of git allows to have\ndifferent remotes, I think its perfectly legitimate to just fork the\nrepositories you need to change. It should be easy to work on a\nrepository that is forked in its entirety, but it should also be possible\n(and properly supported) to only fork some submodules. I know it does\nmake the situation more complex, but I think we should properly define\nthe goal beforehand, so we do not exclude any use-cases. Then we can go\nahead and just implement the simpler stuff (like entire repo forks)\nfirst, while making sure we do not block the more complex use-cases.\n\n> With absolute-path submodules, the push is a simple as creating the branch\n> ref in the submodules' \"home\" repositories -- even if the main \"somewhere\"\n> you're pushing to isn't one of those repositories.\n> \n> With relative-path submodules, the push's target repo *must* also have the\n> submodules in their proper places, so that they can get updated.\n> Furthermore, if you clone a repo that has relative-path submodules you *must*\n> also clone the submodules.\n\nThat is not true. You can have relative submodules and just clone/fetch\nsome from a different remote. Its just a question of how to\nspecifiy/transport this information.\n\n> Robert, I think what you'll say to this is that you still want your branch to\n> track the latest submodules updates from their \"home\" repository. (BTW, I'm\n> confused with how you're using the terms \"upstream\" and \"origin\". I'll use\n> \"home\" to refer to the repository where everything starts from, and \"fork\"\n> for the repository that your branch tracks.) Well, you get the updates you\n> want when your branch tracks a ref in the \"home\" repository. But when your\n> branch starts tracking a ref in another \"fork\" repository then you'll get the\n> submodule updates in that ref's history from that \"fork\" repository.\n> \n> Once your branch is tracking the \"fork\" repository, if you do a pull you\n> won't get any submodule updates because the fork's branch hasn't changed.\n> You need to fetch (recursively) from the \"home\" repo to get the submodule\n> updates (assuming one of the \"home\" repo's branches has updated its\n> submodules). Then, with your branch checked out in the super-repo, if you\n> check out the latest refs in your submodules git will tell you that you have\n> uncommitted changes in your branch. The correct way to get submodule updates\n> into your branch is to commit them. Even though you're doing a pull/rebase,\n> there's nothing to rebase onto in the \"fork\" repository that has the updated\n> submodules.\n\nLet's please use the terms as Junio described them here[1]. Do not add\nconfusion by introducing new terms when we do not need them. If we need\nnew ones lets introduce them properly. Maybe even define them in the\nglossary. I feel for some things it is needed when talking about\nsubmodules, so we talk about the same things.\n\n\nNew --with--remote parameter for 'git submodule'\n------------------------------------------------\n\nWhile having said all that about submodule settings I think a much\nmuch simpler start is to go ahead with a commandline setting, like\nRobert proposed here[2].\n\nFor that we do not have to worry about how it can be stored,\ntransported, defined per submodule or on a branch, since answers to this\nare given at the commandline (and current repository state).\n\nThere are still open questions about this though:\n\n * Should the name in the submodule be 'origin' even though you\n specified --with-remote=somewhere? For me its always confusing to\n have the same/similar remotes named differently in different\n repositories. That why I try to keep the names the same in all my\n clones of repositories (i.e. for my private, github, upstream\n remotes).\n\n * When you do a 'git submodule sync --with-remote=somewhere' should\n the remote be added or replaced.\n\nMy opinion on these are:\n\nThe remote should be named as in the superproject so\n--with-remote=somewhere adds/replaces the remote 'somewhere' in the\nsubmodules named on the commandline (or all in case no submodule is\nspecified). In case of a fresh clone of the submodule, there would be no\norigin but only a remote under the new name.\n\nWould the --with-remote feature I describe be a feasible start for you\nRobert? What do others think? Is the naming of the parameter\n'--with-remote' alright?\n\nCheers Heiko\n\n[1] http://article.gmane.org/gmane.comp.version-control.git/255512\n[2] http://article.gmane.org/gmane.comp.version-control.git/255512\n[3] https://github.com/jlehmann/git-submod-enhancements/wiki#special-ref-overriding-gitmodules-values\n"},{"id":"248227","messageId":"CAHd499D+xAKidgah6xEVTfS8FQb7zJeo-rd=Awo5AyHKpqy_cA@mail.gmail.com","threadId":"37371","inReplyTo":"53F76907.1090904@xiplink.com","subject":"Re: Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-25T13:48:45Z","receivedAt":"2014-08-25T13:48:45Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Fri, Aug 22, 2014 at 11:00 AM, Marc Branchaud wrote:\n> A couple of years ago I started to work on such a thing ([1] [2] [3]), mainly\n> because when we tried to change to relative submodules we got bitten when\n> someone used clone's -o option so that his super-repo had no \"origin\" remote\n> *and* his was checked out on a detached HEAD. So get_default_remote() failed\n> for him.\n>\n> I didn't have time to complete the work -- it ended up being quite involved.\n> But Junio did come up with an excellent transition plan [4] for adopting a\n> default remote setting.\n>\n> [1] (v0) http://thread.gmane.org/gmane.comp.version-control.git/200145\n> [2] (v1) http://thread.gmane.org/gmane.comp.version-control.git/201065\n> [3] (v2) http://thread.gmane.org/gmane.comp.version-control.git/201306\n> [4] http://article.gmane.org/gmane.comp.version-control.git/201332\n>\n>> I think you're on the right path. However I'd suggest something like\n>> the following:\n>>\n>> [submodule]\n>> remote = (e.g. `upstream`)\n>\n> I think remote.default would be more generally useful, especially when\n> working with detached checkouts.\n\nHonestly speaking I don't use default.remote, even now that I know\nabout it thanks to the discussion ongoing here. The reason is that\nsometimes I push my branches to origin, sometimes I push them to my\nfork. I like explicit control as to which one I push to. I also sync\nmy git config file to dropbox and I use it on multiple projects and\nplatforms. I don't use the same push destination workflow on all\nprojects. It seems to get in the way of my workflow more than it\nhelps. I really only ever have two needs:\n\n1. Push explicitly to my remote (e.g. `git push fork` or `git push origin`)\n2. Push to the tracked branch (e.g. `git push`)\n\nI'm also not sure how `push.default = simple` conflicts with the usage\nof `remote.default`, since in the tracked-repo case, you must\nexplicitly specify the source ref to push. Is this behavior documented\nsomewhere?\n\n> (For the record, I would also be happy if clone got rid of its -o option and\n> \"origin\" became the sacred, reserved remote name (perhaps translated into\n> other languages as needed) that clone always uses no matter what.)\n>\n>> [branch.]\n>> submoduleRemote = \n>\n> If I understand correctly, you want this so that your branch can be a fork of\n> only the super-repo while the submodules are not forked and so they should\n> keep tracking their original repo.\n\nThat's correct. But this is case-by-case. Sometimes I make a change\nwhere I want the submodule forked (rare), most times I don't.\nSometimes I can get away with pushing changes to the submodule and\nworrying about it later since I know the submodule ref won't move\nforward unless someone does update --remote (which isn't often or only\ndone as needed).\n\n> To me this seems to be going in the opposite direction of having branches\n> recursively apply to submodules, which I think most of us want.\n>\n> A branch should fork the entire repo, including its submodules. The\n> implication is that if you want to push that branch somewhere, that somewhere\n> needs to be able to accept the forks of the submodules *even if those\n> submodules aren't changed in your branch* because at the very least the\n> branch ref has to exist in the submodules' repositories.\n\nThere are many levels on which this can apply. When it comes to\ncheckouts and such, I agree. However, how will this impact *creating*\nbranches? What about forking? Do you expect submodule forking &\nbranching to be automatic as well? Based on your description, it seems\nso (although a new branch doesn't necessarily have to correspond to a\nnew fork, unless I'm misunderstanding you). This seems difficult to\ndo, especially the forking part since you would need an API for this\n(Github, Atlassian Stash, etc), unless you are thinking of something\nclever like local/relative forks.\n\nHowever the inconvenience of forking manually isn't the main reason\nwhy I avoid forking submodules. It's the complication of pull\nrequests. There is no uniformity there, which is unfortunate.\nRecursive pull requests are something outside the scope of git, I\nrealize that, but it would still be nice. However the suggestion you\nmake here lays the foundation for that I think.\n\n> With absolute-path submodules, the push is a simple as creating the branch\n> ref in the submodules' \"home\" repositories -- even if the main \"somewhere\"\n> you're pushing to isn't one of those repositories.\n>\n> With relative-path submodules, the push's target repo *must* also have the\n> submodules in their proper places, so that they can get updated.\n> Furthermore, if you clone a repo that has relative-path submodules you *must*\n> also clone the submodules.\n>\n> Robert, I think what you'll say to this is that you still want your branch to\n> track the latest submodules updates from their \"home\" repository. (BTW, I'm\n> confused with how you're using the terms \"upstream\" and \"origin\". I'll use\n> \"home\" to refer to the repository where everything starts from, and \"fork\"\n> for the repository that your branch tracks.) Well, you get the updates you\n> want when your branch tracks a ref in the \"home\" repository. But when your\n> branch starts tracking a ref in another \"fork\" repository then you'll get the\n> submodule updates in that ref's history from that \"fork\" repository.\n\nMy usage of 'upstream' and 'origin' were wrong. I don't use upstream\nanymore, based on the explanations I've received here. I use the\nfollowing now:\n\norigin = my central repository (authoritative)\nfork = My fork of the central repo\n\nI like your idea of forking/branching on submodules being recursive\nbased on the super repo, but I just don't see how this is possible.\nHow would git tell github to fork, for example? And would that also\nwork on Stash?\n\n> Once your branch is tracking the \"fork\" repository, if you do a pull you\n> won't get any submodule updates because the fork's branch hasn't changed.\n> You need to fetch (recursively) from the \"home\" repo to get the submodule\n> updates (assuming one of the \"home\" repo's branches has updated its\n> submodules). Then, with your branch checked out in the super-repo, if you\n> check out the latest refs in your submodules git will tell you that you have\n> uncommitted changes in your branch. The correct way to get submodule updates\n> into your branch is to commit them. Even though you're doing a pull/rebase,\n> there's nothing to rebase onto in the \"fork\" repository that has the updated\n> submodules.\n\nI like your ideas, assuming they are technically possible. They sound\nlike great solutions for the long term. However for now, the whole\nprocess of working with remotes is very confusing. At first it was\ncomplicated when it came to triangle workflow. Mostly because the way\nyou set push.default changes completely between the two, especially\nwhen combined with various workflows.\n\nAdd on top of that the complexity of workflows for submodules, and it\nbecomes a complete mess. Maybe for you guys who actively develop and\nunderstand git's internals it isn't so bad. However I don't have that\ndomain knowledge, so I only have a \"user\" perspective on the matter.\n`remote.default` sounds nice but how do I use it based on my response\nin my first paragraph above?\n\nThanks guys, this is great discussion.\n"},{"id":"248229","messageId":"CAHd499Bat5AFc1E4Sfezx9Qi8LPsxkyKKkBb2eg8JQb9kdPzgg@mail.gmail.com","threadId":"37371","inReplyTo":"20140824133428.GA2839@book.hvoigt.net","subject":"Re: Re: Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-25T14:29:07Z","receivedAt":"2014-08-25T14:29:07Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Sun, Aug 24, 2014 at 8:34 AM, Heiko Voigt wrote:\n> New --with--remote parameter for 'git submodule'\n> ------------------------------------------------\n>\n> While having said all that about submodule settings I think a much\n> much simpler start is to go ahead with a commandline setting, like\n> Robert proposed here[2].\n>\n> For that we do not have to worry about how it can be stored,\n> transported, defined per submodule or on a branch, since answers to this\n> are given at the commandline (and current repository state).\n>\n> There are still open questions about this though:\n>\n> * Should the name in the submodule be 'origin' even though you\n> specified --with-remote=somewhere? For me its always confusing to\n> have the same/similar remotes named differently in different\n> repositories. That why I try to keep the names the same in all my\n> clones of repositories (i.e. for my private, github, upstream\n> remotes).\n>\n> * When you do a 'git submodule sync --with-remote=somewhere' should\n> the remote be added or replaced.\n>\n> My opinion on these are:\n>\n> The remote should be named as in the superproject so\n> --with-remote=somewhere adds/replaces the remote 'somewhere' in the\n> submodules named on the commandline (or all in case no submodule is\n> specified). In case of a fresh clone of the submodule, there would be no\n> origin but only a remote under the new name.\n>\n> Would the --with-remote feature I describe be a feasible start for you\n> Robert? What do others think? Is the naming of the parameter\n> '--with-remote' alright?\n>\n> Cheers Heiko\n>\n> [1] http://article.gmane.org/gmane.comp.version-control.git/255512\n> [2] http://article.gmane.org/gmane.comp.version-control.git/255512\n> [3] https://github.com/jlehmann/git-submod-enhancements/wiki#special-ref-overriding-gitmodules-values\n\nHi Heiko,\n\nMy last email response was in violation of your request to keep the\ntwo topics separate, sorry about that. I started typing it this\nweekend and completed the draft this morning, without having read this\nresponse from you first. At this point my only intention was to start\ndiscussion on a possible short-term solution. I realize the Git\ndevelopers are working hard on improving submodule workflow for the\nlong term. In addition I do not have the domain expertise to properly\nmake suggestions in regards to longer-term solutions, so I leave that\nto you :-)\n\nThe --with-remote feature would allow me to begin using relative\nsubmodules because:\n\nOn a per-submodule basis, I can specify the remote it will use. When I\nfork a submodule and need to start tracking it, I can run `git\nsubmodule sync --with-remote fork`, which will take my super repo's\n'fork' remote, REPLACE 'origin' in the submodule with that URL, and\nalso redo the relative URL calculation. This is ideal since I use HTTP\nat home (so I can use my proxy server to access git behind firewall at\nwork) and at work physically I use SSH for performance (to avoid HTTP\nprotocol). I also like the idea of \"never\" having to update my\nsubmodule URLs again if the git server moves, domain name changes, or\nwhatever else.\n\nHere is what I think would make the feature most usable. I think you\nwent over some of these ideas but I just want to clarify, to make sure\nwe're on the same page. Please correct me as needed.\n\n1. Running `git submodule update --with-remote ` shall fail the\ncommand unconditionally.\n2. Using the `--with-remote` option on submodule `update` or `sync`\nwill fail if it detects absolute submodule URLs in .gitmodule\n3. Running `git submodule update --init --with-remote ` shall\nfail the command ONLY if a submodule is being processed that is NOT\nalso being initialized.\n4. The behavior of git submodule's `update` or `sync` commands\ncombined with `--with-remote` will REPLACE or CREATE the 'origin'\nremote in each submodule it is run in. We will not allow the user to\nconfigure what the submodule remote name will end up being (I think\nthis is current behavior and forces good practice; I consider `origin`\nan adopted standard for git, and actually wish it was more enforced\nfor super projects as well!)\n\nLet me know if I've missed anything. Once we clarify requirements I'll\nattempt to start work on this during my free time. I'll start by\ntesting this through msysgit, since I do not have linux installed, but\nI have Linux Mint running in a Virtual Machine so I can test on both\nplatforms as needed (I don't have a lot of experience on Linux\nthough).\n\nI hope you won't mind me reaching out for questions as needed, however\nI will attempt to be as resourceful as possible since I know you're\nall busy. Thanks.\n"},{"id":"248230","messageId":"CAHd499DAY4XNPZakV6gxYLGQykPBjaB2ONY7kwyrEnuGDANVAg@mail.gmail.com","threadId":"37371","inReplyTo":"CAHd499Bat5AFc1E4Sfezx9Qi8LPsxkyKKkBb2eg8JQb9kdPzgg@mail.gmail.com","subject":"Re: Re: Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-25T14:32:27Z","receivedAt":"2014-08-25T14:32:27Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Mon, Aug 25, 2014 at 9:29 AM, Robert Dailey wrote:\n> On Sun, Aug 24, 2014 at 8:34 AM, Heiko Voigt wrote:\n>> New --with--remote parameter for 'git submodule'\n>> ------------------------------------------------\n>>\n>> While having said all that about submodule settings I think a much\n>> much simpler start is to go ahead with a commandline setting, like\n>> Robert proposed here[2].\n>>\n>> For that we do not have to worry about how it can be stored,\n>> transported, defined per submodule or on a branch, since answers to this\n>> are given at the commandline (and current repository state).\n>>\n>> There are still open questions about this though:\n>>\n>> * Should the name in the submodule be 'origin' even though you\n>> specified --with-remote=somewhere? For me its always confusing to\n>> have the same/similar remotes named differently in different\n>> repositories. That why I try to keep the names the same in all my\n>> clones of repositories (i.e. for my private, github, upstream\n>> remotes).\n>>\n>> * When you do a 'git submodule sync --with-remote=somewhere' should\n>> the remote be added or replaced.\n>>\n>> My opinion on these are:\n>>\n>> The remote should be named as in the superproject so\n>> --with-remote=somewhere adds/replaces the remote 'somewhere' in the\n>> submodules named on the commandline (or all in case no submodule is\n>> specified). In case of a fresh clone of the submodule, there would be no\n>> origin but only a remote under the new name.\n>>\n>> Would the --with-remote feature I describe be a feasible start for you\n>> Robert? What do others think? Is the naming of the parameter\n>> '--with-remote' alright?\n>>\n>> Cheers Heiko\n>>\n>> [1] http://article.gmane.org/gmane.comp.version-control.git/255512\n>> [2] http://article.gmane.org/gmane.comp.version-control.git/255512\n>> [3] https://github.com/jlehmann/git-submod-enhancements/wiki#special-ref-overriding-gitmodules-values\n>\n> Hi Heiko,\n>\n> My last email response was in violation of your request to keep the\n> two topics separate, sorry about that. I started typing it this\n> weekend and completed the draft this morning, without having read this\n> response from you first. At this point my only intention was to start\n> discussion on a possible short-term solution. I realize the Git\n> developers are working hard on improving submodule workflow for the\n> long term. In addition I do not have the domain expertise to properly\n> make suggestions in regards to longer-term solutions, so I leave that\n> to you :-)\n>\n> The --with-remote feature would allow me to begin using relative\n> submodules because:\n>\n> On a per-submodule basis, I can specify the remote it will use. When I\n> fork a submodule and need to start tracking it, I can run `git\n> submodule sync --with-remote fork`, which will take my super repo's\n> 'fork' remote, REPLACE 'origin' in the submodule with that URL, and\n> also redo the relative URL calculation. This is ideal since I use HTTP\n> at home (so I can use my proxy server to access git behind firewall at\n> work) and at work physically I use SSH for performance (to avoid HTTP\n> protocol). I also like the idea of \"never\" having to update my\n> submodule URLs again if the git server moves, domain name changes, or\n> whatever else.\n>\n> Here is what I think would make the feature most usable. I think you\n> went over some of these ideas but I just want to clarify, to make sure\n> we're on the same page. Please correct me as needed.\n>\n> 1. Running `git submodule update --with-remote ` shall fail the\n> command unconditionally.\n> 2. Using the `--with-remote` option on submodule `update` or `sync`\n> will fail if it detects absolute submodule URLs in .gitmodule\n> 3. Running `git submodule update --init --with-remote ` shall\n> fail the command ONLY if a submodule is being processed that is NOT\n> also being initialized.\n> 4. The behavior of git submodule's `update` or `sync` commands\n> combined with `--with-remote` will REPLACE or CREATE the 'origin'\n> remote in each submodule it is run in. We will not allow the user to\n> configure what the submodule remote name will end up being (I think\n> this is current behavior and forces good practice; I consider `origin`\n> an adopted standard for git, and actually wish it was more enforced\n> for super projects as well!)\n>\n> Let me know if I've missed anything. Once we clarify requirements I'll\n> attempt to start work on this during my free time. I'll start by\n> testing this through msysgit, since I do not have linux installed, but\n> I have Linux Mint running in a Virtual Machine so I can test on both\n> platforms as needed (I don't have a lot of experience on Linux\n> though).\n>\n> I hope you won't mind me reaching out for questions as needed, however\n> I will attempt to be as resourceful as possible since I know you're\n> all busy. Thanks.\n\nThought of a few more:\n\n5. If `--with-remote` is unspecified, behavior will continue as it\ncurrently does (I'm not clear on the precedence here of various\noptions, but I assume: `remote.default` first, then\n`branch.name.remote`)\n6. `--with-remote` will take precedence over `remote.default` and\n`branch.name.remote`.\n\nI'll add more as I think of them... Sorry for the spam.\n"},{"id":"248280","messageId":"20140826062847.GA12216@book.hvoigt.net","threadId":"37371","inReplyTo":"CAHd499Bat5AFc1E4Sfezx9Qi8LPsxkyKKkBb2eg8JQb9kdPzgg@mail.gmail.com","subject":"Re: Re: Re: Relative submodule URLs","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-08-26T06:28:47Z","receivedAt":"2014-08-26T06:28:47Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Mon, Aug 25, 2014 at 09:29:07AM -0500, Robert Dailey wrote:\n> On Sun, Aug 24, 2014 at 8:34 AM, Heiko Voigt wrote:\n> > New --with--remote parameter for 'git submodule'\n> > ------------------------------------------------\n> >\n> > While having said all that about submodule settings I think a much\n> > much simpler start is to go ahead with a commandline setting, like\n> > Robert proposed here[2].\n> >\n> > For that we do not have to worry about how it can be stored,\n> > transported, defined per submodule or on a branch, since answers to this\n> > are given at the commandline (and current repository state).\n> >\n> > There are still open questions about this though:\n> >\n> > * Should the name in the submodule be 'origin' even though you\n> > specified --with-remote=somewhere? For me its always confusing to\n> > have the same/similar remotes named differently in different\n> > repositories. That why I try to keep the names the same in all my\n> > clones of repositories (i.e. for my private, github, upstream\n> > remotes).\n> >\n> > * When you do a 'git submodule sync --with-remote=somewhere' should\n> > the remote be added or replaced.\n> >\n> > My opinion on these are:\n> >\n> > The remote should be named as in the superproject so\n> > --with-remote=somewhere adds/replaces the remote 'somewhere' in the\n> > submodules named on the commandline (or all in case no submodule is\n> > specified). In case of a fresh clone of the submodule, there would be no\n> > origin but only a remote under the new name.\n> >\n> > Would the --with-remote feature I describe be a feasible start for you\n> > Robert? What do others think? Is the naming of the parameter\n> > '--with-remote' alright?\n> >\n> > Cheers Heiko\n> >\n> > [1] http://article.gmane.org/gmane.comp.version-control.git/255512\n> > [2] http://article.gmane.org/gmane.comp.version-control.git/255512\n> > [3] https://github.com/jlehmann/git-submod-enhancements/wiki#special-ref-overriding-gitmodules-values\n> \n> Hi Heiko,\n> \n> My last email response was in violation of your request to keep the\n> two topics separate, sorry about that. I started typing it this\n> weekend and completed the draft this morning, without having read this\n> response from you first.\n\nThats fine, no problem.\n\n> Here is what I think would make the feature most usable. I think you\n> went over some of these ideas but I just want to clarify, to make sure\n> we're on the same page. Please correct me as needed.\n> \n> 1. Running `git submodule update --with-remote ` shall fail the\n> command unconditionally.\n\nI am not sure but I think you mean\n\n\tgit submodule update --with-remote=\n\nWith the equals sign, without it you would name the submodule paths to\nupdate. No I think that should just add the remote to all\nsubmodules that would be updated and do the normal update operation on\nthem (with the new remote of course).\n\n> 2. Using the `--with-remote` option on submodule `update` or `sync`\n> will fail if it detects absolute submodule URLs in .gitmodule\n\nYes, almost. Since you can have a mixture I suggest to only fail if the\nsubmodules that would be processed have an absolute url in them. If\nprocessed submodules are all relative it can go ahead.\n\n> 3. Running `git submodule update --init --with-remote ` shall\n> fail the command ONLY if a submodule is being processed that is NOT\n> also being initialized.\n\nNo since the --init flag just tells update to initialize submodules\non-demand. It should just go ahead the same way as without\n--with-remote.\n\n> 4. The behavior of git submodule's `update` or `sync` commands\n> combined with `--with-remote` will REPLACE or CREATE the 'origin'\n> remote in each submodule it is run in. We will not allow the user to\n> configure what the submodule remote name will end up being (I think\n> this is current behavior and forces good practice; I consider `origin`\n> an adopted standard for git, and actually wish it was more enforced\n> for super projects as well!)\n\nNo please carefully read my email again. I specifically was describing\nthe opposite. --with-remote= creates/replaces the remote in\nthe submodule. I do not see a benefit in restricting the user from\ncreating different remote names in the submodule. I think it would be\nmore confusing if the remote 'origin' in the superproject does not point\nto the same location as 'origin' in the submodule.\n\n> Let me know if I've missed anything. Once we clarify requirements I'll\n> attempt to start work on this during my free time. I'll start by\n> testing this through msysgit, since I do not have linux installed, but\n> I have Linux Mint running in a Virtual Machine so I can test on both\n> platforms as needed (I don't have a lot of experience on Linux\n> though).\n\nI think it does not matter which development environment you use. In my\nexperience though Linux is around 30x faster when it comes to the\ntypical operations you do when developing git. Especially for running\nthe testsuite that makes a difference between a few hours and minutes.\n\n> I hope you won't mind me reaching out for questions as needed, however\n> I will attempt to be as resourceful as possible since I know you're\n> all busy. Thanks.\n\nNo problem, just post here and we will see.\n\n\nOn Mon, Aug 25, 2014 at 09:32:27AM -0500, Robert Dailey wrote:\n> Thought of a few more:\n> \n> 5. If `--with-remote` is unspecified, behavior will continue as it\n> currently does (I'm not clear on the precedence here of various\n> options, but I assume: `remote.default` first, then\n> `branch.name.remote`)\n\nYes. And I hope that is ensured enough through the testsuite for this\ncase. So run it to ensure this. Have a look what kind of tests exist and\nmaybe even write one or two for the code you change. Thats a good start\nfor practise and also makes sure you do no break existing behavior.\n\nJohan Herland also recently collected some update tests here[1]\n\nAFAIK, remote.default was WIP and does not exist yet. So you only need\nto worry \n\n> 6. `--with-remote` will take precedence over `remote.default` and\n> `branch.name.remote`.\n\nYes.\n\n> I'll add more as I think of them... Sorry for the spam.\n\nI think the code for the new commandline switch will not be too\ncomplicated/big so I think its best if you just go ahead, write it and\nthen send a patch to the list once you are happy. Its common to add a\nRFC if you just want some comments on your current status and do not\nthink its ready for inclusion yet. Expect it to go a few rounds until\neverything is ironed out.\n\nCheers Heiko\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/246312\n"},{"id":"248309","messageId":"CAHd499CC3XxE5X3OPobK9yNKqypCLP5pe1Q7cFyW_wWne6pnnQ@mail.gmail.com","threadId":"37371","inReplyTo":"20140826062847.GA12216@book.hvoigt.net","subject":"Re: Re: Re: Relative submodule URLs","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2014-08-26T15:18:48Z","receivedAt":"2014-08-26T15:18:48Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Tue, Aug 26, 2014 at 1:28 AM, Heiko Voigt wrote:\n>> Hi Heiko,\n>>\n>> My last email response was in violation of your request to keep the\n>> two topics separate, sorry about that. I started typing it this\n>> weekend and completed the draft this morning, without having read this\n>> response from you first.\n>\n> Thats fine, no problem.\n>\n>> Here is what I think would make the feature most usable. I think you\n>> went over some of these ideas but I just want to clarify, to make sure\n>> we're on the same page. Please correct me as needed.\n>>\n>> 1. Running `git submodule update --with-remote ` shall fail the\n>> command unconditionally.\n>\n> I am not sure but I think you mean\n>\n> git submodule update --with-remote=\n>\n> With the equals sign, without it you would name the submodule paths to\n> update. No I think that should just add the remote to all\n> submodules that would be updated and do the normal update operation on\n> them (with the new remote of course).\n\nI'm not sure about Linux but at least with msysgit on Windows, typing\na two-dash option (such as --with-remote) forces command line\nevaluation to use the next placement parameter as the parameter for\nit. I've seen this work the same way with argparse in python too. In\nmy experience, command line has worked that way, I'm not sure if that\nis by design or not though. I never use equal signs with git commands,\nnever had a problem for some reason.\n\nFor example:\n\ngit rebase --onto release/1.0 head~3 head\n\nThe `--onto` option knows to use `release/1.0` as its parameter.\n\n>> 2. Using the `--with-remote` option on submodule `update` or `sync`\n>> will fail if it detects absolute submodule URLs in .gitmodule\n>\n> Yes, almost. Since you can have a mixture I suggest to only fail if the\n> submodules that would be processed have an absolute url in them. If\n> processed submodules are all relative it can go ahead.\n\nFor example if it processes 3 submodules in the following order:\n\n1. relative\n2. absolute\n3. relative\n\nShould it fail before or after processing the 3rd relative submodule?\nI was thinking it would fail while trying to sync/update the 2nd one\n(which is absolute) and stop before processing the 3rd.\n\n>> 3. Running `git submodule update --init --with-remote ` shall\n>> fail the command ONLY if a submodule is being processed that is NOT\n>> also being initialized.\n>\n> No since the --init flag just tells update to initialize submodules\n> on-demand. It should just go ahead the same way as without\n> --with-remote.\n\nBut doesn't the on-demand initialization need to evaluate relative\nURLs and convert them to absolute based on the .gitmodules\nconfiguration? I thought the idea was to make `--with-remote` invalid\nfor initialization/sync of absolute URLs.\n\nIn other words if I did:\n\ngit submodule init --with-remote fork my-submodule-dir\n\nand if my-submodule-dir was not relative in .gitmodules, then the\n`--with-remote` flag becomes useless. We could fail silently but for\neducational purposes to the user I thought we were failing in these\nscenarios. Maybe I misunderstood your original intent with the\nfailures? Is init not doing the relative to absolute evaluation like\nI'm thinking? Please correct me where I'm wrong.\n\n>> 4. The behavior of git submodule's `update` or `sync` commands\n>> combined with `--with-remote` will REPLACE or CREATE the 'origin'\n>> remote in each submodule it is run in. We will not allow the user to\n>> configure what the submodule remote name will end up being (I think\n>> this is current behavior and forces good practice; I consider `origin`\n>> an adopted standard for git, and actually wish it was more enforced\n>> for super projects as well!)\n>\n> No please carefully read my email again. I specifically was describing\n> the opposite. --with-remote= creates/replaces the remote in\n> the submodule. I do not see a benefit in restricting the user from\n> creating different remote names in the submodule. I think it would be\n> more confusing if the remote 'origin' in the superproject does not point\n> to the same location as 'origin' in the submodule.\n\nWell the reason why I said it would be 'origin' is so that the\nsubmodule knows which remote to use internally during an update. I'm\nassuming 'update' uses 'origin' internally in the submodule to know\nwhich remote to pull from. My understanding of how `git submodule\nupdate` knows which URL to pull from is probably incorrect. I'm not\nfamiliar on the internal mechanics of how this works. Perhaps you\ncould explain or send me to some reading material on it?\n"},{"id":"248345","messageId":"20140826203434.GA13156@book.hvoigt.net","threadId":"37371","inReplyTo":"CAHd499CC3XxE5X3OPobK9yNKqypCLP5pe1Q7cFyW_wWne6pnnQ@mail.gmail.com","subject":"Re: Re: Re: Re: Relative submodule URLs","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-08-26T20:34:34Z","receivedAt":"2014-08-26T20:34:34Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Tue, Aug 26, 2014 at 10:18:48AM -0500, Robert Dailey wrote:\n> On Tue, Aug 26, 2014 at 1:28 AM, Heiko Voigt wrote:\n> >> My last email response was in violation of your request to keep the\n> >> two topics separate, sorry about that. I started typing it this\n> >> weekend and completed the draft this morning, without having read this\n> >> response from you first.\n> >\n> > Thats fine, no problem.\n> >\n> >> Here is what I think would make the feature most usable. I think you\n> >> went over some of these ideas but I just want to clarify, to make sure\n> >> we're on the same page. Please correct me as needed.\n> >>\n> >> 1. Running `git submodule update --with-remote ` shall fail the\n> >> command unconditionally.\n> >\n> > I am not sure but I think you mean\n> >\n> > git submodule update --with-remote=\n> >\n> > With the equals sign, without it you would name the submodule paths to\n> > update. No I think that should just add the remote to all\n> > submodules that would be updated and do the normal update operation on\n> > them (with the new remote of course).\n> \n> I'm not sure about Linux but at least with msysgit on Windows, typing\n> a two-dash option (such as --with-remote) forces command line\n> evaluation to use the next placement parameter as the parameter for\n> it. I've seen this work the same way with argparse in python too. In\n> my experience, command line has worked that way, I'm not sure if that\n> is by design or not though. I never use equal signs with git commands,\n> never had a problem for some reason.\n>\n> For example:\n> \n> git rebase --onto release/1.0 head~3 head\n> \n> The `--onto` option knows to use `release/1.0` as its parameter.\n\nIf you are on Window or Linux does not make a difference here. I just\nrealized we are quite inconsistent:\n\n$ git grep -E -e \"--\\w+=<\\w+>\" -- Documentation/ | wc -l\n 226\n\n$ git grep -E -e \"--\\w+ <\\w+>\" -- Documentation/ | wc -l\n 75\n\nI would prefer the first though since that one is used more often. But\nwe can leave that for later, once we have some code to talk about.\n\n> >> 2. Using the `--with-remote` option on submodule `update` or `sync`\n> >> will fail if it detects absolute submodule URLs in .gitmodule\n> >\n> > Yes, almost. Since you can have a mixture I suggest to only fail if the\n> > submodules that would be processed have an absolute url in them. If\n> > processed submodules are all relative it can go ahead.\n> \n> For example if it processes 3 submodules in the following order:\n> \n> 1. relative\n> 2. absolute\n> 3. relative\n> \n> Should it fail before or after processing the 3rd relative submodule?\n> I was thinking it would fail while trying to sync/update the 2nd one\n> (which is absolute) and stop before processing the 3rd.\n\nFor consistency I would prefer if it fails right from the beginning in\nthis situation since the command can not be completed.\n\n> >> 3. Running `git submodule update --init --with-remote ` shall\n> >> fail the command ONLY if a submodule is being processed that is NOT\n> >> also being initialized.\n> >\n> > No since the --init flag just tells update to initialize submodules\n> > on-demand. It should just go ahead the same way as without\n> > --with-remote.\n> \n> But doesn't the on-demand initialization need to evaluate relative\n> URLs and convert them to absolute based on the .gitmodules\n> configuration? I thought the idea was to make `--with-remote` invalid\n> for initialization/sync of absolute URLs.\n> \n> In other words if I did:\n> \n> git submodule init --with-remote fork my-submodule-dir\n> \n> and if my-submodule-dir was not relative in .gitmodules, then the\n> `--with-remote` flag becomes useless. We could fail silently but for\n> educational purposes to the user I thought we were failing in these\n> scenarios. Maybe I misunderstood your original intent with the\n> failures? Is init not doing the relative to absolute evaluation like\n> I'm thinking? Please correct me where I'm wrong.\n\nYes it does the relative to absolute evaluation. But that is a different\ntopic. For absolute urls in .gitmodules it should fail, but you were\ntalking about --init in general and in general that should not fail IMO.\nSo e.g.\n\n\tgit submodule update --init --with-remote=\n\nwhen all submodule urls are relative in .gitmodules and some submodules\nhave already been initialized should succeed.\n\n> >> 4. The behavior of git submodule's `update` or `sync` commands\n> >> combined with `--with-remote` will REPLACE or CREATE the 'origin'\n> >> remote in each submodule it is run in. We will not allow the user to\n> >> configure what the submodule remote name will end up being (I think\n> >> this is current behavior and forces good practice; I consider `origin`\n> >> an adopted standard for git, and actually wish it was more enforced\n> >> for super projects as well!)\n> >\n> > No please carefully read my email again. I specifically was describing\n> > the opposite. --with-remote= creates/replaces the remote in\n> > the submodule. I do not see a benefit in restricting the user from\n> > creating different remote names in the submodule. I think it would be\n> > more confusing if the remote 'origin' in the superproject does not point\n> > to the same location as 'origin' in the submodule.\n> \n> Well the reason why I said it would be 'origin' is so that the\n> submodule knows which remote to use internally during an update. I'm\n> assuming 'update' uses 'origin' internally in the submodule to know\n> which remote to pull from. My understanding of how `git submodule\n> update` knows which URL to pull from is probably incorrect. I'm not\n> familiar on the internal mechanics of how this works. Perhaps you\n> could explain or send me to some reading material on it?\n\nYes your assumptions are almost true. Except that submodule update does\nnot do a pull but a fetch (without any arguments) by default. But your\nimplementation could change (I actually first thought that was already\nthe case) the fetch in submodule update to fetch from all remotes before\nupdating if there is no remote specified. But I have not thought that\nthrough. Additionally the implementation of --with-remote could be used\nto specify from which remote to fetch. Regarding the reading I can only\nsuggest the code of git-submodule.sh to you.\n\nI can understand that altering the 'origin' remote to also save the\nremote for future fetches would help you in your case but we have to\nkeep other workflow in mind and not all people (me included) want only\none remote in their submodule. --with-remote should also help those\npeople which it does when it adds a remote under that name. Changing the\ndefault from which a submodule is fetched by submodule update is a\nseparate topic for the additional configuration which we split from this\ntopic. I think.\n\nCheers Heiko\n"},{"id":"248506","messageId":"53FF6A72.50605@xiplink.com","threadId":"37371","inReplyTo":"53F76907.1090904@xiplink.com","subject":"Re: Relative submodule URLs","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2014-08-28T17:44:18Z","receivedAt":"2014-08-28T17:44:18Z","isPatch":false,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"Sorry for dropping out of the conversation; the last few days were a bit hectic.\n\nRegarding recursive branching, I agree that a super-repo's branch names are\nnot necessarily appropriate for its submodules, and that Heiko's \"simple\nworkflow\" is a workable base to build upon. More thought is needed here, but\nthat's for another day.\n\nRegarding remote.default, Robert please understand that the feature doesn't\nexist, and the idea is to only serve as a fallback when the current methods\nfor remote selection end up resorting to the hardcoded \"origin\" name. More\nthought is also needed here, but not today.\n\nBoth Heiko and Robert took issue with this statement of mine:\n\nOn 14-08-22 12:00 PM, Marc Branchaud wrote:\n> A branch should fork the entire repo, including its submodules. The\n> implication is that if you want to push that branch somewhere, that\n> somewhere needs to be able to accept the forks of the submodules *even\n> if those submodules aren't changed in your branch* because at the very\n> least the branch ref has to exist in the submodules' repositories.\n\nHeiko said: \"It should be easy to work on a repository that is forked in its\nentirety, but it should also be possible (and properly supported) to only\nfork some submodules.\"\n\nYou're right, I overstated it when I said that the branch ref has to exist in\nthe unchanged submodules. The super-repo branch records which submodules it\nupdates, and when pushing the branch somewhere only those submodules' changes\nneed to be pushed.\n\nRobert asked: \"How will this impact *creating* branches? What about forking?\nDo you expect submodule forking & branching to be automatic as well? ... This\nseems difficult to do, especially the forking part since you would need an\nAPI for this (Github, Atlassian Stash, etc), unless you are thinking of\nsomething clever like local/relative forks.\"\n\nI meant \"fork\" in the local-branch sense: The branch represents a topic in\nthe repository, and it should encompass the entire repository including its\nsubmodules (just like the branch encompasses all the files in the repository,\neven though the branch's commits only change a subset of those files). I\nthink you're talking about \"fork\" in the sense of setting up a mirror of a\nrepository. I agree that there aren't really any tools for automatically\ndoing that with repositories that contain relative-path submodules. I think\n\"git clone\" could learn to do it, though.\n\n\nHeiko also said this:\n> On Fri, Aug 22, 2014 at 12:00:07PM -0400, Marc Branchaud wrote:\n>> With relative-path submodules, the push's target repo *must* also have the\n>> submodules in their proper places, so that they can get updated.\n>> Furthermore, if you clone a repo that has relative-path submodules you\n>> *must* also clone the submodules.\n>\n> That is not true. You can have relative submodules and just clone/fetch\n> some from a different remote. Its just a question of how to\n> specifiy/transport this information.\n\nI meant that more as a general guideline than some kind of physical law.\nSure, it's possible to scatter the submodules across all sorts of hosts, but\nit's not a good idea. When it comes to relative-path submodules, pushing and\nfetching submodule changes in the super-repo should just involve the one\nremote host (whatever way that's determined). This keeps things tractable,\nbecause otherwise your branch's changes are scattered among many different\nhosts and you end up considering weird things like \"this part of the branch's\nchanges are on host A but this other part are on host B, so let's record that\nsomewhere, oh but what if host B is down when I'm trying to fetch, but I know\nthat host C has the changes too so why don't I just fetch what I want from\nthere\".\n\nIt's a nightmare. It's infinitely better to treat a repository and its\nrelative-path submodules as an atomic unit, so that any remote that hosts the\nrepository also hosts the submodules. When pushing a branch with submodule\nchanges, expect to find those submodules on the target remote and update\nthem. Regardless of how the target remote is determined. Same thing for\nfetching. It's just so much simpler to work this way.\n\nSo please, let's not try to specify submodule remotes per-branch or make that\ninfo pushable. It's enough for a branch's local configuration to say that it\ntracks fetch/pull refs on different remotes. The rest should flow from that.\n\n\t\tM.\n"},{"id":"248515","messageId":"20140828193533.GA20040@book.hvoigt.net","threadId":"37371","inReplyTo":"53FF6A72.50605@xiplink.com","subject":"Re: Re: Relative submodule URLs","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-08-28T19:35:33Z","receivedAt":"2014-08-28T19:35:33Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Thu, Aug 28, 2014 at 01:44:18PM -0400, Marc Branchaud wrote:\n> Heiko also said this:\n> > On Fri, Aug 22, 2014 at 12:00:07PM -0400, Marc Branchaud wrote:\n> >> With relative-path submodules, the push's target repo *must* also have the\n> >> submodules in their proper places, so that they can get updated.\n> >> Furthermore, if you clone a repo that has relative-path submodules you\n> >> *must* also clone the submodules.\n> >\n> > That is not true. You can have relative submodules and just clone/fetch\n> > some from a different remote. Its just a question of how to\n> > specifiy/transport this information.\n> \n> I meant that more as a general guideline than some kind of physical law.\n> Sure, it's possible to scatter the submodules across all sorts of hosts, but\n> it's not a good idea. When it comes to relative-path submodules, pushing and\n> fetching submodule changes in the super-repo should just involve the one\n> remote host (whatever way that's determined). This keeps things tractable,\n> because otherwise your branch's changes are scattered among many different\n> hosts and you end up considering weird things like \"this part of the branch's\n> changes are on host A but this other part are on host B, so let's record that\n> somewhere, oh but what if host B is down when I'm trying to fetch, but I know\n> that host C has the changes too so why don't I just fetch what I want from\n> there\".\n> \n> It's a nightmare. It's infinitely better to treat a repository and its\n> relative-path submodules as an atomic unit, so that any remote that hosts the\n> repository also hosts the submodules. When pushing a branch with submodule\n> changes, expect to find those submodules on the target remote and update\n> them. Regardless of how the target remote is determined. Same thing for\n> fetching. It's just so much simpler to work this way.\n\nYou are right, its simpler. But I would not say \"better\". Depending on\nyour project it might be \"better\" to just fork some submodules.\n\n> So please, let's not try to specify submodule remotes per-branch or make that\n> info pushable. It's enough for a branch's local configuration to say that it\n> tracks fetch/pull refs on different remotes. The rest should flow from that.\n\nWhy not? Git is all about flexibility. Of course if you organise your\nsubmodules in chaos you will get chaos. But consider this:\n\nYou have this big project which consists of submodule (e.g. like Android\nwith hundreds of submodules). Now you want to develop on something that\ninvolves just a subset of submodules, lets say two submodules.\n\nNow if someone just wants to publish a small change to some submodules\nyou are demanding to setup a mirror of *all* submodules that are in this\nbig project. That might not even be feasible depending on the projects\nsize and the remote quota. Not to speak about having to first create a\nfork of hundreds of repositories. So in this situation we should support\njust referring some submodules to other places.\n\nRegarding transporting this information. If you ask someone to try out\nyour change it should be as simple as possible. It should be enough to\nsay. clone from there and checkout that branch (once recursive checkout\nand fetch for submodules is in place). So here we need a way to\ntransport this configuration for a fork.\n\nYes for a small project where its feasible to simply clone all\nsubmodules you can just say: please fork everything. But for bigger\nprojects thats not necessarily an option. So we should at least give the\nusers that option. Then its a matter of policy how you work with a\nproject.\n\nI am not saying that everything for this should be implemented in the\nfirst steps but we should keep it in mind and design everything in such\na way that it is still possible to implement such a kind of workflow\nlater.\n\nCheers Heiko\n"},{"id":"248544","messageId":"540097A0.9040003@xiplink.com","threadId":"37371","inReplyTo":"20140828193533.GA20040@book.hvoigt.net","subject":"Re: Relative submodule URLs","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2014-08-29T15:09:20Z","receivedAt":"2014-08-29T15:09:20Z","isPatch":false,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 14-08-28 03:35 PM, Heiko Voigt wrote:\n> On Thu, Aug 28, 2014 at 01:44:18PM -0400, Marc Branchaud wrote:\n>> Heiko also said this:\n>>> On Fri, Aug 22, 2014 at 12:00:07PM -0400, Marc Branchaud wrote:\n>>>> With relative-path submodules, the push's target repo *must* also have the\n>>>> submodules in their proper places, so that they can get updated.\n>>>> Furthermore, if you clone a repo that has relative-path submodules you\n>>>> *must* also clone the submodules.\n>>>\n>>> That is not true. You can have relative submodules and just clone/fetch\n>>> some from a different remote. Its just a question of how to\n>>> specifiy/transport this information.\n>>\n>> I meant that more as a general guideline than some kind of physical law.\n>> Sure, it's possible to scatter the submodules across all sorts of hosts, but\n>> it's not a good idea. When it comes to relative-path submodules, pushing and\n>> fetching submodule changes in the super-repo should just involve the one\n>> remote host (whatever way that's determined). This keeps things tractable,\n>> because otherwise your branch's changes are scattered among many different\n>> hosts and you end up considering weird things like \"this part of the branch's\n>> changes are on host A but this other part are on host B, so let's record that\n>> somewhere, oh but what if host B is down when I'm trying to fetch, but I know\n>> that host C has the changes too so why don't I just fetch what I want from\n>> there\".\n>>\n>> It's a nightmare. It's infinitely better to treat a repository and its\n>> relative-path submodules as an atomic unit, so that any remote that hosts the\n>> repository also hosts the submodules. When pushing a branch with submodule\n>> changes, expect to find those submodules on the target remote and update\n>> them. Regardless of how the target remote is determined. Same thing for\n>> fetching. It's just so much simpler to work this way.\n> \n> You are right, its simpler. But I would not say \"better\". Depending on\n> your project it might be \"better\" to just fork some submodules.\n\nI think we need a clear definition of \"fork\" here. Are you concerned that\nthere's are copies of the submodule repositories that are \"unused\" in the\nbranch? (Indeed, yes, you are, as I see below.)\n\n>> So please, let's not try to specify submodule remotes per-branch or make that\n>> info pushable. It's enough for a branch's local configuration to say that it\n>> tracks fetch/pull refs on different remotes. The rest should flow from that.\n> \n> Why not? Git is all about flexibility. Of course if you organise your\n> submodules in chaos you will get chaos. But consider this:\n> \n> You have this big project which consists of submodule (e.g. like Android\n> with hundreds of submodules). Now you want to develop on something that\n> involves just a subset of submodules, lets say two submodules.\n> \n> Now if someone just wants to publish a small change to some submodules\n> you are demanding to setup a mirror of *all* submodules that are in this\n> big project. That might not even be feasible depending on the projects\n> size and the remote quota. Not to speak about having to first create a\n> fork of hundreds of repositories. So in this situation we should support\n> just referring some submodules to other places.\n\nI feel that this scenario is something of a straw-man. At the very least,\nthe developer already has a clone of all the submodules. Disk space is cheap.\n\n(If the developer doesn't need all the submodules then I suggest that the\nsuper-project is badly organized and should use intermediate submodules to\nmake it easier for developers to focus on isolated areas. That being said, I\ncan appreciate that repository hygiene is more art than science, and that a\nlarge and/or long-lived project could end up with some pretty funky\nconfigurations.)\n\n> Regarding transporting this information. If you ask someone to try out\n> your change it should be as simple as possible. It should be enough to\n> say. clone from there and checkout that branch (once recursive checkout\n> and fetch for submodules is in place). So here we need a way to\n> transport this configuration for a fork.\n\nYou're assuming that the super-project is organized in such a way that\nsubmodule-reliant code changes can live in isolation from the rest of the\nproject. That's a bit like saying you can try out a change in gitk without\nhaving the rest of git. The super-project exists as a complete thing, and I\ndon't believe there are many projects where it would make sense to only try\nout a change in isolation. I'm not familiar with the Android project, but\nI'd be mighty impressed if changes to any arbitrary subset of its submodules\ncould be thoroughly tested without a full Android system.\n\nSo I don't believe the scenario you're suggesting is at all simple. The\nperson trying out the change can't just \"clone from there\" because the\nsubmodules uanffected by the branch aren't there. At the very least this\nperson needs to start with \"origin\" clones of the super-project and all of\nits required submodules, not just the ones changed in the branch. Then the\nperson can add the \"fork\" host as a remote and fetch the branch.\n\nBut it's still not that simple. Because now you're expecting that the branch\nsomehow has information that overrides some submodules' URLs stored in\n.git/config. Coding that might be easy, I don't know, but as you say the\noverride information needs to be stored somewhere transportable and\nbranchable, like maybe a .gitmodules-fork file or something. Because\nobviously different branches will have different submodule overrides.\n\nAnd that makes it even more complicated! If the remote-overriding\ninformation is stored as part of the branch then in fact that branch can't\njust be merged and pushed to the \"origin\" host, because the submodules there\nmust not have their remotes overridden. So now the branch has to be changed\nin order to remove the overrides. Users have to remember to do that, or\nthey'll break the origin's submodules. But when the branch changes suddenly\nwhatever people reviewed in the \"fork\" isn't what gets pushed back to the\n\"origin\".\n\n> Yes for a small project where its feasible to simply clone all\n> submodules you can just say: please fork everything. But for bigger\n> projects thats not necessarily an option. So we should at least give the\n> users that option. Then its a matter of policy how you work with a\n> project.\n\nOK, but even if we want to eventually do both perhaps it would be wiser to\nstart with the simple fork-everything model. Maybe just teach \"clone\n--mirror\" to also create relative-path submodules in their proper locations,\nso that forks become easier.\n\n> I am not saying that everything for this should be implemented in the\n> first steps but we should keep it in mind and design everything in such\n> a way that it is still possible to implement such a kind of workflow\n> later.\n\nI agree with using an incremental approach, but it's important to understand\nwhere we want to go before suggesting a first step. I'm just trying to think\nthrough the implications of what's been suggested. Please set me straight if\nI'm not thinking about this properly.\n\n\t\tM.\n"}]}