{"thread":{"id":"40066","subject":"proper remote ref namespaces","startedAt":"2015-08-12T06:43:56Z","lastAt":"2015-08-12T19:35:47Z","messageCount":4,"participants":["Jacob Keller","Marc Branchaud","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"267895","messageId":"CA+P7+xpj+8DZ=K0pna299Mu3nsQ4+JV_JUK=WFzzAFnJN+Bkbg@mail.gmail.com","threadId":"40066","inReplyTo":null,"subject":"proper remote ref namespaces","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2015-08-12T06:43:56Z","receivedAt":"2015-08-12T06:43:56Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"Hello,\n\nRecently there was some discussion about git-notes and how we do not\nfetch notes from remotes by default. The big problem with doing so is\nbecause refs/remotes/* hierarchy is only setup for branches (heads),\nso we don't have any clean location to put them.\n\nAround the time of git 1.8.0, Johan Herland made a proposal for\nremotes to put all their refs in refs/remtoes/*, by moving heads into\nrefs/remotes/heads/* [1]\n\nIn addition, his proposal was to include remote tags into\nrefs/remotes//tags and also refs/remotes//replace and\nnotes similarly.\n\nDuring this discussion there was many people who liked the idea, and\nothers who rejected it. The main rejection reason was two fold:\n\n(a) tags are \"global\" per project, so their namespace should be\ntreated global as it is now.\n\nThe proposal's counter to this is that tags aren't guaranteed to be\nglobal, because today two remotes you fetch might have tags that are\nthe same name with different pointers. This is currently hidden, and\ngit silently picks the tag it fetched first.\n\n(b) script compatibility, as changing the ref layout such that new\ngit can't work with old repository would be bad\n\nthe counter to this, is that we make git smart enough to recognize old\nremote format, and continue to work with it. Scripts which depend on\nthis layout will break, but that may not be such a huge concern.\n\nPersonally, I think this proposal at least for heads, notes, replace,\nand other remote refs we'd like to pull is very useful. I don't\nrightly know the answer for tags. The linked discussion below covers\nseveral pages of back and forth between a few people about which\nmethod is best.\n\nI like the idea of simplifying tags and branches and notes and others\nto all fetch the same way. local stuff is in refs/heads or refs/notes\nand remote stuff is (by default) in refs/remotes//tags etc\n\nBut it does bring up some discussion as today we \"auto follow\" tags\ninto refs/tags, and it can get weird for tags since now \"ambiguous\"\ntags must mean if there are tags of same name which point to different\nrefs, and we'd need to teach a bunch of logic to the ref lookup code.\n\nI am looking at ways to help git-notes be easier to use, so that we by\ndefault fetch notes, and enable easier merge, since we'd have default\nlocations to merge from and to. This would make the sharing of notes a\nlot easier, which is one of their primary sticking points.. you can't\nreally share them without *everyone* working to do it the same way you\ndo. By making a default policy, sharing becomes natural, and users can\neasily add *public* notes to commits for things such as bug ids and\nother things which are not discovered until after the commit is\ncreated.\n\nIn addition, the easy ability to share replaces might also be helpful,\nthough IMHO not as valuable as git-notes.\n\nI think that the only logical refs layout is\n\"refs/remotes//(heads|tags|notes|replace)\"\n\nand adding \"refs/remote-notes\" and \"refs/remote-replace\" is not really\na clean solution.\n\nGiven that the 1.8.0 proposal mostly died, does anyone have any thoughts now?\n\nThe proposal suggested by Johan makes sense to me, and I believe we\ncan code up logic to make it easy for new git to keep logic of the old\nlayout.\n\nPersonally, I think the best solution is to only store that layout for\na given clone, using a config option that defaults to false, where\nnew-git sets it to true for all clones. Then, provide a command to\nrenew remotes-layout that does this if the user wishes. Thus, clones\nfor the old style will be handled, and new clones would have the new\nlayout. (ie: no mixing layouts in a single repository).\n\nI'm really not sure if this is the best solution, but seems like the\ncleanest solution.\n\nRegards,\nJake\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/165799/focus=165885\n"},{"id":"267912","messageId":"55CB63C5.8090708@xiplink.com","threadId":"40066","inReplyTo":"CA+P7+xpj+8DZ=K0pna299Mu3nsQ4+JV_JUK=WFzzAFnJN+Bkbg@mail.gmail.com","subject":"Re: proper remote ref namespaces","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2015-08-12T15:18:29Z","receivedAt":"2015-08-12T15:18:29Z","isPatch":false,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 15-08-12 02:43 AM, Jacob Keller wrote:\n> Hello,\n> \n> Recently there was some discussion about git-notes and how we do not fetch\n> notes from remotes by default. The big problem with doing so is because\n> refs/remotes/* hierarchy is only setup for branches (heads), so we don't\n> have any clean location to put them.\n> \n> Around the time of git 1.8.0, Johan Herland made a proposal for remotes to\n> put all their refs in refs/remtoes/*, by moving heads into \n> refs/remotes/heads/* [1]\n\nThanks for resurrecting this discussion. I feel this is a fundamental\ninconsistency in git's core design. Not a fatal flaw by any means, but\nsomething that keeps git from reaching its full potential.\n\n> In addition, his proposal was to include remote tags into \n> refs/remotes//tags and also refs/remotes//replace and \n> notes similarly.\n> \n> During this discussion there was many people who liked the idea, and \n> others who rejected it. The main rejection reason was two fold:\n> \n> (a) tags are \"global\" per project, so their namespace should be treated\n> global as it is now.\n> \n> The proposal's counter to this is that tags aren't guaranteed to be \n> global, because today two remotes you fetch might have tags that are the\n> same name with different pointers. This is currently hidden, and git\n> silently picks the tag it fetched first.\n> \n> (b) script compatibility, as changing the ref layout such that new git\n> can't work with old repository would be bad\n> \n> the counter to this, is that we make git smart enough to recognize old \n> remote format, and continue to work with it. Scripts which depend on this\n> layout will break, but that may not be such a huge concern.\n> \n> Personally, I think this proposal at least for heads, notes, replace, and\n> other remote refs we'd like to pull is very useful. I don't rightly know\n> the answer for tags. The linked discussion below covers several pages of\n> back and forth between a few people about which method is best.\n> \n> I like the idea of simplifying tags and branches and notes and others to\n> all fetch the same way. local stuff is in refs/heads or refs/notes and\n> remote stuff is (by default) in refs/remotes//tags etc\n> \n> But it does bring up some discussion as today we \"auto follow\" tags into\n> refs/tags, and it can get weird for tags since now \"ambiguous\" tags must\n> mean if there are tags of same name which point to different refs,\n\nThe tags would be disambiguated by their remote name, e.g. \"origin/tags/v5.6\"\nvs. \"hacker/tags/v5.6\". The change would actually simplify tag auto-following.\n\n> and we'd need to teach a bunch of logic to the ref lookup code.\n\nNot a lot. Existing DWIMery already handles ambiguous branches, by\npreferring a local branch name over any remote ones. The only teaching\nthat's really needed is to resolve shorthand like \"origin/v5.6\" to\n\"refs/remotes/origin/tags/v5.6\" (i.e. to look for anything matching\n\"refs/remotes/origin/*/v5.6\") but that doesn't seem very difficult.\n\nThere is the question of how the user can even know if there's ambiguity.\nAside from paying attention to \"git fetch\" output, I think we could extend\n\"git tag\" (and \"git branch\") in the case where the user specifies an existing\ntag (or branch). Right now both commands fail because the name already\nexists. All we need to do is extend that error message a bit, e.g.:\n\n\t> git tag v5.6\n\tfatal: tag 'next' already exists as\n\tv5.6 -> a3a0e5d67d554a685abd897bc3ce4ffa4e74c812\n\torigin/v5.6 remoteX/v5.6 -> 504346bbf9b02387b51f232f4db9c1860789b135\n\thacker/v5.6 -> fb4aa3533f81700789b3fb119e527410678e8d8d\n\nHere we see that our local v5.6, and hacker's v5.6, are unique, but origin\nand remoteX both have the same v5.6.\n\nWell, same-ish. I think those SHA IDs should be the things the tags point\nat, not the tag objects' IDs. This keeps things consistent between\nlightweight and annotated/signed tags. It does mean though that origin/v5.6\nand remoteX/v5.6 might be different tag objects that happen to point at the\nsame thing. I'm not sure the distinction is all that germane, and it's\nsomething that the user could disambiguate with \"git tag -v\" or \"git show\".\n\n> I am looking at ways to help git-notes be easier to use, so that we by \n> default fetch notes, and enable easier merge, since we'd have default \n> locations to merge from and to. This would make the sharing of notes a lot\n> easier, which is one of their primary sticking points.. you can't really\n> share them without *everyone* working to do it the same way you do. By\n> making a default policy, sharing becomes natural, and users can easily add\n> *public* notes to commits for things such as bug ids and other things\n> which are not discovered until after the commit is created.\n> \n> In addition, the easy ability to share replaces might also be helpful, \n> though IMHO not as valuable as git-notes.\n> \n> I think that the only logical refs layout is \n> \"refs/remotes//(heads|tags|notes|replace)\"\n> \n> and adding \"refs/remote-notes\" and \"refs/remote-replace\" is not really a\n> clean solution.\n> \n> Given that the 1.8.0 proposal mostly died, does anyone have any thoughts\n> now?\n> \n> The proposal suggested by Johan makes sense to me, and I believe we can\n> code up logic to make it easy for new git to keep logic of the old \n> layout.\n> \n> Personally, I think the best solution is to only store that layout for a\n> given clone, using a config option that defaults to false, where new-git\n> sets it to true for all clones. Then, provide a command to renew\n> remotes-layout that does this if the user wishes. Thus, clones for the old\n> style will be handled, and new clones would have the new layout. (ie: no\n> mixing layouts in a single repository).\n> \n> I'm really not sure if this is the best solution, but seems like the \n> cleanest solution.\n\nI think that proposal is good. Perhaps something like\nconfig.remoterefslayout, where 0 means the existing layout and 1 is the new\nlayout (with larger numbers reserved for future changes).\n\nImplementing the new layout should, I think, just involve changing the\ndefault fetch refspecs when adding a remote.\n\nThen it's just a matter of making the test suite pass. :)\n\n\t\tM.\n"},{"id":"267914","messageId":"xmqqfv3omqsh.fsf@gitster.dls.corp.google.com","threadId":"40066","inReplyTo":"CA+P7+xpj+8DZ=K0pna299Mu3nsQ4+JV_JUK=WFzzAFnJN+Bkbg@mail.gmail.com","subject":"Re: proper remote ref namespaces","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-08-12T16:10:06Z","receivedAt":"2015-08-12T16:10:06Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jacob Keller writes:\n\n> Recently there was some discussion about git-notes and how we do not\n> fetch notes from remotes by default. The big problem with doing so is\n> because refs/remotes/* hierarchy is only setup for branches (heads),\n> so we don't have any clean location to put them.\n\nI wouldn't call this topic \"proper\" namespaces, though. What we\nhave is widely used and it shouldn't be broken. Call it \"enhanced\",\nperhaps.\n\nSome design boundaries:\n\n - Moving the remote-tracking branch hierarchy from refs/remotes/$O/*\n to refs/remotes/$O/heads/* would not fly, because it will break\n existing repositories. Do not even waste time on pursuing\n refs/remotes/$O/{heads,tags,notes...}/*\n\n - Extending the current refs/remotes/$O/* (for branches) and doing\n refs/remote-tags/$O/* (for tags) may work, would not break\n existing repositories, and could to be extended further to cover\n refs/remote-notes/$O and friends. It however may not look pretty\n (weak objection) and more importantly, it would make it harder to\n \"cull\" things that came from a single remote.\n\nJust thinking aloud, perhaps we can introduce a brand new top level\nhierarchy refs/remote/$O/{heads,tags,notes,...}, and give backward\ncompatibility by making a moral equivalent of a symbolic link from\nrefs/remote/$O/heads to refs/remotes/$O/. The true remote-tracknig\nrefs continue to live in refs/remotes/$O/* and old tools that do not\nknow the new layout would not be hurt. New tools that want to\nignore and look only at refs/remote/$O/* can do so through the moral\nequivalent of a symbolic link. \"remote remove\" needs to be taught\nabout this single exception (i.e. \"the symbolic link\"), but the\nplaces it needs to touch is limited only to two places and will not\ngrow.\n\nIf somebody got confused, notice that in the above description, I\nsaid refs/remotes/ and refs/remote/. The former must stay. The\nname of the latter is open to bikeshedding. Some may prefer a name\nthat is more distinct (refs/tracking/ or something, perhaps?). I\nhappen to prefer a name that is similar, but this preference is very\nweak and I can persuaded to go either way.\n"},{"id":"267940","messageId":"xmqq1tf8l2p8.fsf@gitster.dls.corp.google.com","threadId":"40066","inReplyTo":"55CB63C5.8090708@xiplink.com","subject":"Re: proper remote ref namespaces","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-08-12T19:35:47Z","receivedAt":"2015-08-12T19:35:47Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marc Branchaud writes:\n\n> Not a lot. Existing DWIMery already handles ambiguous branches, by\n> preferring a local branch name over any remote ones. The only teaching\n> that's really needed is ...\n\nYou need to remember that there are five useful things you can do to\nmutable things.\n\n - Creation can be covered by teaching \"clone\" to put a new-style\n refspec but the same change needs to also go to \"remote add\".\n\n - Reading is done by the DWIMery in ref_rev_parse_rules[] (your\n point above).\n\n - Updating is automatic, as \"fetch\" does not have any funny\n built-in intuit and blindly follows configured fetch refspec.\n\n - Deletion by \"branch -d -r\" and \"remote remove\" needs to be\n careful about designing how the case where both old and new\n hierarchies exist should be handled (my gut feeling is \"delete\n both\", but there may be funny corner cases).\n\n - Enumeration by \"branch -l -r\" probably shares the same issue as\n deletion.\n"}]}