{"thread":{"id":"40072","subject":"Re: enhanced remote ref namespaces","startedAt":"2015-08-12T18:34:50Z","lastAt":"2015-08-12T22:52:58Z","messageCount":5,"participants":["Jacob Keller","Junio C Hamano","Johan Herland"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"267930","messageId":"CA+P7+xocd+LE2A+srH0p1qTuXKRXanTp5E+imC1GE+9-biqR6A@mail.gmail.com","threadId":"40072","inReplyTo":null,"subject":"Re: enhanced remote ref namespaces","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2015-08-12T18:34:50Z","receivedAt":"2015-08-12T18:34:50Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Wed, Aug 12, 2015 at 9:10 AM, Junio C Hamano wrote:\n> Jacob Keller writes:\n>\n>> Recently there was some discussion about git-notes and how we do not\n>> fetch notes from remotes by default. The big problem with doing so is\n>> because refs/remotes/* hierarchy is only setup for branches (heads),\n>> so we don't have any clean location to put them.\n>\n> I wouldn't call this topic \"proper\" namespaces, though. What we\n> have is widely used and it shouldn't be broken. Call it \"enhanced\",\n> perhaps.\n>\n\nOk.\n\n> Some design boundaries:\n>\n> - Moving the remote-tracking branch hierarchy from refs/remotes/$O/*\n> to refs/remotes/$O/heads/* would not fly, because it will break\n> existing repositories. Do not even waste time on pursuing\n> refs/remotes/$O/{heads,tags,notes...}/*\n>\n\neven if we maintained new git's abililty to work with this, ie: only\nexternal-to-git scripts would break and only for new clones? Maybe we\ndon't want to go this route, but it seems like the way that the\noriginal proposal was headed.\n\n> - Extending the current refs/remotes/$O/* (for branches) and doing\n> refs/remote-tags/$O/* (for tags) may work, would not break\n> existing repositories, and could to be extended further to cover\n> refs/remote-notes/$O and friends. It however may not look pretty\n> (weak objection) and more importantly, it would make it harder to\n> \"cull\" things that came from a single remote.\n>\n> Just thinking aloud, perhaps we can introduce a brand new top level\n> hierarchy refs/remote/$O/{heads,tags,notes,...}, and give backward\n> compatibility by making a moral equivalent of a symbolic link from\n> refs/remote/$O/heads to refs/remotes/$O/. The true remote-tracknig\n> refs continue to live in refs/remotes/$O/* and old tools that do not\n> know the new layout would not be hurt. New tools that want to\n> ignore and look only at refs/remote/$O/* can do so through the moral\n> equivalent of a symbolic link. \"remote remove\" needs to be taught\n> about this single exception (i.e. \"the symbolic link\"), but the\n> places it needs to touch is limited only to two places and will not\n> grow.\n>\n\n\nI think this proposal makes the sense.. I'd go with something like:\n\nrefs/tracking//(heads|tags|notes|replace|etc)/*\n\nwith a symlink from or into\n\nrefs/tracking//heads -> refs/remotes/\n\nI prefer tracking vs remote because \"remote\" and \"remotes\" are too\nsimilar for my taste. tracking I think gets the idea across pretty\nstraight forward. Using a symlink personally I'd symlink the\nrefs/tracking into refs/remotes rather than make refs/remotes the real\nstorage.\n\n\n> If somebody got confused, notice that in the above description, I\n> said refs/remotes/ and refs/remote/. The former must stay. The\n> name of the latter is open to bikeshedding. Some may prefer a name\n> that is more distinct (refs/tracking/ or something, perhaps?). I\n> happen to prefer a name that is similar, but this preference is very\n> weak and I can persuaded to go either way.\n\nI don't like it being so similar that we now have typos between\nremotes and remote.. ie: remotes/ works for heads, but\n\"remotes//tags\" does not... that sounds like it would get\nconfusing.\n\nSymlinking the old location seems reasonable to me, as it would leave\nall the same data in the locations expected by the old tools, while\nkeeping all actual storage in the new location.\n\nIn this way, we would no longer need configuration settings. It\nhonestly doesn't matter to me which direction we symlink either.\n\nAs for the other complex issue is what to do about \"refs/tracking//tags\n\nThe big discussion on the original thread is about how tags would\nwork. I'm personally ok with *ignoring* tags and leaving it the way it\nis for now, and just doing this as a solid place to stick\nnotes/replace/etc.\n\nOr, we could go the route of continuing to stick tags into \"refs/tags\"\nat the same time as also sticking them into\nrefs/tracking//tags\n\nOr.. we could go the full route of fixing up lookup of tags such that\nwe put tags in refs/tracking//tags and we have it lookup tags\nthere via something like:\n\n1) local tags preferred\n\n2) any remote tag as long as all remote tags point to the same commit\nobject (how we select which to use is not very relevant here... we\ncould actually go with as long as the tag object is the same so two\nremotes with annotated tags pointing to the same object but different\ntag id would be ambiguous as well)\n\n3) warn the user must disambiguate the tag via the remote name.\n\nWe could also teach fetch to warn about remote tags which are\nambiguous (ie: two remotes with same named tag objects pointing to\ndifferent things)\n\nHow goes this sound? I think it makes sense... I don't know how to do\nall this without breaking some backwards compatibility though... I\nthink we can maintain expectations for the general user but I feel\nthat any change here will break *someones* scripts.\n\nRegards,\nJake\n"},{"id":"267931","messageId":"xmqqegj8l4lw.fsf@gitster.dls.corp.google.com","threadId":"40072","inReplyTo":"CA+P7+xocd+LE2A+srH0p1qTuXKRXanTp5E+imC1GE+9-biqR6A@mail.gmail.com","subject":"Re: enhanced remote ref namespaces","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-08-12T18:54:35Z","receivedAt":"2015-08-12T18:54:35Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jacob Keller writes:\n\n>> Just thinking aloud, perhaps we can introduce a brand new top level\n>> hierarchy refs/remote/$O/{heads,tags,notes,...}, and give backward\n>> compatibility by making a moral equivalent of a symbolic link from\n>> refs/remote/$O/heads to refs/remotes/$O/. The true remote-tracknig\n>> refs continue to live in refs/remotes/$O/* and old tools that do not\n>> know the new layout would not be hurt. New tools that want to\n>> ignore and look only at refs/remote/$O/* can do so through the moral\n>> equivalent of a symbolic link. \"remote remove\" needs to be taught\n>> about this single exception (i.e. \"the symbolic link\"), but the\n>> places it needs to touch is limited only to two places and will not\n>> grow.\n>\n> I think this proposal makes the sense.. I'd go with something like:\n>\n> refs/tracking//(heads|tags|notes|replace|etc)/*\n>\n> with a symlink from or into\n>\n> refs/tracking//heads -> refs/remotes/\n\nI actually do not think we even need the \"symlink\" thing.\n\nWe can just say refs/remotes/$O has been and forever will be where\nthe remote-tracking branches will live. With or without the symlink\nfor backward compatibility, the updated Git will need to be aware of\nthe two places to deal with older and newer repositories anyway.\n\n\"everything is now under refs/tracking/$O, not spread over many\nunbounded number of places like refs/remotes-$foo/$O\" may appear to\nbe cleaner but two is already not too bad and will greatly reduce\nthe transition pain.\n"},{"id":"267932","messageId":"CA+P7+xq_v+TjqSDC3aR2bdUmCJyj=zLXSE5SbVgOEr+SojOUQQ@mail.gmail.com","threadId":"40072","inReplyTo":"xmqqegj8l4lw.fsf@gitster.dls.corp.google.com","subject":"Re: enhanced remote ref namespaces","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2015-08-12T19:02:07Z","receivedAt":"2015-08-12T19:02:07Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Wed, Aug 12, 2015 at 11:54 AM, Junio C Hamano wrote:\n> Jacob Keller writes:\n>\n>>> Just thinking aloud, perhaps we can introduce a brand new top level\n>>> hierarchy refs/remote/$O/{heads,tags,notes,...}, and give backward\n>>> compatibility by making a moral equivalent of a symbolic link from\n>>> refs/remote/$O/heads to refs/remotes/$O/. The true remote-tracknig\n>>> refs continue to live in refs/remotes/$O/* and old tools that do not\n>>> know the new layout would not be hurt. New tools that want to\n>>> ignore and look only at refs/remote/$O/* can do so through the moral\n>>> equivalent of a symbolic link. \"remote remove\" needs to be taught\n>>> about this single exception (i.e. \"the symbolic link\"), but the\n>>> places it needs to touch is limited only to two places and will not\n>>> grow.\n>>\n>> I think this proposal makes the sense.. I'd go with something like:\n>>\n>> refs/tracking//(heads|tags|notes|replace|etc)/*\n>>\n>> with a symlink from or into\n>>\n>> refs/tracking//heads -> refs/remotes/\n>\n> I actually do not think we even need the \"symlink\" thing.\n>\n> We can just say refs/remotes/$O has been and forever will be where\n> the remote-tracking branches will live. With or without the symlink\n> for backward compatibility, the updated Git will need to be aware of\n> the two places to deal with older and newer repositories anyway.\n>\n> \"everything is now under refs/tracking/$O, not spread over many\n> unbounded number of places like refs/remotes-$foo/$O\" may appear to\n> be cleaner but two is already not too bad and will greatly reduce\n> the transition pain.\n\nOk, so.... just have branches be in refs/remotes/$O and all new things\nbe in refs/tracking/$O/category\n\nthat sounds reasonable enough to me.\n\nThat still hasn't really resolved the question of how to deal with\ntags, but it does solve the question of how to deal with replace and\nnotes refs.\n\nRegards,\nJake\n"},{"id":"267942","messageId":"xmqqwpx0jnhc.fsf@gitster.dls.corp.google.com","threadId":"40072","inReplyTo":"CA+P7+xq_v+TjqSDC3aR2bdUmCJyj=zLXSE5SbVgOEr+SojOUQQ@mail.gmail.com","subject":"Re: enhanced remote ref namespaces","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-08-12T19:49:51Z","receivedAt":"2015-08-12T19:49:51Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jacob Keller writes:\n\n> That still hasn't really resolved the question of how to deal with\n> tags, but it does solve the question of how to deal with replace and\n> notes refs.\n\nI do not think it would be a good change to add a\n\n [remote \"foo\"]\n fetch = refs/tags/*:refs/tracking/foo/tags/*\n\nas people do expect to see tags from upstream to appear in their\nlocal tag namespace when cloning from the central meeting point of\ntheir project (e.g. Linus's kernel repository).\n\nI'm willing to believe those who argued that they have both private\ntags and public tags in their repository, but I think that would be\nsomething better supported by adding \"git tag --local foo\" that\ncreates a local tag in e.g. refs/local-tags/foo, and by teaching\nDWIMmery to also look in there when the user says \"foo\".\n\nAlternatively, they can use their local branch namespace, which is\nalready private to them.\n"},{"id":"267963","messageId":"CALKQrgdVD6HX9_wKUwQn4VK8Rd=SDNAtCJccSZqh6r5vQ35QcA@mail.gmail.com","threadId":"40072","inReplyTo":"CA+P7+xocd+LE2A+srH0p1qTuXKRXanTp5E+imC1GE+9-biqR6A@mail.gmail.com","subject":"Re: enhanced remote ref namespaces","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2015-08-12T22:52:58Z","receivedAt":"2015-08-12T22:52:58Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"On Wed, Aug 12, 2015 at 8:34 PM, Jacob Keller wrote:\n> On Wed, Aug 12, 2015 at 9:10 AM, Junio C Hamano wrote:\n>> Some design boundaries:\n>>\n>> - Moving the remote-tracking branch hierarchy from refs/remotes/$O/*\n>> to refs/remotes/$O/heads/* would not fly, because it will break\n>> existing repositories. Do not even waste time on pursuing\n>> refs/remotes/$O/{heads,tags,notes...}/*\n>\n> even if we maintained new git's abililty to work with this, ie: only\n> external-to-git scripts would break and only for new clones? Maybe we\n> don't want to go this route, but it seems like the way that the\n> original proposal was headed.\n\nI don't think it's worth trying to go that route. Even though it's\ntheoretically possible to solve it, the complexity added by e.g.\nmultiple git versions operating on the same repository (maybe even\nsimultaneously?) suggests that it's much simpler to just go with a new\nclean hierarchy that simply Works for new Git versions, and the older\nversions don't have to deal with at all. IMHO, maybe even going as far\nas incrementing core.repositoryFormatVersion, so that older Git\nversions will refuse to work with new-style repos...\n\n[...]\n>> If somebody got confused, notice that in the above description, I\n>> said refs/remotes/ and refs/remote/. The former must stay. The\n>> name of the latter is open to bikeshedding. Some may prefer a name\n>> that is more distinct (refs/tracking/ or something, perhaps?). I\n>> happen to prefer a name that is similar, but this preference is very\n>> weak and I can persuaded to go either way.\n>\n> I don't like it being so similar that we now have typos between\n> remotes and remote.. ie: remotes/ works for heads, but\n> \"remotes//tags\" does not... that sounds like it would get\n> confusing.\n\nI like refs/tracking. At some point I was planning to use refs/peers,\nbut that's merely another color for the bikeshed...\n\n> Symlinking the old location seems reasonable to me, as it would leave\n> all the same data in the locations expected by the old tools, while\n> keeping all actual storage in the new location.\n>\n> In this way, we would no longer need configuration settings. It\n> honestly doesn't matter to me which direction we symlink either.\n>\n> As for the other complex issue is what to do about \"refs/tracking//tags\n>\n> The big discussion on the original thread is about how tags would\n> work. I'm personally ok with *ignoring* tags and leaving it the way it\n> is for now, and just doing this as a solid place to stick\n> notes/replace/etc.\n\nThat is probably the best plan for now: Solve most of the problem, and\npunt on the controversial parts.\n\n> Or, we could go the route of continuing to stick tags into \"refs/tags\"\n> at the same time as also sticking them into\n> refs/tracking//tags\n\nI don't like the copying approach; makes it harder to deduce which tags\nare (truly) local, and which came from a remote.\n\n> Or.. we could go the full route of fixing up lookup of tags such that\n> we put tags in refs/tracking//tags and we have it lookup tags\n> there via something like:\n>\n> 1) local tags preferred\n>\n> 2) any remote tag as long as all remote tags point to the same commit\n> object (how we select which to use is not very relevant here... we\n> could actually go with as long as the tag object is the same so two\n> remotes with annotated tags pointing to the same object but different\n> tag id would be ambiguous as well)\n>\n> 3) warn the user must disambiguate the tag via the remote name.\n\nAs was probably obvious from the old threads, this is where I'd like\nto go. Eventually.\n\n> We could also teach fetch to warn about remote tags which are\n> ambiguous (ie: two remotes with same named tag objects pointing to\n> different things)\n\nAgreed. This would actually be a good improvement Right Now, without\nchanging any ref layout at all: If get fetch would otherwise copy a\nremote tag into refs/tags/*, but doesn't because there is already a\n_different_ tag in its place, then warn loudly to the user, so that\nuser has a chance to discover the tag difference.\n\n> How goes this sound? I think it makes sense... I don't know how to do\n> all this without breaking some backwards compatibility though... I\n> think we can maintain expectations for the general user but I feel\n> that any change here will break *someones* scripts.\n\nAs I said above: Punt on tags for now, and you might be able to not\nbreak anyone's scripts (and if you do, it's probably a poorly written\nscript). Provided that you leave a symlink to/from refs/remotes/$O in\nplace, you're AFAICS only adding functionality, not (visibly) changing\nexisting behavior.\n\n\nBTW, thanks for resurrecting this topic!\n\n...Johan\n\n-- \nJohan Herland, \nwww.herland.net\n"}]}