{"thread":{"id":"58018","subject":"Plumbing for mapping from a remote tracking ref to the remote ref?","startedAt":"2022-06-15T19:12:35Z","lastAt":"2023-09-06T04:21:57Z","messageCount":7,"participants":["Tao Klerks","Junio C Hamano","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"457306","messageId":"CAPMMpogUxq59zj+=7UDiURYbydAwvymOqhEWaheT9fkU8HaP4Q@mail.gmail.com","threadId":"58018","inReplyTo":null,"subject":"Plumbing for mapping from a remote tracking ref to the remote ref?","fromName":"Tao Klerks","fromEmail":"tao@klerks.biz","sentAt":"2022-06-15T19:12:18Z","receivedAt":"2022-06-15T19:12:35Z","isPatch":false,"sender":{"key":"tao@klerks.biz","avatar":"https://avatars.githubusercontent.com/u/531704?v=4"},"body":"Hi folks,\n\nGiven the following configured fetch refspec for a remote:\n\n[remote \"origin\"]\n        url = git@someserver:somerepo.git\n        fetch = +refs/heads/*:refs/remotes/somepath/*\n\nAnd given a ref of the form \"refs/remotes/somepath/branch_A\",\n\nI'm wondering whether there is any plumbing that would be able to tell\nme what to put in a \"fetch\" command, to get\n\"refs/remotes/somepath/branch_A\" fetched - in other words, is there\nany plumbing that can use the configured fetch refspecs to map\n\"refs/remotes/somepath/branch_A\" to \"refs/heads/branch_A\" for me, so\nthat I can then do \"git fetch origin refs/heads/branch_A\".\n\nI understand I can parse the fetch refspecs myself, assuming any\nasterisk is only ever on the tail end of the ref pattern... but that\nseems very complicated, given that this is *probably* something others\nhave needed to do in the past?\n\nFwiw I've noticed that \"git rev-parse --symbolic-full-name\" knows how\nto do almost exactly the *opposite* of that, when presented with the\n\"@{u}\" pattern - it looks up the \"branch.XXX.merge\" value, which is\nwritten in a remote-relative form (\"refs/heads/branch_A\" in this\nexample), and converts that to the \"local\" fetch destination (eg\n\"refs/remotes/somepath/branch_A\"). But I don't know how to go the\nopposite way, given only a local fetch destination and wanting to tell\nfetch what to get - it expects a remote-relative ref.\n\nAny help appreciated!\n\nThanks,\nTao\n"},{"id":"457312","messageId":"xmqqilp1znn1.fsf@gitster.g","threadId":"58018","inReplyTo":"CAPMMpogUxq59zj+=7UDiURYbydAwvymOqhEWaheT9fkU8HaP4Q@mail.gmail.com","subject":"Re: Plumbing for mapping from a remote tracking ref to the remote ref?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-15T20:18:26Z","receivedAt":"2022-06-15T20:18:42Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Tao Klerks <tao@klerks.biz> writes:\n\n> Given the following configured fetch refspec for a remote:\n>\n> [remote \"origin\"]\n>         url = git@someserver:somerepo.git\n>         fetch = +refs/heads/*:refs/remotes/somepath/*\n>\n> And given a ref of the form \"refs/remotes/somepath/branch_A\",\n>\n> I'm wondering whether there is any plumbing that would be able to tell\n> me what to put in a \"fetch\" command, to get\n> \"refs/remotes/somepath/branch_A\" fetched - in other words, is there\n> any plumbing that can use the configured fetch refspecs to map\n> \"refs/remotes/somepath/branch_A\" to \"refs/heads/branch_A\" for me, so\n> that I can then do \"git fetch origin refs/heads/branch_A\".\n\nI am fairly certain that I never have written one myself ;-)\n\nI wonder how the end-user experience should look like.\n\n\t$ git refmap refs/remotes/somepath/branch-A\n\torigin refs/heads/branch-A\n\n\t$ git refmap refs/remotes/somepath/{branch-A,branch-B}\n\torigin refs/heads/branch-A\n\torigin refs/heads/branch-B\n\nIOW, you give name(s) of remote-tracking branches and then you get\nthe remote and their ref for these?\n\nI do not oppose to such a command existing, but I do not know what\nthe right answer should be for a case like this:\n\n\t[remote \"origin\"]\n\t\turl = ... the official project repository ...\n\t\tfetch = +refs/heads/*:refs/remotes/upstream/*\n\n\t[remote \"mirror\"]\n\t\turl = ... a local mirror you'd use regularly ...\n\t\tfetch = +refs/heads/*:refs/remotes/upstream/*\n\nIn order to support such a \"more than one can update the same\" case\nsensibly, the output may have to repeat the input, e.g.\n\n\t$ git refmap refs/remotes/upstream/main\n\trefs/remotes/upstream/main\torigin refs/heads/main\n\trefs/remotes/upstream/main\tmirror refs/heads/main\n\nperhaps?\n\n"},{"id":"457506","messageId":"nycvar.QRO.7.76.6.2206182358350.349@tvgsbejvaqbjf.bet","threadId":"58018","inReplyTo":"xmqqilp1znn1.fsf@gitster.g","subject":"Re: Plumbing for mapping from a remote tracking ref to the remote ref?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-06-18T22:04:08Z","receivedAt":"2022-06-18T22:04:35Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Wed, 15 Jun 2022, Junio C Hamano wrote:\n\n> Tao Klerks <tao@klerks.biz> writes:\n>\n> > Given the following configured fetch refspec for a remote:\n> >\n> > [remote \"origin\"]\n> >         url = git@someserver:somerepo.git\n> >         fetch = +refs/heads/*:refs/remotes/somepath/*\n> >\n> > And given a ref of the form \"refs/remotes/somepath/branch_A\",\n> >\n> > I'm wondering whether there is any plumbing that would be able to tell\n> > me what to put in a \"fetch\" command, to get\n> > \"refs/remotes/somepath/branch_A\" fetched - in other words, is there\n> > any plumbing that can use the configured fetch refspecs to map\n> > \"refs/remotes/somepath/branch_A\" to \"refs/heads/branch_A\" for me, so\n> > that I can then do \"git fetch origin refs/heads/branch_A\".\n>\n> I am fairly certain that I never have written one myself ;-)\n\nI looked for something like that, but did not find it. We seem to have the\nfunctions `apply_refspecs()`, `query_refspecs()` and\n`remote_find_tracking()` that could be used to that end, but I do not see\nany of them being used in plumbing that would expose the ref mapping in\nthe desired way.\n\n> I wonder how the end-user experience should look like.\n>\n> \t$ git refmap refs/remotes/somepath/branch-A\n> \torigin refs/heads/branch-A\n>\n> \t$ git refmap refs/remotes/somepath/{branch-A,branch-B}\n> \torigin refs/heads/branch-A\n> \torigin refs/heads/branch-B\n>\n> IOW, you give name(s) of remote-tracking branches and then you get\n> the remote and their ref for these?\n\nModulo introducing a new top-level command (a subcommand of `git remote`\nwould make much more sense and make the feature eminently more\ndiscoverable), and modulo allowing patterns in the ref to match, I agree.\n\n> I do not oppose to such a command existing, but I do not know what\n> the right answer should be for a case like this:\n>\n> \t[remote \"origin\"]\n> \t\turl = ... the official project repository ...\n> \t\tfetch = +refs/heads/*:refs/remotes/upstream/*\n>\n> \t[remote \"mirror\"]\n> \t\turl = ... a local mirror you'd use regularly ...\n> \t\tfetch = +refs/heads/*:refs/remotes/upstream/*\n>\n> In order to support such a \"more than one can update the same\" case\n> sensibly, the output may have to repeat the input, e.g.\n>\n> \t$ git refmap refs/remotes/upstream/main\n> \trefs/remotes/upstream/main\torigin refs/heads/main\n> \trefs/remotes/upstream/main\tmirror refs/heads/main\n>\n> perhaps?\n\nSounds like a good plan to me.\n\nCiao,\nDscho\n"},{"id":"457508","messageId":"xmqqczf5lgk3.fsf@gitster.g","threadId":"58018","inReplyTo":"nycvar.QRO.7.76.6.2206182358350.349@tvgsbejvaqbjf.bet","subject":"Re: Plumbing for mapping from a remote tracking ref to the remote ref?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-18T23:04:12Z","receivedAt":"2022-06-18T23:04:23Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n>> \t$ git refmap refs/remotes/somepath/{branch-A,branch-B}\n>> \torigin refs/heads/branch-A\n>> \torigin refs/heads/branch-B\n>>\n>> IOW, you give name(s) of remote-tracking branches and then you get\n>> the remote and their ref for these?\n>\n> Modulo introducing a new top-level command (a subcommand of `git remote`\n> would make much more sense and make the feature eminently more\n> discoverable), and modulo allowing patterns in the ref to match, I agree.\n\n\"git remote\" is primarily about \"I have this remote---tell me more\nabout it\", but this query goes in the other direction, and that is\nwhy I threw a non-existing command to solicit alternatives that are\npotentially better than \"git remote\".\n\nFWIW, I did not have any opinion on where the feature should appear\nor what the syntax to query the remote.<nick>.fetch refmap should\nbe, when I wrote the above.  I still do not (yet) have a strong\nopinion.\n\nI do not oppose to the \"find remotes I can fetch from to update\nthese remote-tracking branches\" feature existing.  I just wanted to\nmake sure whoever will work on it is aware of the fact that they\nmust consider the case with overlapping destinations.\n\nThanks.\n"},{"id":"481347","messageId":"CAPMMpojUrfSmpgWVh3TTn_uamPCcyHRQf2R3APSpEjsqujNXvA@mail.gmail.com","threadId":"58018","inReplyTo":"xmqqczf5lgk3.fsf@gitster.g","subject":"Re: Plumbing for mapping from a remote tracking ref to the remote ref?","fromName":"Tao Klerks","fromEmail":"tao@klerks.biz","sentAt":"2023-09-03T07:16:00Z","receivedAt":"2023-09-03T07:16:17Z","isPatch":false,"sender":{"key":"tao@klerks.biz","avatar":"https://avatars.githubusercontent.com/u/531704?v=4"},"body":"On Sun, Jun 19, 2022 at 1:04 AM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n>\n> >>      $ git refmap refs/remotes/somepath/{branch-A,branch-B}\n> >>      origin refs/heads/branch-A\n> >>      origin refs/heads/branch-B\n> >>\n> >> IOW, you give name(s) of remote-tracking branches and then you get\n> >> the remote and their ref for these?\n> >\n> > Modulo introducing a new top-level command (a subcommand of `git remote`\n> > would make much more sense and make the feature eminently more\n> > discoverable), and modulo allowing patterns in the ref to match, I agree.\n>\n> \"git remote\" is primarily about \"I have this remote---tell me more\n> about it\", but this query goes in the other direction, and that is\n> why I threw a non-existing command to solicit alternatives that are\n> potentially better than \"git remote\".\n\nThank you for the responses here, and my apologies for not following\nup (much) earlier.\n\nGiven that \"git remote\" already deals with different types of args\n(remotes, URLs, remote branches), could it make sense to introduce a\ndedicated new subcommand, not directly related to \"set-branches\", eg\n\"map-refs\"? I agree with Dscho that keeping it under \"git remote\"\nwould help with discoverability and avoid clutter in the global\nnamespace: Git already has many top-level commands, the \"theme\" under\nwhich this one fits is definitely \"remote stuff\", and \"git remote\"\nalready does a number of substantially-different things all related\nwith *remote configuration*.\n\nIn fact that's another way of seeing things: most of \"git remote\"'s\ncurrent subcommands are just syntactic sugar over \"git config\" (the\ntwo that operate outside of the config are \"set-head\" and \"update\"),\nand this new one would be config-focused in exactly the same way.\n\nTo Junio's question along the lines of \"what if someone mapped\nmultiple remote namespaces to a single 'tracking namespace' location\nin the local repo?\" (which I hope is rare - I seem to recall there are\nat least some operations that warn when this is detected), this\nambiguity would be absent in a \"git remote\" subcommand, as it would\ntake a remote name.\n\nI had also considered some new weird \"@{remotemapping}\"-style syntax\nto rev-parse, but here precisely there *would* be no implicit remote\ncontext, and so getting more than one answer would be an option, and\nthat doesn't make sense for rev-parse.\n\nRegarding patterns and wildcards, for *my* purpose at least, they\ndon't make much sense: The whole purpose of the exercise is to say \"I\nknow the ref I want updated in my repo, I know what remote that it is\nmapped to or that I want to update it from, I want to know exactly\nwhat to put in a \"git fetch <remote_name> <remote_ref>...\" call, to\nget that ref updated correctly/consistently for the current repo,\nwithout affecting any other refs that this repo has mapped for that\nremote.\n\nWould something like the following be mutually agreeable?\n\n       $ git remote origin map-ref\nrefs/remotes/my-favorite-remotes/origin/someref\n      refs/heads/someref\n\n       $ git remote origin map-ref\nrefs/remotes/my-favorite-remotes/origin/someref\nrefs/remotes/my-favorite-remotes/origin/someotherref\n      refs/heads/someref\n      refs/heads/someotherref\n\n     $ git remote origin map-ref refs/remotes/someotherpath/someref\n      error: the ref \"refs/remotes/someotherpath/someref\" is not\nmapped in any configured refspec of remote \"origin\".\n\n\nThanks,\nTao\n"},{"id":"481416","messageId":"xmqqpm2wqn6h.fsf@gitster.g","threadId":"58018","inReplyTo":"CAPMMpojUrfSmpgWVh3TTn_uamPCcyHRQf2R3APSpEjsqujNXvA@mail.gmail.com","subject":"Re: Plumbing for mapping from a remote tracking ref to the remote ref?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-09-05T22:18:14Z","receivedAt":"2023-09-05T22:18:21Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Tao Klerks <tao@klerks.biz> writes:\n\n> On Sun, Jun 19, 2022 at 1:04 AM Junio C Hamano <gitster@pobox.com> wrote:\n>>\n>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n>>\n>> >>      $ git refmap refs/remotes/somepath/{branch-A,branch-B}\n>> >>      origin refs/heads/branch-A\n>> >>      origin refs/heads/branch-B\n>> >>\n>> >> IOW, you give name(s) of remote-tracking branches and then you get\n>> >> the remote and their ref for these?\n>> >\n>> > Modulo introducing a new top-level command (a subcommand of `git remote`\n>> > would make much more sense and make the feature eminently more\n>> > discoverable), and modulo allowing patterns in the ref to match, I agree.\n>>\n>> \"git remote\" is primarily about \"I have this remote---tell me more\n>> about it\", but this query goes in the other direction, and that is\n>> why I threw a non-existing command to solicit alternatives that are\n>> potentially better than \"git remote\".\n>\n> Thank you for the responses here, and my apologies for not following\n> up (much) earlier.\n> ...\n> Would something like the following be mutually agreeable?\n>\n>        $ git remote origin map-ref\n> refs/remotes/my-favorite-remotes/origin/someref\n>       refs/heads/someref\n\nWith strainge line wrapping, I cannot quite tell what is the input\nand what is the output, but if you meant that the part up to the\nlong-ish refname in the refs/remotes is the command line, and map-ref\nis the new subcommand name in the \"git remote\" command, i.e.\n\n    $ git remote map-ref origin refs/remotes/my-favorite-remotes/origin/someref\n\nis the input, to which the output \n\n    refs/heads/someref\n\nis given, I am not sure what its value is.  First of all, the user\nis giving a ref in a hierarchy that is usually used for the remote\nwhose name is \"my-favorite-remotes\".  What made this user _know_\nthat the remote reference belongs to \"origin\"?  Isn't that part of\nwhat the user may want to _find_ _out_, instead of required to give\nas input?\n\nSo, no, I do not think it is agreeable at least not from this end,\nbut I may be misunderstanding what you meant to present as your\ndesign.\n\nI would understand if it were more like\n\n    $ git remote refmap refs/remotes/somepath/{branch-A,branch-B}\n    origin:refs/heads/branch-A refs/remotes/somepath/branch-A\n    origin:refs/heads/branch-B refs/remotes/somepath/branch-B\n\nthat is,\n\n (1) the new subcommand (refmap) takes one or more refs from the\n     command line; they typically are in the refs/remotes hiearchy\n     and each asks which remote's which ref needs to be fetched to\n     update the ref.  Note that the user does *not* need to know\n     which remote the refs will be updated from.\n\n (2) the subcommand goes through the \"remote.*.fetch\" configuration\n     items (and its older equivalents in .git/remotes, whose support\n     should come free if you used remote.c API properly) to find\n     what ref from what remote is fetched to update the refs given\n     from the command line.\n\n (3) the output is \"<remote>:<ref-at-remote> <our-remote-tracking-branch>\"\n     one line at a time.\n\nNote that this format allows the \"two remotes can both update the\nsame remote tracking branches we have\" arrangement.\n\n"},{"id":"481425","messageId":"CAPMMpoh7=riapMO-91e81MrK-uR+sm7ttgCu8433dNNQU0ZsQw@mail.gmail.com","threadId":"58018","inReplyTo":"xmqqpm2wqn6h.fsf@gitster.g","subject":"Re: Plumbing for mapping from a remote tracking ref to the remote ref?","fromName":"Tao Klerks","fromEmail":"tao@klerks.biz","sentAt":"2023-09-06T04:21:45Z","receivedAt":"2023-09-06T04:21:57Z","isPatch":false,"sender":{"key":"tao@klerks.biz","avatar":"https://avatars.githubusercontent.com/u/531704?v=4"},"body":"On Wed, Sep 6, 2023 at 12:18 AM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Tao Klerks <tao@klerks.biz> writes:\n>\n> > ...\n> > Would something like the following be mutually agreeable?\n> >\n> >        $ git remote origin map-ref\n> > refs/remotes/my-favorite-remotes/origin/someref\n> >       refs/heads/someref\n>\n> With strainge line wrapping, I cannot quite tell what is the input\n> and what is the output, but if you meant that the part up to the\n> long-ish refname in the refs/remotes is the command line, and map-ref\n> is the new subcommand name in the \"git remote\" command, i.e.\n>\n>     $ git remote map-ref origin refs/remotes/my-favorite-remotes/origin/someref\n>\n> is the input, to which the output\n>\n>     refs/heads/someref\n>\n> is given,\n\nMy apologies: in addition to automatic line wrapping, I also got the\narg order wrong.\n\nYes, this is what I meant.\n\n> I am not sure what its value is.  First of all, the user\n> is giving a ref in a hierarchy that is usually used for the remote\n> whose name is \"my-favorite-remotes\".  What made this user _know_\n> that the remote reference belongs to \"origin\"?\n\nUnderstanding that it's dangerous to make assumptions about what's\ntypical, I am positing that the user typically knows what remote\nthey're working with / looking for stuff in. I would guess that the\nset of repos, in the world, that have multiple remotes with different\nref path structures, mapped onto the same remote tracking ref\nnamespace, is much smaller than the set of repos that have some set of\nrefs mapped differently to the standard\n\"refs/heads/*:refs/remotes/originname/*\" mapping. My \"selfish\" intent\nwas to address the latter, without worrying much about the former.\n\n> Isn't that part of\n> what the user may want to _find_ _out_, instead of required to give\n> as input?\n\nThere's certainly value in enabling them to do so!\n\n>\n> So, no, I do not think it is agreeable at least not from this end,\n> but I may be misunderstanding what you meant to present as your\n> design.\n\nNo misunderstanding. I was unfortunately more concerned with \"fitting\nin\" with other \"git remote\" subcommands (which take a remote name)\nthan making the most useful functionality.\n\n>\n> I would understand if it were more like\n>\n>     $ git remote refmap refs/remotes/somepath/{branch-A,branch-B}\n>     origin:refs/heads/branch-A refs/remotes/somepath/branch-A\n>     origin:refs/heads/branch-B refs/remotes/somepath/branch-B\n>\n\nThis is a much better proposal, from my perspective!\n\n> that is,\n>\n>  (1) the new subcommand (refmap) takes one or more refs from the\n>      command line; they typically are in the refs/remotes hiearchy\n>      and each asks which remote's which ref needs to be fetched to\n>      update the ref.  Note that the user does *not* need to know\n>      which remote the refs will be updated from.\n>\n>  (2) the subcommand goes through the \"remote.*.fetch\" configuration\n>      items (and its older equivalents in .git/remotes, whose support\n>      should come free if you used remote.c API properly) to find\n>      what ref from what remote is fetched to update the refs given\n>      from the command line.\n>\n>  (3) the output is \"<remote>:<ref-at-remote> <our-remote-tracking-branch>\"\n>      one line at a time.\n>\n> Note that this format allows the \"two remotes can both update the\n> same remote tracking branches we have\" arrangement.\n>\n"}]}