{"thread":{"id":"57974","subject":"Best way to update `HEAD` in mirrored repos","startedAt":"2022-06-08T20:36:16Z","lastAt":"2022-06-15T04:19:07Z","messageCount":9,"participants":["Rodrigo Silva Mendoza","Junio C Hamano","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"456893","messageId":"CANWRddN4R6AceeaOyZm1vs8AXBNv3J+cE5MOyrhKVhcqddjUOA@mail.gmail.com","threadId":"57974","inReplyTo":null,"subject":"Best way to update `HEAD` in mirrored repos","fromName":"Rodrigo Silva Mendoza","fromEmail":"rodrigosilvamendoza3@gmail.com","sentAt":"2022-06-08T20:36:02Z","receivedAt":"2022-06-08T20:36:16Z","isPatch":false,"sender":{"key":"rodrigosilvamendoza3@gmail.com","avatar":null},"body":"Hi all,\n\nGiven a repo cloned like so:\n`git clone --mirror someRemote`\n\nAnd then with its origin updated like so\n`git remote set-url origin someRemote`\n\nWhat would be the best way to update the `HEAD` ref in the clone, if\nsome change happens to the remote HEAD? Like say, the default branch\nchanging from `main` to `dev` or being renamed from `main` to\n`main_2`.\n\nWhat I've got that I think works\n1. Get the ref from the origin that points to `HEAD`. Extract the ref\nfrom the output.\n2. Manually update the `HEAD` ref with the extracted output prior step\n\nLike so:\n```\n$ git ls-remote --symref origin HEAD\nref: refs/heads/good_main_3 HEAD\n0666a519f94b8500ab6f14bdf7c9c2e5ca7d5821 HEAD\n\n$ git symbolic-ref HEAD refs/heads/good_main_3\n```\n\nDoes this make sense?\n\nThe following are some of the things I've tried - they all fail to\nupdate the `HEAD` file, or fail for some other reason.\n`git fetch -p`\n`git fetch -p -a`\n`get fetch -p -a -u`\n`git remote set-head origin -a`\n Fails with \"error: Not a valid ref:\nrefs/remotes/origin/good_main_3\". The ref it fails with is whatever\nthe remote HEAD is.\n`git remote update --prune`\n\nAnd my use case:\nI have a set of git repos cloned/mirrored. They are then indexed for\nsearch, using the `HEAD` ref. The indexer runs periodically, and\nbefore indexing it always runs `get fetch -p` to update repos. For\neach repo, to get the latest commit to index, I use the libgit2\nequivalent of `git rev-parse HEAD^0`. Right now, previously cloned\nrepos whose HEAD is updated after the initial clone will fail at this\nstep, which causes them to not get indexed. Here's an example run, all\nfor the same repo `x`.\n\n1. `git clone & set origin for repo x` (indexer first clones repo if\nit doesn't exist)\n2. `git rev-parse HEAD^0` - successfully outputs some commit sha\n3. Github default branch is renamed from `main` to `main_3`\n4. `git fetch -p` - indexer runs this to update\n5. `git rev-parse HEAD^0` - now outputs \"fatal: ambiguous argument\n'HEAD^0': unknown revision or path not in the working tree.\"\n\nThanks for your time,\n\nRodrigo\n"},{"id":"456904","messageId":"xmqq5ylapy0s.fsf@gitster.g","threadId":"57974","inReplyTo":"CANWRddN4R6AceeaOyZm1vs8AXBNv3J+cE5MOyrhKVhcqddjUOA@mail.gmail.com","subject":"Re: Best way to update `HEAD` in mirrored repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-08T22:53:55Z","receivedAt":"2022-06-08T22:54:08Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Rodrigo Silva Mendoza writes:\n\n> What I've got that I think works\n> 1. Get the ref from the origin that points to `HEAD`. Extract the ref\n> from the output.\n> 2. Manually update the `HEAD` ref with the extracted output prior step\n>\n> Like so:\n> ```\n> $ git ls-remote --symref origin HEAD\n> ref: refs/heads/good_main_3 HEAD\n> 0666a519f94b8500ab6f14bdf7c9c2e5ca7d5821 HEAD\n>\n> $ git symbolic-ref HEAD refs/heads/good_main_3\n> ```\n>\n> Does this make sense?\n\nYes. That soudns like the right thing to do.\n\n"},{"id":"456916","messageId":"CANWRddPDhM1g6rtu-a2a=EogXD_hOFwSDsgMCbVvB7dibMaEqw@mail.gmail.com","threadId":"57974","inReplyTo":"xmqq5ylapy0s.fsf@gitster.g","subject":"Re: Best way to update `HEAD` in mirrored repos","fromName":"Rodrigo Silva Mendoza","fromEmail":"rodrigosilvamendoza3@gmail.com","sentAt":"2022-06-09T05:01:40Z","receivedAt":"2022-06-09T05:01:56Z","isPatch":false,"sender":{"key":"rodrigosilvamendoza3@gmail.com","avatar":null},"body":"Great, thanks for the confirmation. Appreciate it!\n\nRodrigo\n\nOn Wed, Jun 8, 2022 at 3:54 PM Junio C Hamano wrote:\n>\n> Rodrigo Silva Mendoza writes:\n>\n> > What I've got that I think works\n> > 1. Get the ref from the origin that points to `HEAD`. Extract the ref\n> > from the output.\n> > 2. Manually update the `HEAD` ref with the extracted output prior step\n> >\n> > Like so:\n> > ```\n> > $ git ls-remote --symref origin HEAD\n> > ref: refs/heads/good_main_3 HEAD\n> > 0666a519f94b8500ab6f14bdf7c9c2e5ca7d5821 HEAD\n> >\n> > $ git symbolic-ref HEAD refs/heads/good_main_3\n> > ```\n> >\n> > Does this make sense?\n>\n> Yes. That soudns like the right thing to do.\n>\n"},{"id":"456924","messageId":"nycvar.QRO.7.76.6.2206091000590.349@tvgsbejvaqbjf.bet","threadId":"57974","inReplyTo":"CANWRddN4R6AceeaOyZm1vs8AXBNv3J+cE5MOyrhKVhcqddjUOA@mail.gmail.com","subject":"Re: Best way to update `HEAD` in mirrored repos","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-06-09T08:02:16Z","receivedAt":"2022-06-09T08:02:25Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Rodrigo,\n\nOn Wed, 8 Jun 2022, Rodrigo Silva Mendoza wrote:\n\n> `git remote set-head origin -a`\n> Fails with \"error: Not a valid ref:\n> refs/remotes/origin/good_main_3\". The ref it fails with is whatever\n> the remote HEAD is.\n\nThe reason is that `set-head` expects options to come before arguments,\nlike so:\n\n\tgit remote set-head -a origin\n\nCiao,\nJohannes\n"},{"id":"456942","messageId":"CANWRddNa1nB9shoppfXuA2yqmd2353HVBopoufkh0SevnVrngA@mail.gmail.com","threadId":"57974","inReplyTo":"nycvar.QRO.7.76.6.2206091000590.349@tvgsbejvaqbjf.bet","subject":"Re: Best way to update `HEAD` in mirrored repos","fromName":"Rodrigo Silva Mendoza","fromEmail":"rodrigosilvamendoza3@gmail.com","sentAt":"2022-06-09T14:46:04Z","receivedAt":"2022-06-09T14:46:19Z","isPatch":false,"sender":{"key":"rodrigosilvamendoza3@gmail.com","avatar":null},"body":"Hi Johannes,\n\nOn Thu, Jun 9, 2022 at 1:02 AM Johannes Schindelin\n wrote:\n>\n>\n> The reason is that `set-head` expects options to come before arguments,\n> like so:\n>\n> git remote set-head -a origin\n>\n> Ciao,\n> Johannes\n\n\nHmm, that doesn't seem to work for me either - I get the same type of\nerror as before.\nHere's a minimal repro:\n $ git clone git@github.com:git/gitscm-old.git --mirror\n $ cd gitscm-old.git\n $ git remote set-head -a origin\n error: Not a valid ref: refs/remotes/origin/master\n\nGit version: git version 2.32.1 (Apple Git-133)\n\nLet me know if I'm missing anything.\n\nCheers,\nRodrigo\n"},{"id":"457046","messageId":"nycvar.QRO.7.76.6.2206102327540.353@tvgsbejvaqbjf.bet","threadId":"57974","inReplyTo":"CANWRddNa1nB9shoppfXuA2yqmd2353HVBopoufkh0SevnVrngA@mail.gmail.com","subject":"Re: Best way to update `HEAD` in mirrored repos","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-06-10T21:46:37Z","receivedAt":"2022-06-10T21:46:59Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Rodrigo,\n\nOn Thu, 9 Jun 2022, Rodrigo Silva Mendoza wrote:\n\n> On Thu, Jun 9, 2022 at 1:02 AM Johannes Schindelin\n> wrote:\n> >\n> > The reason is that `set-head` expects options to come before arguments,\n> > like so:\n> >\n> > git remote set-head -a origin\n>\n> Hmm, that doesn't seem to work for me either - I get the same type of\n> error as before.\n> Here's a minimal repro:\n> $ git clone git@github.com:git/gitscm-old.git --mirror\n\nHere, you set this up as a fetch mirror. That is something I've missed\nbefore. It is also important because it means that there is no\n`refs/remotes/origin/HEAD`.\n\n> $ cd gitscm-old.git\n> $ git remote set-head -a origin\n> error: Not a valid ref: refs/remotes/origin/master\n\n... yet that's exactly what `set-head` assumes. The following patch fixes\nit for me:\n\n-- snip --\ndiff --git a/builtin/remote.c b/builtin/remote.c\nindex eddd40c8f87..fead15adb97 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -1344,7 +1344,7 @@ static int show(int argc, const char **argv)\n\n static int set_head(int argc, const char **argv)\n {\n-\tint i, opt_a = 0, opt_d = 0, result = 0;\n+\tint i, opt_a = 0, opt_d = 0, result = 0, is_mirror = 0;\n \tstruct strbuf buf = STRBUF_INIT, buf2 = STRBUF_INIT;\n \tchar *head_name = NULL;\n\n@@ -1357,8 +1357,16 @@ static int set_head(int argc, const char **argv)\n \t};\n \targc = parse_options(argc, argv, NULL, options, builtin_remote_sethead_usage,\n \t\t\t 0);\n-\tif (argc)\n-\t\tstrbuf_addf(&buf, \"refs/remotes/%s/HEAD\", argv[0]);\n+\tif (argc) {\n+\t\tstruct remote *remote = remote_get(argv[0]);\n+\n+\t\tif (!remote || !remote->mirror)\n+\t\t\tstrbuf_addf(&buf, \"refs/remotes/%s/HEAD\", argv[0]);\n+\t\telse {\n+\t\t\tis_mirror = 1;\n+\t\t\tstrbuf_addstr(&buf, \"HEAD\");\n+\t\t}\n+\t}\n\n \tif (!opt_a && !opt_d && argc == 2) {\n \t\thead_name = xstrdup(argv[1]);\n@@ -1383,7 +1391,10 @@ static int set_head(int argc, const char **argv)\n \t\tusage_with_options(builtin_remote_sethead_usage, options);\n\n \tif (head_name) {\n-\t\tstrbuf_addf(&buf2, \"refs/remotes/%s/%s\", argv[0], head_name);\n+\t\tif (is_mirror)\n+\t\t\tstrbuf_addf(&buf2, \"refs/heads/%s\", head_name);\n+\t\telse\n+\t\t\tstrbuf_addf(&buf2, \"refs/remotes/%s/%s\", argv[0], head_name);\n \t\t/* make sure it's valid */\n \t\tif (!ref_exists(buf2.buf))\n \t\t\tresult |= error(_(\"Not a valid ref: %s\"), buf2.buf);\n\n-- snap --\n\nWould you have a chance to build Git with this patch and verify that it\nworks for you, too?\n\nCiao,\nJohannes\n"},{"id":"457047","messageId":"xmqq5yl8chhz.fsf@gitster.g","threadId":"57974","inReplyTo":"nycvar.QRO.7.76.6.2206102327540.353@tvgsbejvaqbjf.bet","subject":"Re: Best way to update `HEAD` in mirrored repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-10T21:53:44Z","receivedAt":"2022-06-10T21:53:50Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> diff --git a/builtin/remote.c b/builtin/remote.c\n> index eddd40c8f87..fead15adb97 100644\n> --- a/builtin/remote.c\n> +++ b/builtin/remote.c\n> @@ -1344,7 +1344,7 @@ static int show(int argc, const char **argv)\n>\n> static int set_head(int argc, const char **argv)\n> {\n> -\tint i, opt_a = 0, opt_d = 0, result = 0;\n> +\tint i, opt_a = 0, opt_d = 0, result = 0, is_mirror = 0;\n> \tstruct strbuf buf = STRBUF_INIT, buf2 = STRBUF_INIT;\n> \tchar *head_name = NULL;\n>\n> @@ -1357,8 +1357,16 @@ static int set_head(int argc, const char **argv)\n> \t};\n> \targc = parse_options(argc, argv, NULL, options, builtin_remote_sethead_usage,\n> \t\t\t 0);\n> -\tif (argc)\n> -\t\tstrbuf_addf(&buf, \"refs/remotes/%s/HEAD\", argv[0]);\n> +\tif (argc) {\n> +\t\tstruct remote *remote = remote_get(argv[0]);\n> +\n> +\t\tif (!remote || !remote->mirror)\n> +\t\t\tstrbuf_addf(&buf, \"refs/remotes/%s/HEAD\", argv[0]);\n> +\t\telse {\n> +\t\t\tis_mirror = 1;\n> +\t\t\tstrbuf_addstr(&buf, \"HEAD\");\n> +\t\t}\n> +\t}\n\nGood.\n\nRegardless of the \"mirror\" issue, it makes tons of sense to ask the\nremote API how the remote-tracking refs for the given remote is set\nup, instead of assuming that it must be \"refs/remotes/\"\nblindly like in the original code.\n\nThat way, we could even handle a case like so:\n\n\t[remote \"frotz\"]\n\t\tfetch = +refs/heads/*:refs/remotes/nitfol/*\n\nTheir HEAD should be mapped to refs/remotes/nitfol/HEAD on our end,\nso set-head should be able to notice that, too, if we go further\nwith your \"do not assume, instead ask remote API\" approach.\n\nThanks.\n"},{"id":"457249","messageId":"CANWRddMXrhs2grTfq9NtHM7KqxaFRejvaXxvDGV90_m=Kmm-uA@mail.gmail.com","threadId":"57974","inReplyTo":"xmqq5yl8chhz.fsf@gitster.g","subject":"Re: Best way to update `HEAD` in mirrored repos","fromName":"Rodrigo Silva Mendoza","fromEmail":"rodrigosilvamendoza3@gmail.com","sentAt":"2022-06-15T04:11:27Z","receivedAt":"2022-06-15T04:11:43Z","isPatch":false,"sender":{"key":"rodrigosilvamendoza3@gmail.com","avatar":null},"body":"Hello,\n\nApologies for the delay, I'd never built Git from source so I put it\noff until I had a chunk of time, assuming it might take me a bit -\nthankfully it was painless.\n\nJohannes Schindelin writes:\n> Here, you set this up as a fetch mirror. That is something I've missed\n> before. It is also important because it means that there is no\n> `refs/remotes/origin/HEAD`.\n\nAhhh, the reason it wasn't working before clicked for me now, thanks\nfor the explanation.\n\n> diff --git a/builtin/remote.c b/builtin/remote.c\n> index eddd40c8f87..fead15adb97 100644\n> --- a/builtin/remote.c\n> +++ b/builtin/remote.c\n> @@ -1344,7 +1344,7 @@ static int show(int argc, const char **argv)\n>\n> static int set_head(int argc, const char **argv)\n> {\n> - int i, opt_a = 0, opt_d = 0, result = 0;\n> + int i, opt_a = 0, opt_d = 0, result = 0, is_mirror = 0;\n> struct strbuf buf = STRBUF_INIT, buf2 = STRBUF_INIT;\n> char *head_name = NULL;\n>\n> @@ -1357,8 +1357,16 @@ static int set_head(int argc, const char **argv)\n> };\n> argc = parse_options(argc, argv, NULL, options, builtin_remote_sethead_usage,\n> 0);\n> - if (argc)\n> - strbuf_addf(&buf, \"refs/remotes/%s/HEAD\", argv[0]);\n> + if (argc) {\n> + struct remote *remote = remote_get(argv[0]);\n> +\n> + if (!remote || !remote->mirror)\n> + strbuf_addf(&buf, \"refs/remotes/%s/HEAD\", argv[0]);\n> + else {\n> + is_mirror = 1;\n> + strbuf_addstr(&buf, \"HEAD\");\n> + }\n> + }\n\n> Would you have a chance to build Git with this patch and verify that it\n> works for you, too?\n\nGood news Johannes, your patch worked for me as well! Here's a run\nwith the patched code.\n\n $ git --version\n git version 2.37.0.rc0.dirty\n $ git clone git@github.com:xvandish/livegrep-fragment.git --mirror\n $ cd livegrep-fragment\n $ git symbolic-ref HEAD\n refs/heads/good_main_5\n $ // I changed the github default branch name at this moment\n $ git ls-remote --symref origin HEAD\n ref: refs/heads/good_main_5 HEAD\n 0666a519f94b8500ab6f14bdf7c9c2e5ca7d5821 HEAD\n $ git fetch -p\n From github.com:xvandish/livegrep-fragment\n - [deleted] (none) -> good_main_4\n * [new branch] good_main_5 -> good_main_5\n $ git remote set-head -a origin\n origin/HEAD set to good_main_5\n $ git symbolic-ref HEAD\n refs/heads/good_main_5\n\n\nOn Fri, Jun 10, 2022 at 2:53 PM Junio C Hamano wrote:\n> Good.\n>\n> Regardless of the \"mirror\" issue, it makes tons of sense to ask the\n> remote API how the remote-tracking refs for the given remote is set\n> up, instead of assuming that it must be \"refs/remotes/\"\n> blindly like in the original code.\n>\n> That way, we could even handle a case like so:\n>\n> [remote \"frotz\"]\n> fetch = +refs/heads/*:refs/remotes/nitfol/*\n>\n> Their HEAD should be mapped to refs/remotes/nitfol/HEAD on our end,\n> so set-head should be able to notice that, too, if we go further\n> with your \"do not assume, instead ask remote API\" approach.\n>\n> Thanks.\n\nThis makes sense to me as well, as a continuation of the \"ask remote \" style.\nI only have a very vague idea of how I'd implement this from a look at\nthe codebase, so I'd be of more\nhelp testing things out. If this idea carries forward let me know if I\ncan be of help testing, or if you'd like me to\nlike me to try to put a patch up let me know.\n\n\nThanks to both of you for your time.\n"},{"id":"457250","messageId":"CANWRddP3eUNiOTL0Jq_MoTXHUQFP430UcnCJBsm8pyqyVgrV1w@mail.gmail.com","threadId":"57974","inReplyTo":"CANWRddMXrhs2grTfq9NtHM7KqxaFRejvaXxvDGV90_m=Kmm-uA@mail.gmail.com","subject":"Re: Best way to update `HEAD` in mirrored repos","fromName":"Rodrigo Silva Mendoza","fromEmail":"rodrigosilvamendoza3@gmail.com","sentAt":"2022-06-15T04:18:51Z","receivedAt":"2022-06-15T04:19:07Z","isPatch":false,"sender":{"key":"rodrigosilvamendoza3@gmail.com","avatar":null},"body":"On Tue, Jun 14, 2022 at 9:11 PM Rodrigo Silva Mendoza\n wrote:\n\n> Good news Johannes, your patch worked for me as well! Here's a run\n> with the patched code.\n>\n> $ git --version\n> git version 2.37.0.rc0.dirty\n> $ git clone git@github.com:xvandish/livegrep-fragment.git --mirror\n> $ cd livegrep-fragment\n> $ git symbolic-ref HEAD\n> refs/heads/good_main_5\n....\n\nWhoops, botched the output of the first `git symbolic-ref HEAD` call.\nIt should be:\n\n $ git symbolic-ref HEAD\n refs/heads/good_main_4\n\nCheers,\nRodrigo\n"}]}