{"thread":{"id":"53841","subject":"FETCH_HEAD files and mirrored repos","startedAt":"2020-07-11T20:48:55Z","lastAt":"2020-07-14T04:11:53Z","messageCount":19,"participants":["Konstantin Ryabitsev","Junio C Hamano","Johannes Sixt","Taylor Blau","Jeff King","Jonathan Nieder"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"401399","messageId":"20200711204849.khfbyundun7ujqzw@chatter.i7.local","threadId":"53841","inReplyTo":null,"subject":"FETCH_HEAD files and mirrored repos","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-07-11T20:48:49Z","receivedAt":"2020-07-11T20:48:55Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"Hi, all:\n\nAre there any downsides to deleting FETCH_HEAD after performing \"git \nremote update\" in repos that are only used for mirrored hosting? E.g. \nsource.codeaurora.org has thousands of repos where various Android/CAF \nautomation creates tons of refs, so FETCH_HEAD files are routinely many \nMB in size:\n\n$ ls -ahl FETCH_HEAD\n-rw-rw-r--. 1 mricon mricon 4.4M Jul 11 04:28 FETCH_HEAD\n$ wc -l FETCH_HEAD\n29122 FETCH_HEAD\n\nAs far as I know, FETCH_HEAD info is only used in local-repo operations, \nso there should be no downside to deleting these files when git is only \nused for mirroring -- but I wanted to check in case I'm wrong.\n\nTIA,\n-K\n"},{"id":"401401","messageId":"xmqqimetrcay.fsf@gitster.c.googlers.com","threadId":"53841","inReplyTo":"20200711204849.khfbyundun7ujqzw@chatter.i7.local","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-07-11T21:07:01Z","receivedAt":"2020-07-11T21:07:09Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Konstantin Ryabitsev writes:\n\n> Are there any downsides to deleting FETCH_HEAD after performing \"git \n> remote update\" in repos that are only used for mirrored hosting?\n\nThe only time Git itself uses FETCH_HEAD is during \"git pull\".\n\n\"git pull\" is a two step process---it first calls \"git fetch\" and\nthen calls \"git merge\" (or \"git rebase\") to integrate the fetched\nhistory with the history of your current branch. \"git fetch\" leaves\nwhat it got in that file, and \"git pull\" figures out what parameters\nand message to use to drive the second step. After that, FETCH_HEAD\nis not used by Git. If you call \"git fetch\" yourself, FETCH_HEAD\nleft by that process is not used by Git itself, but you can use what\nis in it to manually execute the second step of what \"git pull\"\nwould do.\n\nSo, unless your script depends on the presence and/or the contents\nof FETCH_HEAD, you can safely remove it.\n\n"},{"id":"401402","messageId":"20200711211915.h5bdgmbkisyc23m3@chatter.i7.local","threadId":"53841","inReplyTo":"xmqqimetrcay.fsf@gitster.c.googlers.com","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-07-11T21:19:15Z","receivedAt":"2020-07-11T21:19:21Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Sat, Jul 11, 2020 at 02:07:01PM -0700, Junio C Hamano wrote:\n> So, unless your script depends on the presence and/or the contents\n> of FETCH_HEAD, you can safely remove it.\n\nExcellent, that just saved me 20G per each mirror. :)\n\nThanks,\n-K\n"},{"id":"401414","messageId":"xmqqblkkr63u.fsf@gitster.c.googlers.com","threadId":"53841","inReplyTo":"20200711211915.h5bdgmbkisyc23m3@chatter.i7.local","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-07-12T17:33:09Z","receivedAt":"2020-07-12T17:33:17Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Konstantin Ryabitsev writes:\n\n> On Sat, Jul 11, 2020 at 02:07:01PM -0700, Junio C Hamano wrote:\n>> So, unless your script depends on the presence and/or the contents\n>> of FETCH_HEAD, you can safely remove it.\n>\n> Excellent, that just saved me 20G per each mirror. :)\n\nMeaning that minimum 4k per file adds up that much due to 5 millions\nof repositories, or something?\n"},{"id":"401422","messageId":"20200712202502.x4p2c7rf6ctflt5g@chatter.i7.local","threadId":"53841","inReplyTo":"xmqqblkkr63u.fsf@gitster.c.googlers.com","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-07-12T20:25:02Z","receivedAt":"2020-07-12T20:26:37Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Sun, Jul 12, 2020 at 10:33:09AM -0700, Junio C Hamano wrote:\n> >> So, unless your script depends on the presence and/or the contents\n> >> of FETCH_HEAD, you can safely remove it.\n> >\n> > Excellent, that just saved me 20G per each mirror. :)\n> \n> Meaning that minimum 4k per file adds up that much due to 5 millions\n> of repositories, or something?\n\nGosh, no. Only 28,000 repositories, with many FETCH_HEAD files being a \nfew MB in size.\n\nI also discovered that symlinking FETCH_HEAD to /dev/null works as well \nand thus saves a few IO cycles.\n\n-K\n"},{"id":"401423","messageId":"xmqqr1tgpibi.fsf@gitster.c.googlers.com","threadId":"53841","inReplyTo":"20200712202502.x4p2c7rf6ctflt5g@chatter.i7.local","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-07-12T20:52:17Z","receivedAt":"2020-07-12T20:52:40Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Konstantin Ryabitsev writes:\n\n> I also discovered that symlinking FETCH_HEAD to /dev/null works as well \n> and thus saves a few IO cycles.\n\nWe could teach \"git fetch\" a new \"--write-fetch-head\" option and\nteach \"git pull\" to pass it when calling \"git fetch\". It is very\nplausible that some folks rely on \"git fetch\" to leave FETCH_HEAD\nwithout being told, so the option must default to on for a few\ndevelopment cycles before we flip the default to off, and for those\nwho want to live in the future a bit earlier, we can also introduce\nfetch.writeFetchHEAD configuration variable that determines what\nhappens when \"git fetch\" is run without \"--write-fetch-head\" option\non the command line. That way, you could drop \n\n\t[fetch]\n\t\twriteFetchHEAD = 0\n\nin say /etc/gitconfig or in ~/.gitconfig of the user that runs the\nmirroring automation.\n\n\n\n"},{"id":"401425","messageId":"20200712215433.cfjyqzaepsuktny2@chatter.i7.local","threadId":"53841","inReplyTo":"xmqqr1tgpibi.fsf@gitster.c.googlers.com","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-07-12T21:54:33Z","receivedAt":"2020-07-12T21:54:38Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Sun, Jul 12, 2020 at 01:52:17PM -0700, Junio C Hamano wrote:\n> Konstantin Ryabitsev writes:\n> \n> > I also discovered that symlinking FETCH_HEAD to /dev/null works as well \n> > and thus saves a few IO cycles.\n> \n> We could teach \"git fetch\" a new \"--write-fetch-head\" option and\n> teach \"git pull\" to pass it when calling \"git fetch\". It is very\n> plausible that some folks rely on \"git fetch\" to leave FETCH_HEAD\n> without being told, so the option must default to on for a few\n> development cycles before we flip the default to off, and for those\n> who want to live in the future a bit earlier, we can also introduce\n> fetch.writeFetchHEAD configuration variable that determines what\n> happens when \"git fetch\" is run without \"--write-fetch-head\" option\n> on the command line. That way, you could drop \n> \n> \t[fetch]\n> \t\twriteFetchHEAD = 0\n> \n> in say /etc/gitconfig or in ~/.gitconfig of the user that runs the\n> mirroring automation.\n\nYeah, that would be great for me. We can simply leave it on by default \nto preserve existing behaviour and just provide a way to disable writing \nit for those in fairly unique situations where that matters (like mine). \nOne downside of symlinking it to /dev/null is that it creates a \npotential pitfall for software that expects FETCH_HEAD to always be a \nregular file and can behave unpredictably when it finds anything else \nthere.\n\nThanks,\n-K\n"},{"id":"401492","messageId":"79a56e7b-3c31-80c0-6a8b-807dcb4d5af3@kdbg.org","threadId":"53841","inReplyTo":"xmqqr1tgpibi.fsf@gitster.c.googlers.com","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2020-07-13T17:09:29Z","receivedAt":"2020-07-13T17:09:34Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 12.07.20 um 22:52 schrieb Junio C Hamano:\n> We could teach \"git fetch\" a new \"--write-fetch-head\" option and\n> teach \"git pull\" to pass it when calling \"git fetch\". It is very\n> plausible that some folks rely on \"git fetch\" to leave FETCH_HEAD\n> without being told, so the option must default to on for a few\n> development cycles before we flip the default to off,\n\nNah, really??? It's one of the benefits of git-fetch that it writes\nFETCH_HEAD and the primary reason in many cases where I use the command!\nSo, either I don't care that FETCH_HEAD is written, or I do use it. IMO,\nnot wanting to write FETCH_HEAD is the odd case and would need a\nconfiguration tweak, not the other way round.\n\n-- Hannes\n"},{"id":"401493","messageId":"xmqqk0z7nxs7.fsf@gitster.c.googlers.com","threadId":"53841","inReplyTo":"79a56e7b-3c31-80c0-6a8b-807dcb4d5af3@kdbg.org","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-07-13T17:13:28Z","receivedAt":"2020-07-13T17:13:33Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt writes:\n\n> Am 12.07.20 um 22:52 schrieb Junio C Hamano:\n>> We could teach \"git fetch\" a new \"--write-fetch-head\" option and\n>> teach \"git pull\" to pass it when calling \"git fetch\". It is very\n>> plausible that some folks rely on \"git fetch\" to leave FETCH_HEAD\n>> without being told, so the option must default to on for a few\n>> development cycles before we flip the default to off,\n>\n> Nah, really??? It's one of the benefits of git-fetch that it writes\n> FETCH_HEAD and the primary reason in many cases where I use the command!\n> So, either I don't care that FETCH_HEAD is written, or I do use it. IMO,\n> not wanting to write FETCH_HEAD is the odd case and would need a\n> configuration tweak, not the other way round.\n\nYeah, that's even easier to arrange. \n\nJust the \"--[no-]write-fetch-head\" command line option and the\nfetch.writeFetchHEAD configuration variable are introduced and left\noff by default forever.\n\n"},{"id":"401494","messageId":"xmqqft9vnvce.fsf_-_@gitster.c.googlers.com","threadId":"53841","inReplyTo":"xmqqk0z7nxs7.fsf@gitster.c.googlers.com","subject":"[PATCH] fetch: optionally allow disabling FETCH_HEAD update","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-07-13T18:06:09Z","receivedAt":"2020-07-13T18:06:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"If you run fetch but record the result in remote-tracking branches,\nand either if you do nothing with the fetched refs (e.g. you are\nmerely mirroring) or if you always work from the remote-tracking\nrefs (e.g. you fetch and then merge origin/branchname separately),\nyou can get away without having FETCH_HEAD at all.\n\nTeach \"git fetch\" a command line option \"--[no-]write-fetch-head\"\nand \"fetch.writeFetchHEAD\" configuration variable. Without either,\nthe default is to write FETCH_HEAD, and the usual rule that the\ncommand line option defeats configured default applies.\n\nNote that under \"--dry-run\" mode, FETCH_HEAD is never written;\notherwise you'd see list of objects in the file that you do not\nactually have.\n\nAlso note that this option is explicitly passed when \"git pull\"\ninternally invokes \"git fetch\", so that those who configured their\n\"git fetch\" not to write FETCH_HEAD would not be able to break the\ncooperation between these two commands. \"git pull\" must see what\n\"git fetch\" got recorded in FETCH_HEAD to work correctly.\n\nSigned-off-by: Junio C Hamano \n---\n\n Junio C Hamano writes:\n\n > Johannes Sixt writes:\n >\n >> Nah, really??? It's one of the benefits of git-fetch that it writes\n >> FETCH_HEAD and the primary reason in many cases where I use the command!\n >> So, either I don't care that FETCH_HEAD is written, or I do use it. IMO,\n >> not wanting to write FETCH_HEAD is the odd case and would need a\n >> configuration tweak, not the other way round.\n >\n > Yeah, that's even easier to arrange. \n >\n > Just the \"--[no-]write-fetch-head\" command line option and the\n > fetch.writeFetchHEAD configuration variable are introduced and left\n > off by default forever.\n\n Something like this, perhaps. Obviously I won't be pushing this\n topic further while in prerelase freeze, but Konstantin or\n anybody else interested can locally apply the patch to their own\n copy of Git to test it out.\n\n Thanks.\n\n builtin/fetch.c | 19 ++++++++++++++++---\n builtin/pull.c | 3 ++-\n t/t5510-fetch.sh | 39 +++++++++++++++++++++++++++++++++++++--\n 3 files changed, 55 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 82ac4be8a5..3ccf69753f 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -56,6 +56,7 @@ static int prune_tags = -1; /* unspecified */\n #define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */\n \n static int all, append, dry_run, force, keep, multiple, update_head_ok;\n+static int write_fetch_head = 1;\n static int verbosity, deepen_relative, set_upstream;\n static int progress = -1;\n static int enable_auto_gc = 1;\n@@ -118,6 +119,10 @@ static int git_fetch_config(const char *k, const char *v, void *cb)\n \t\treturn 0;\n \t}\n \n+\tif (!strcmp(k, \"fetch.writefetchhead\")) {\n+\t\twrite_fetch_head = git_config_bool(k, v);\n+\t\treturn 0;\n+\t}\n \treturn git_default_config(k, v, cb);\n }\n \n@@ -162,6 +167,8 @@ static struct option builtin_fetch_options[] = {\n \t\t PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),\n \tOPT_BOOL(0, \"dry-run\", &dry_run,\n \t\t N_(\"dry run\")),\n+\tOPT_BOOL(0, \"write-fetch-head\", &write_fetch_head,\n+\t\t N_(\"write fetched references to the FETCH_HEAD file\")),\n \tOPT_BOOL('k', \"keep\", &keep, N_(\"keep downloaded pack\")),\n \tOPT_BOOL('u', \"update-head-ok\", &update_head_ok,\n \t\t N_(\"allow updating of HEAD ref\")),\n@@ -893,7 +900,9 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,\n \tconst char *what, *kind;\n \tstruct ref *rm;\n \tchar *url;\n-\tconst char *filename = dry_run ? \"/dev/null\" : git_path_fetch_head(the_repository);\n+\tconst char *filename = (!write_fetch_head\n+\t\t\t\t? \"/dev/null\"\n+\t\t\t\t: git_path_fetch_head(the_repository));\n \tint want_status;\n \tint summary_width = transport_summary_width(ref_map);\n \n@@ -1327,7 +1336,7 @@ static int do_fetch(struct transport *transport,\n \t}\n \n \t/* if not appending, truncate FETCH_HEAD */\n-\tif (!append && !dry_run) {\n+\tif (!append && write_fetch_head) {\n \t\tretcode = truncate_fetch_head();\n \t\tif (retcode)\n \t\t\tgoto cleanup;\n@@ -1594,7 +1603,7 @@ static int fetch_multiple(struct string_list *list, int max_children)\n \tint i, result = 0;\n \tstruct argv_array argv = ARGV_ARRAY_INIT;\n \n-\tif (!append && !dry_run) {\n+\tif (!append && write_fetch_head) {\n \t\tint errcode = truncate_fetch_head();\n \t\tif (errcode)\n \t\t\treturn errcode;\n@@ -1795,6 +1804,10 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \tif (depth || deepen_since || deepen_not.nr)\n \t\tdeepen = 1;\n \n+\t/* FETCH_HEAD never gets updated in --dry-run mode */\n+\tif (dry_run)\n+\t\twrite_fetch_head = 0;\n+\n \tif (all) {\n \t\tif (argc == 1)\n \t\t\tdie(_(\"fetch --all does not take a repository argument\"));\ndiff --git a/builtin/pull.c b/builtin/pull.c\nindex 8159c5d7c9..e988d92b53 100644\n--- a/builtin/pull.c\n+++ b/builtin/pull.c\n@@ -527,7 +527,8 @@ static int run_fetch(const char *repo, const char **refspecs)\n \tstruct argv_array args = ARGV_ARRAY_INIT;\n \tint ret;\n \n-\targv_array_pushl(&args, \"fetch\", \"--update-head-ok\", NULL);\n+\targv_array_pushl(&args, \"fetch\", \"--update-head-ok\",\n+\t\t\t \"--write-fetch-head\", NULL);\n \n \t/* Shared options */\n \targv_push_verbosity(&args);\ndiff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\nindex a66dbe0bde..3052c2d8d5 100755\n--- a/t/t5510-fetch.sh\n+++ b/t/t5510-fetch.sh\n@@ -539,13 +539,48 @@ test_expect_success 'fetch into the current branch with --update-head-ok' '\n \n '\n \n-test_expect_success 'fetch --dry-run' '\n-\n+test_expect_success 'fetch --dry-run does not touch FETCH_HEAD' '\n \trm -f .git/FETCH_HEAD &&\n \tgit fetch --dry-run . &&\n \t! test -f .git/FETCH_HEAD\n '\n \n+test_expect_success '--no-write-fetch-head does not touch FETCH_HEAD' '\n+\trm -f .git/FETCH_HEAD &&\n+\tgit fetch --no-write-fetch-head . &&\n+\t! test -f .git/FETCH_HEAD\n+'\n+\n+test_expect_success '--write-fetch-head gets defeated by --dry-run' '\n+\trm -f .git/FETCH_HEAD &&\n+\tgit fetch --dry-run --write-fetch-head . &&\n+\t! test -f .git/FETCH_HEAD\n+'\n+\n+test_expect_success 'fetch.writeFetchHEAD and FETCH_HEAD' '\n+\trm -f .git/FETCH_HEAD &&\n+\tgit -c fetch.writeFetchHEAD=no fetch . &&\n+\t! test -f .git/FETCH_HEAD\n+'\n+\n+test_expect_success 'fetch.writeFetchHEAD gets defeated by --dry-run' '\n+\trm -f .git/FETCH_HEAD &&\n+\tgit -c fetch.writeFetchHEAD=yes fetch --dry-run . &&\n+\t! test -f .git/FETCH_HEAD\n+'\n+\n+test_expect_success 'fetch.writeFetchHEAD and --no-write-fetch-head' '\n+\trm -f .git/FETCH_HEAD &&\n+\tgit -c fetch.writeFetchHEAD=yes fetch --no-write-fetch-head . &&\n+\t! test -f .git/FETCH_HEAD\n+'\n+\n+test_expect_success 'fetch.writeFetchHEAD and --write-fetch-head' '\n+\trm -f .git/FETCH_HEAD &&\n+\tgit -c fetch.writeFetchHEAD=no fetch --write-fetch-head . &&\n+\ttest -f .git/FETCH_HEAD\n+'\n+\n test_expect_success \"should be able to fetch with duplicate refspecs\" '\n \tmkdir dups &&\n \t(\n-- \n2.28.0-rc0\n\n"},{"id":"401495","messageId":"20200713190834.GA77607@syl.lan","threadId":"53841","inReplyTo":"xmqqft9vnvce.fsf_-_@gitster.c.googlers.com","subject":"Re: [PATCH] fetch: optionally allow disabling FETCH_HEAD update","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2020-07-13T19:08:34Z","receivedAt":"2020-07-13T19:08:44Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Mon, Jul 13, 2020 at 11:06:09AM -0700, Junio C Hamano wrote:\n> If you run fetch but record the result in remote-tracking branches,\n> and either if you do nothing with the fetched refs (e.g. you are\n> merely mirroring) or if you always work from the remote-tracking\n> refs (e.g. you fetch and then merge origin/branchname separately),\n> you can get away without having FETCH_HEAD at all.\n>\n> Teach \"git fetch\" a command line option \"--[no-]write-fetch-head\"\n> and \"fetch.writeFetchHEAD\" configuration variable. Without either,\n> the default is to write FETCH_HEAD, and the usual rule that the\n> command line option defeats configured default applies.\n>\n> Note that under \"--dry-run\" mode, FETCH_HEAD is never written;\n> otherwise you'd see list of objects in the file that you do not\n> actually have.\n>\n> Also note that this option is explicitly passed when \"git pull\"\n> internally invokes \"git fetch\", so that those who configured their\n> \"git fetch\" not to write FETCH_HEAD would not be able to break the\n> cooperation between these two commands. \"git pull\" must see what\n> \"git fetch\" got recorded in FETCH_HEAD to work correctly.\n>\n> Signed-off-by: Junio C Hamano \n> ---\n>\n> Junio C Hamano writes:\n>\n> > Johannes Sixt writes:\n> >\n> >> Nah, really??? It's one of the benefits of git-fetch that it writes\n> >> FETCH_HEAD and the primary reason in many cases where I use the command!\n> >> So, either I don't care that FETCH_HEAD is written, or I do use it. IMO,\n> >> not wanting to write FETCH_HEAD is the odd case and would need a\n> >> configuration tweak, not the other way round.\n> >\n> > Yeah, that's even easier to arrange.\n> >\n> > Just the \"--[no-]write-fetch-head\" command line option and the\n> > fetch.writeFetchHEAD configuration variable are introduced and left\n> > off by default forever.\n>\n> Something like this, perhaps. Obviously I won't be pushing this\n> topic further while in prerelase freeze, but Konstantin or\n> anybody else interested can locally apply the patch to their own\n> copy of Git to test it out.\n>\n> Thanks.\n>\n> builtin/fetch.c | 19 ++++++++++++++++---\n> builtin/pull.c | 3 ++-\n> t/t5510-fetch.sh | 39 +++++++++++++++++++++++++++++++++++++--\n> 3 files changed, 55 insertions(+), 6 deletions(-)\n>\n> diff --git a/builtin/fetch.c b/builtin/fetch.c\n> index 82ac4be8a5..3ccf69753f 100644\n> --- a/builtin/fetch.c\n> +++ b/builtin/fetch.c\n> @@ -56,6 +56,7 @@ static int prune_tags = -1; /* unspecified */\n> #define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */\n>\n> static int all, append, dry_run, force, keep, multiple, update_head_ok;\n> +static int write_fetch_head = 1;\n> static int verbosity, deepen_relative, set_upstream;\n> static int progress = -1;\n> static int enable_auto_gc = 1;\n> @@ -118,6 +119,10 @@ static int git_fetch_config(const char *k, const char *v, void *cb)\n> \t\treturn 0;\n> \t}\n>\n> +\tif (!strcmp(k, \"fetch.writefetchhead\")) {\n> +\t\twrite_fetch_head = git_config_bool(k, v);\n> +\t\treturn 0;\n> +\t}\n> \treturn git_default_config(k, v, cb);\n> }\n>\n> @@ -162,6 +167,8 @@ static struct option builtin_fetch_options[] = {\n> \t\t PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),\n> \tOPT_BOOL(0, \"dry-run\", &dry_run,\n> \t\t N_(\"dry run\")),\n> +\tOPT_BOOL(0, \"write-fetch-head\", &write_fetch_head,\n> +\t\t N_(\"write fetched references to the FETCH_HEAD file\")),\n> \tOPT_BOOL('k', \"keep\", &keep, N_(\"keep downloaded pack\")),\n> \tOPT_BOOL('u', \"update-head-ok\", &update_head_ok,\n> \t\t N_(\"allow updating of HEAD ref\")),\n> @@ -893,7 +900,9 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,\n> \tconst char *what, *kind;\n> \tstruct ref *rm;\n> \tchar *url;\n> -\tconst char *filename = dry_run ? \"/dev/null\" : git_path_fetch_head(the_repository);\n> +\tconst char *filename = (!write_fetch_head\n> +\t\t\t\t? \"/dev/null\"\n> +\t\t\t\t: git_path_fetch_head(the_repository));\n\nAh, because dry_run ==> !write_fetch_head, so this is an equivalent\ntranslation. Makes sense.\n\n> \tint want_status;\n> \tint summary_width = transport_summary_width(ref_map);\n>\n> @@ -1327,7 +1336,7 @@ static int do_fetch(struct transport *transport,\n> \t}\n>\n> \t/* if not appending, truncate FETCH_HEAD */\n> -\tif (!append && !dry_run) {\n> +\tif (!append && write_fetch_head) {\n> \t\tretcode = truncate_fetch_head();\n> \t\tif (retcode)\n> \t\t\tgoto cleanup;\n> @@ -1594,7 +1603,7 @@ static int fetch_multiple(struct string_list *list, int max_children)\n> \tint i, result = 0;\n> \tstruct argv_array argv = ARGV_ARRAY_INIT;\n>\n> -\tif (!append && !dry_run) {\n> +\tif (!append && write_fetch_head) {\n> \t\tint errcode = truncate_fetch_head();\n> \t\tif (errcode)\n> \t\t\treturn errcode;\n> @@ -1795,6 +1804,10 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n> \tif (depth || deepen_since || deepen_not.nr)\n> \t\tdeepen = 1;\n>\n> +\t/* FETCH_HEAD never gets updated in --dry-run mode */\n> +\tif (dry_run)\n> +\t\twrite_fetch_head = 0;\n> +\n> \tif (all) {\n> \t\tif (argc == 1)\n> \t\t\tdie(_(\"fetch --all does not take a repository argument\"));\n> diff --git a/builtin/pull.c b/builtin/pull.c\n> index 8159c5d7c9..e988d92b53 100644\n> --- a/builtin/pull.c\n> +++ b/builtin/pull.c\n> @@ -527,7 +527,8 @@ static int run_fetch(const char *repo, const char **refspecs)\n> \tstruct argv_array args = ARGV_ARRAY_INIT;\n> \tint ret;\n>\n> -\targv_array_pushl(&args, \"fetch\", \"--update-head-ok\", NULL);\n> +\targv_array_pushl(&args, \"fetch\", \"--update-head-ok\",\n> +\t\t\t \"--write-fetch-head\", NULL);\n\n...and here we pass '--write-fetch-head' explicitly, because we don't\nwant a user who has set 'fetch.writeFetchHead' to 'false' to suddenly\nhave their 'git pull's stop working. Makes sense.\n\n>\n> \t/* Shared options */\n> \targv_push_verbosity(&args);\n> diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\n> index a66dbe0bde..3052c2d8d5 100755\n> --- a/t/t5510-fetch.sh\n> +++ b/t/t5510-fetch.sh\n> @@ -539,13 +539,48 @@ test_expect_success 'fetch into the current branch with --update-head-ok' '\n>\n> '\n>\n> -test_expect_success 'fetch --dry-run' '\n> -\n> +test_expect_success 'fetch --dry-run does not touch FETCH_HEAD' '\n> \trm -f .git/FETCH_HEAD &&\n> \tgit fetch --dry-run . &&\n> \t! test -f .git/FETCH_HEAD\n> '\n>\n> +test_expect_success '--no-write-fetch-head does not touch FETCH_HEAD' '\n> +\trm -f .git/FETCH_HEAD &&\n> +\tgit fetch --no-write-fetch-head . &&\n> +\t! test -f .git/FETCH_HEAD\n> +'\n> +\n> +test_expect_success '--write-fetch-head gets defeated by --dry-run' '\n> +\trm -f .git/FETCH_HEAD &&\n> +\tgit fetch --dry-run --write-fetch-head . &&\n> +\t! test -f .git/FETCH_HEAD\n> +'\n> +\n> +test_expect_success 'fetch.writeFetchHEAD and FETCH_HEAD' '\n> +\trm -f .git/FETCH_HEAD &&\n> +\tgit -c fetch.writeFetchHEAD=no fetch . &&\n> +\t! test -f .git/FETCH_HEAD\n> +'\n> +\n> +test_expect_success 'fetch.writeFetchHEAD gets defeated by --dry-run' '\n> +\trm -f .git/FETCH_HEAD &&\n> +\tgit -c fetch.writeFetchHEAD=yes fetch --dry-run . &&\n> +\t! test -f .git/FETCH_HEAD\n> +'\n> +\n> +test_expect_success 'fetch.writeFetchHEAD and --no-write-fetch-head' '\n> +\trm -f .git/FETCH_HEAD &&\n> +\tgit -c fetch.writeFetchHEAD=yes fetch --no-write-fetch-head . &&\n> +\t! test -f .git/FETCH_HEAD\n> +'\n> +\n> +test_expect_success 'fetch.writeFetchHEAD and --write-fetch-head' '\n> +\trm -f .git/FETCH_HEAD &&\n> +\tgit -c fetch.writeFetchHEAD=no fetch --write-fetch-head . &&\n> +\ttest -f .git/FETCH_HEAD\n> +'\n> +\n> test_expect_success \"should be able to fetch with duplicate refspecs\" '\n> \tmkdir dups &&\n> \t(\n\nTest coverage all looks good, thanks for working on this. I don't think\nthere's anything left, so this would be great after 2.28 is released.\n\n Reviewed-by: Taylor Blau \n\n> --\n> 2.28.0-rc0\n>\n\nThanks,\nTaylor\n"},{"id":"401505","messageId":"xmqq7dv7nqqo.fsf@gitster.c.googlers.com","threadId":"53841","inReplyTo":"20200713190834.GA77607@syl.lan","subject":"Re: [PATCH] fetch: optionally allow disabling FETCH_HEAD update","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-07-13T19:45:35Z","receivedAt":"2020-07-13T19:45:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau writes:\n\n>> -\tconst char *filename = dry_run ? \"/dev/null\" : git_path_fetch_head(the_repository);\n>> +\tconst char *filename = (!write_fetch_head\n>> +\t\t\t\t? \"/dev/null\"\n>> +\t\t\t\t: git_path_fetch_head(the_repository));\n>\n> Ah, because dry_run ==> !write_fetch_head, so this is an equivalent\n> translation. Makes sense.\n\nYup, now dry_run is merely one of the two cases we turn\nwrite_fetch_head off, and the logic to prevent us from writing\nFETCH_HEAD (hopefully without any other side effect) was already\nthere in this function.\n\n> Test coverage all looks good, thanks for working on this. I don't think\n> there's anything left, so this would be great after 2.28 is released.\n>\n> Reviewed-by: Taylor Blau \n\nThanks.\n"},{"id":"401510","messageId":"20200713200049.6a22nek4xckwtpt6@chatter.i7.local","threadId":"53841","inReplyTo":"xmqqk0z7nxs7.fsf@gitster.c.googlers.com","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-07-13T20:00:49Z","receivedAt":"2020-07-13T20:00:55Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Mon, Jul 13, 2020 at 10:13:28AM -0700, Junio C Hamano wrote:\n> > Nah, really??? It's one of the benefits of git-fetch that it writes\n> > FETCH_HEAD and the primary reason in many cases where I use the command!\n> > So, either I don't care that FETCH_HEAD is written, or I do use it. IMO,\n> > not wanting to write FETCH_HEAD is the odd case and would need a\n> > configuration tweak, not the other way round.\n> \n> Yeah, that's even easier to arrange. \n> \n> Just the \"--[no-]write-fetch-head\" command line option and the\n> fetch.writeFetchHEAD configuration variable are introduced and left\n> off by default forever.\n\nDoes it make sense to add logic for whether this is done in a bare repo? \nI can't imagine common cases where a FETCH_HEAD would be useful outside \nof a checkout where a merge is likely to happen.\n\n-K\n"},{"id":"401511","messageId":"xmqqtuybmba1.fsf@gitster.c.googlers.com","threadId":"53841","inReplyTo":"20200713200049.6a22nek4xckwtpt6@chatter.i7.local","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-07-13T20:04:54Z","receivedAt":"2020-07-13T20:04:59Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Konstantin Ryabitsev writes:\n\n> Does it make sense to add logic for whether this is done in a bare repo? \n> I can't imagine common cases where a FETCH_HEAD would be useful outside \n> of a checkout where a merge is likely to happen.\n\nIt is entirely valid to respond to a one-shot \"My work is published\nthere; could you see if I am doing anything obviously wrong?\" with\n\n git fetch git://k.org/somebody/linux.git check-me-please\n git log ..FETCH_HEAD\n git diff ...FETCH_HEAD\n\ni.e. without touching any of our refs, and possibly in a bare\nrepository, no?\n\n\n\n\n\n"},{"id":"401513","messageId":"20200713202211.GA2355588@coredump.intra.peff.net","threadId":"53841","inReplyTo":"xmqqtuybmba1.fsf@gitster.c.googlers.com","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-07-13T20:22:11Z","receivedAt":"2020-07-13T20:22:15Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 13, 2020 at 01:04:54PM -0700, Junio C Hamano wrote:\n\n> Konstantin Ryabitsev writes:\n> \n> > Does it make sense to add logic for whether this is done in a bare repo? \n> > I can't imagine common cases where a FETCH_HEAD would be useful outside \n> > of a checkout where a merge is likely to happen.\n> \n> It is entirely valid to respond to a one-shot \"My work is published\n> there; could you see if I am doing anything obviously wrong?\" with\n> \n> git fetch git://k.org/somebody/linux.git check-me-please\n> git log ..FETCH_HEAD\n> git diff ...FETCH_HEAD\n> \n> i.e. without touching any of our refs, and possibly in a bare\n> repository, no?\n\nI occasionally use FETCH_HEAD for such things. If we were to stop\nwriting it automatically, I think the key thing to notice is whether the\nresult was actually stored anywhere else. Or more accurately, whether\nthe user asked for any refspecs on the command line (since we'd still\nupdate tracking refs in some cases).\n\nIf I do:\n\n git fetch\n\nor:\n\n git fetch origin refs/heads/foo:refs/heads/foo\n\nthen I probably don't care about FETCH_HEAD. But if I do:\n\n git fetch origin refs/heads/foo\n\nthen I'm probably interested in picking the result out of FETCH_HEAD.\n\n-Peff\n"},{"id":"401514","messageId":"20200713203427.apajagayqx4gpadf@chatter.i7.local","threadId":"53841","inReplyTo":"20200713202211.GA2355588@coredump.intra.peff.net","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-07-13T20:34:27Z","receivedAt":"2020-07-13T20:34:34Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Mon, Jul 13, 2020 at 04:22:11PM -0400, Jeff King wrote:\n> I occasionally use FETCH_HEAD for such things. If we were to stop\n> writing it automatically, I think the key thing to notice is whether the\n> result was actually stored anywhere else. Or more accurately, whether\n> the user asked for any refspecs on the command line (since we'd still\n> update tracking refs in some cases).\n> \n> If I do:\n> \n> git fetch\n> \n> or:\n> \n> git fetch origin refs/heads/foo:refs/heads/foo\n> \n> then I probably don't care about FETCH_HEAD. But if I do:\n> \n> git fetch origin refs/heads/foo\n> \n> then I'm probably interested in picking the result out of FETCH_HEAD.\n\nI think adding all this logic is not worth the effort. For vast numbers \nof people who aren't running Android mirrors, FETCH_HEAD is only going \nto be a few KB in size, so they won't benefit from this change at all.\n\nI'm happy with just an option that I have to enable to turn off writing \nFETCH_HEAD.\n\n-K\n"},{"id":"401515","messageId":"234fc5b6-9a64-725c-afd3-e102ea849e97@kdbg.org","threadId":"53841","inReplyTo":"20200713202211.GA2355588@coredump.intra.peff.net","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2020-07-13T20:43:49Z","receivedAt":"2020-07-13T20:43:54Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 13.07.20 um 22:22 schrieb Jeff King:\n> I occasionally use FETCH_HEAD for such things. If we were to stop\n> writing it automatically, I think the key thing to notice is whether the\n> result was actually stored anywhere else. Or more accurately, whether\n> the user asked for any refspecs on the command line (since we'd still\n> update tracking refs in some cases).\n> \n> If I do:\n> \n> git fetch\n> \n> or:\n> \n> git fetch origin refs/heads/foo:refs/heads/foo\n> \n> then I probably don't care about FETCH_HEAD. But if I do:\n> \n> git fetch origin refs/heads/foo\n> \n> then I'm probably interested in picking the result out of FETCH_HEAD.\n\nVery good observation. The latter is exactly the \"many cases\" I\nmentioned where I do care about FETCH_HEAD.\n\n-- Hannes\n"},{"id":"401517","messageId":"20200713205039.GA2356680@coredump.intra.peff.net","threadId":"53841","inReplyTo":"20200713203427.apajagayqx4gpadf@chatter.i7.local","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-07-13T20:50:39Z","receivedAt":"2020-07-13T20:50:41Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 13, 2020 at 04:34:27PM -0400, Konstantin Ryabitsev wrote:\n\n> I think adding all this logic is not worth the effort. For vast numbers \n> of people who aren't running Android mirrors, FETCH_HEAD is only going \n> to be a few KB in size, so they won't benefit from this change at all.\n> \n> I'm happy with just an option that I have to enable to turn off writing \n> FETCH_HEAD.\n\nYeah, I'm totally fine with that, too. And because it's an incremental\nset of steps (add the option, then possibly flip the default), we can\neasily stop at step 1 and see if anybody cares enough about step 2 to\ncontinue.\n\n-Peff\n"},{"id":"401533","messageId":"20200714041148.GB2208896@google.com","threadId":"53841","inReplyTo":"79a56e7b-3c31-80c0-6a8b-807dcb4d5af3@kdbg.org","subject":"Re: FETCH_HEAD files and mirrored repos","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2020-07-14T04:11:48Z","receivedAt":"2020-07-14T04:11:53Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Johannes Sixt wrote:\n> Am 12.07.20 um 22:52 schrieb Junio C Hamano:\n\n>> We could teach \"git fetch\" a new \"--write-fetch-head\" option and\n>> teach \"git pull\" to pass it when calling \"git fetch\". It is very\n>> plausible that some folks rely on \"git fetch\" to leave FETCH_HEAD\n>> without being told, so the option must default to on for a few\n>> development cycles before we flip the default to off,\n>\n> Nah, really??? It's one of the benefits of git-fetch that it writes\n> FETCH_HEAD and the primary reason in many cases where I use the command!\n> So, either I don't care that FETCH_HEAD is written, or I do use it. IMO,\n> not wanting to write FETCH_HEAD is the odd case and would need a\n> configuration tweak, not the other way round.\n\nSpeaking for myself as a user, I also rely on (the first line of)\nFETCH_HEAD being written so that I can run commands like \"git log\nFETCH_HEAD\". What I don't rely on is the massive amounts of data that\nmay get written after that first line.\n\nThanks,\nJonathan\n"}]}