{"thread":{"id":"60878","subject":"[PATCH] git: --no-lazy-fetch option","startedAt":"2024-02-08T23:17:37Z","lastAt":"2024-03-09T01:57:16Z","messageCount":20,"participants":["Junio C Hamano","Linus Arver","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"488260","messageId":"xmqq1q9mmtpw.fsf@gitster.g","threadId":"60878","inReplyTo":null,"subject":"[PATCH] git: --no-lazy-fetch option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-08T23:17:31Z","receivedAt":"2024-02-08T23:17:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Sometimes, especially during tests of low level machinery, it is\nhandy to have a way to disable lazy fetching of objects.  This\nallows us to say, for example, \"git cat-file -e <object-name>\", to\nsee if the object is locally available.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n git.c | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/git.c b/git.c\nindex 7068a184b0..fd9b0e6a9e 100644\n--- a/git.c\n+++ b/git.c\n@@ -4,6 +4,7 @@\n #include \"exec-cmd.h\"\n #include \"gettext.h\"\n #include \"help.h\"\n+#include \"object-file.h\"\n #include \"pager.h\"\n #include \"read-cache-ll.h\"\n #include \"run-command.h\"\n@@ -186,6 +187,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n \t\t\tuse_pager = 0;\n \t\t\tif (envchanged)\n \t\t\t\t*envchanged = 1;\n+\t\t} else if (!strcmp(cmd, \"--no-lazy-fetch\")) {\n+\t\t\tfetch_if_missing = 0;\n \t\t} else if (!strcmp(cmd, \"--no-replace-objects\")) {\n \t\t\tdisable_replace_refs();\n \t\t\tsetenv(NO_REPLACE_OBJECTS_ENVIRONMENT, \"1\", 1);\n-- \n2.43.0-581-g5216f8f5c4\n\n"},{"id":"488573","messageId":"owlyr0hg9kr3.fsf@fine.c.googlers.com","threadId":"60878","inReplyTo":"xmqq1q9mmtpw.fsf@gitster.g","subject":"Re: [PATCH] git: --no-lazy-fetch option","fromName":"Linus Arver","fromEmail":"linusa@google.com","sentAt":"2024-02-13T20:23:12Z","receivedAt":"2024-02-13T20:23:14Z","isPatch":true,"sender":{"key":"linus@ucla.edu","avatar":null},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Sometimes, especially during tests of low level machinery, it is\n> handy to have a way to disable lazy fetching of objects.  This\n> allows us to say, for example, \"git cat-file -e <object-name>\", to\n> see if the object is locally available.\n\nNit: perhaps s/locally/already locally/ is better?\n"},{"id":"488578","messageId":"owlyo7ck9k3y.fsf@fine.c.googlers.com","threadId":"60878","inReplyTo":"owlyr0hg9kr3.fsf@fine.c.googlers.com","subject":"Re: [PATCH] git: --no-lazy-fetch option","fromName":"Linus Arver","fromEmail":"linusa@google.com","sentAt":"2024-02-13T20:37:05Z","receivedAt":"2024-02-13T20:37:07Z","isPatch":true,"sender":{"key":"linus@ucla.edu","avatar":null},"body":"Linus Arver <linusa@google.com> writes:\n\n> Junio C Hamano <gitster@pobox.com> writes:\n>\n>> Sometimes, especially during tests of low level machinery, it is\n>> handy to have a way to disable lazy fetching of objects.  This\n>> allows us to say, for example, \"git cat-file -e <object-name>\", to\n>> see if the object is locally available.\n>\n> Nit: perhaps s/locally/already locally/ is better?\n\nI forgot to mention that the new flag is missing a documentation entry\nin Documentation/git.txt. Perhaps something like\n\n    --no-lazy-fetch::\n        Do not fetch missing objects. Useful together with `git cat-file -e\n        <object-name>`, for example, to see if the object is already\n        locally available.\n\n?\n"},{"id":"488581","messageId":"xmqqsf1w3x9g.fsf@gitster.g","threadId":"60878","inReplyTo":"owlyo7ck9k3y.fsf@fine.c.googlers.com","subject":"Re: [PATCH] git: --no-lazy-fetch option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-13T20:49:31Z","receivedAt":"2024-02-13T20:49:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Arver <linusa@google.com> writes:\n\n> Linus Arver <linusa@google.com> writes:\n>\n>> Junio C Hamano <gitster@pobox.com> writes:\n>>\n>>> Sometimes, especially during tests of low level machinery, it is\n>>> handy to have a way to disable lazy fetching of objects.  This\n>>> allows us to say, for example, \"git cat-file -e <object-name>\", to\n>>> see if the object is locally available.\n>>\n>> Nit: perhaps s/locally/already locally/ is better?\n>\n> I forgot to mention that the new flag is missing a documentation entry\n> in Documentation/git.txt. Perhaps something like\n>\n>     --no-lazy-fetch::\n>         Do not fetch missing objects. Useful together with `git cat-file -e\n>         <object-name>`, for example, to see if the object is already\n>         locally available.\n>\n> ?\n\nWonderful.  Thanks for an extra set of eyes.\n"},{"id":"488682","messageId":"20240215053056.GD2821179@coredump.intra.peff.net","threadId":"60878","inReplyTo":"xmqq1q9mmtpw.fsf@gitster.g","subject":"Re: [PATCH] git: --no-lazy-fetch option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-02-15T05:30:56Z","receivedAt":"2024-02-15T05:30:57Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 08, 2024 at 03:17:31PM -0800, Junio C Hamano wrote:\n\n> Sometimes, especially during tests of low level machinery, it is\n> handy to have a way to disable lazy fetching of objects.  This\n> allows us to say, for example, \"git cat-file -e <object-name>\", to\n> see if the object is locally available.\n\nThat seems like a good feature, but...\n\n> @@ -186,6 +187,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n>  \t\t\tuse_pager = 0;\n>  \t\t\tif (envchanged)\n>  \t\t\t\t*envchanged = 1;\n> +\t\t} else if (!strcmp(cmd, \"--no-lazy-fetch\")) {\n> +\t\t\tfetch_if_missing = 0;\n>  \t\t} else if (!strcmp(cmd, \"--no-replace-objects\")) {\n>  \t\t\tdisable_replace_refs();\n>  \t\t\tsetenv(NO_REPLACE_OBJECTS_ENVIRONMENT, \"1\", 1);\n\nThis will only help builtin commands, and even then only the top-level\none. If I run \"git --no-lazy-fetch foo\" and \"foo\" is a script or an\nalias, I'd expect it to still take effect. Ditto for sub-commands kicked\noff by a builtin (say, a \"rev-list\" connectivity check caused by a\nfetch).\n\nSo this probably needs to be modeled after --no-replace-objects, etc,\nwhere we set an environment variable that makes it to child processes.\n\n-Peff\n"},{"id":"488741","messageId":"xmqqv86pslos.fsf@gitster.g","threadId":"60878","inReplyTo":"20240215053056.GD2821179@coredump.intra.peff.net","subject":"Re: [PATCH] git: --no-lazy-fetch option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-15T17:04:51Z","receivedAt":"2024-02-15T17:08:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> On Thu, Feb 08, 2024 at 03:17:31PM -0800, Junio C Hamano wrote:\n>\n>> Sometimes, especially during tests of low level machinery, it is\n>> handy to have a way to disable lazy fetching of objects.  This\n>> allows us to say, for example, \"git cat-file -e <object-name>\", to\n>> see if the object is locally available.\n>\n> That seems like a good feature, but...\n>\n>> @@ -186,6 +187,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n>>  \t\t\tuse_pager = 0;\n>>  \t\t\tif (envchanged)\n>>  \t\t\t\t*envchanged = 1;\n>> +\t\t} else if (!strcmp(cmd, \"--no-lazy-fetch\")) {\n>> +\t\t\tfetch_if_missing = 0;\n>>  \t\t} else if (!strcmp(cmd, \"--no-replace-objects\")) {\n>>  \t\t\tdisable_replace_refs();\n>>  \t\t\tsetenv(NO_REPLACE_OBJECTS_ENVIRONMENT, \"1\", 1);\n>\n> This will only help builtin commands, and even then only the top-level\n> one. If I run \"git --no-lazy-fetch foo\" and \"foo\" is a script or an\n> alias, I'd expect it to still take effect. Ditto for sub-commands kicked\n> off by a builtin (say, a \"rev-list\" connectivity check caused by a\n> fetch).\n>\n> So this probably needs to be modeled after --no-replace-objects, etc,\n> where we set an environment variable that makes it to child processes.\n\nYuck, I was hoping that we can get away with the tiny change only\nfor builtins,but you're right.\n"},{"id":"488757","messageId":"owly4je9a1gi.fsf@fine.c.googlers.com","threadId":"60878","inReplyTo":"20240215053056.GD2821179@coredump.intra.peff.net","subject":"Re: [PATCH] git: --no-lazy-fetch option","fromName":"Linus Arver","fromEmail":"linusa@google.com","sentAt":"2024-02-15T20:59:09Z","receivedAt":"2024-02-15T20:59:12Z","isPatch":true,"sender":{"key":"linus@ucla.edu","avatar":null},"body":"Jeff King <peff@peff.net> writes:\n\n> On Thu, Feb 08, 2024 at 03:17:31PM -0800, Junio C Hamano wrote:\n>\n>> Sometimes, especially during tests of low level machinery, it is\n>> handy to have a way to disable lazy fetching of objects.  This\n>> allows us to say, for example, \"git cat-file -e <object-name>\", to\n>> see if the object is locally available.\n>\n> That seems like a good feature, but...\n>\n>> @@ -186,6 +187,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n>>  \t\t\tuse_pager = 0;\n>>  \t\t\tif (envchanged)\n>>  \t\t\t\t*envchanged = 1;\n>> +\t\t} else if (!strcmp(cmd, \"--no-lazy-fetch\")) {\n>> +\t\t\tfetch_if_missing = 0;\n>>  \t\t} else if (!strcmp(cmd, \"--no-replace-objects\")) {\n>>  \t\t\tdisable_replace_refs();\n>>  \t\t\tsetenv(NO_REPLACE_OBJECTS_ENVIRONMENT, \"1\", 1);\n>\n> This will only help builtin commands, and even then only the top-level\n> one. If I run \"git --no-lazy-fetch foo\" and \"foo\" is a script or an\n> alias, I'd expect it to still take effect. Ditto for sub-commands kicked\n> off by a builtin (say, a \"rev-list\" connectivity check caused by a\n> fetch).\n>\n> So this probably needs to be modeled after --no-replace-objects, etc,\n> where we set an environment variable that makes it to child processes.\n\nThanks for the helpful explanation, very much appreciated.\n"},{"id":"488799","messageId":"xmqq1q9cl3xv.fsf@gitster.g","threadId":"60878","inReplyTo":"xmqqv86pslos.fsf@gitster.g","subject":"Re: [PATCH] git: --no-lazy-fetch option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-16T17:22:20Z","receivedAt":"2024-02-16T17:22:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Yuck, I was hoping that we can get away with the tiny change only\n> for builtins,but you're right.\n\nHere is a preliminary clean-up only for Documentation.  Will not be\nqueuing before the final, but just so that I won't forget.\n\n------- >8 ------------- >8 ------------- >8 -------\nSubject: [PATCH] git: document GIT_NO_REPLACE_OBJECTS environment variable\n\nThis variable is used as the primary way to disable the object\nreplacement mechanism, with the \"--no-replace-objects\" command line\noption as an end-user visible way to set it, but has not been\ndocumented.\n\nThe original reason why it was left undocumented might be because it\nwas meant as an internal implementation detail, but the thing is,\nthat our tests use the environment variable directly without the\ncommand line option, and there certainly are folks who learned its\nuse from there, making it impossible to deprecate or change its\nbehaviour by now.\n\nAdd documentation and note that for this variable, unlike many\nboolean-looking environment variables, only the presence matters,\nnot what value it is set to.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n Documentation/git.txt | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git c/Documentation/git.txt i/Documentation/git.txt\nindex 95f451b22b..2f1cb3ef4e 100644\n--- c/Documentation/git.txt\n+++ i/Documentation/git.txt\n@@ -174,8 +174,10 @@ If you just want to run git as if it was started in `<path>` then use\n \tdirectory.\n \n --no-replace-objects::\n-\tDo not use replacement refs to replace Git objects. See\n-\tlinkgit:git-replace[1] for more information.\n+\tDo not use replacement refs to replace Git objects.\n+\tThis is equivalent to exporting the `GIT_NO_REPLACE_OBJECTS`\n+\tenvironment variable with any value.\n+\tSee linkgit:git-replace[1] for more information.\n \n --no-lazy-fetch::\n \tDo not fetch missing objects from the promisor remote on\n"},{"id":"488812","messageId":"xmqqr0hcglpk.fsf_-_@gitster.g","threadId":"60878","inReplyTo":"xmqq1q9cl3xv.fsf@gitster.g","subject":"[PATCH] git: extend --no-lazy-fetch to work across subprocesses","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-16T21:09:43Z","receivedAt":"2024-02-16T21:09:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Modeling after how the `--no-replace-objects` option is made usable\nacross subprocess spawning (e.g., cURL based remote helpers are\nspawned as a separate process while running \"git fetch\"), allow the\n`--no-lazy-fetch` option to be passed across process boundary.\n\nDo not model how the value of GIT_NO_REPLACE_OBJECTS environment\nvariable is ignored, though.  Just use the usual git_env_bool() to\nallow \"export GIT_NO_LAZY_FETCH=0\" and \"unset GIT_NO_LAZY_FETCH\"\nto be equivalents.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n\n * And this comes on top of the original one plus the documentation\n   update for GIT_NO_REPLACE_OBJECTS.\n\n Documentation/git.txt    | 2 ++\n environment.c            | 4 ++++\n environment.h            | 1 +\n git.c                    | 3 +++\n t/t0410-partial-clone.sh | 9 +++++++++\n 5 files changed, 19 insertions(+)\n\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex 2f1cb3ef4e..be2829003d 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -183,6 +183,8 @@ If you just want to run git as if it was started in `<path>` then use\n \tDo not fetch missing objects from the promisor remote on\n \tdemand.  Useful together with `git cat-file -e <object>` to\n \tsee if the object is locally available.\n+\tThis is equivalent to setting the `GIT_NO_LAZY_FETCH`\n+\tenvironment variable to `1`.\n \n --literal-pathspecs::\n \tTreat pathspecs literally (i.e. no globbing, no pathspec magic).\ndiff --git a/environment.c b/environment.c\nindex 9e37bf58c0..afad78a3f8 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -136,6 +136,7 @@ const char * const local_repo_env[] = {\n \tGRAFT_ENVIRONMENT,\n \tINDEX_ENVIRONMENT,\n \tNO_REPLACE_OBJECTS_ENVIRONMENT,\n+\tNO_LAZY_FETCH_ENVIRONMENT,\n \tGIT_REPLACE_REF_BASE_ENVIRONMENT,\n \tGIT_PREFIX_ENVIRONMENT,\n \tGIT_SHALLOW_FILE_ENVIRONMENT,\n@@ -207,6 +208,9 @@ void setup_git_env(const char *git_dir)\n \tshallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);\n \tif (shallow_file)\n \t\tset_alternate_shallow_file(the_repository, shallow_file, 0);\n+\n+\tif (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))\n+\t\tfetch_if_missing = 0;\n }\n \n int is_bare_repository(void)\ndiff --git a/environment.h b/environment.h\nindex e5351c9dd9..74b3124f55 100644\n--- a/environment.h\n+++ b/environment.h\n@@ -35,6 +35,7 @@ const char *getenv_safe(struct strvec *argv, const char *name);\n #define EXEC_PATH_ENVIRONMENT \"GIT_EXEC_PATH\"\n #define CEILING_DIRECTORIES_ENVIRONMENT \"GIT_CEILING_DIRECTORIES\"\n #define NO_REPLACE_OBJECTS_ENVIRONMENT \"GIT_NO_REPLACE_OBJECTS\"\n+#define NO_LAZY_FETCH_ENVIRONMENT \"GIT_NO_LAZY_FETCH\"\n #define GIT_REPLACE_REF_BASE_ENVIRONMENT \"GIT_REPLACE_REF_BASE\"\n #define GITATTRIBUTES_FILE \".gitattributes\"\n #define INFOATTRIBUTES_FILE \"info/attributes\"\ndiff --git a/git.c b/git.c\nindex 28e8bf7497..d11d4dc77b 100644\n--- a/git.c\n+++ b/git.c\n@@ -189,6 +189,9 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n \t\t\t\t*envchanged = 1;\n \t\t} else if (!strcmp(cmd, \"--no-lazy-fetch\")) {\n \t\t\tfetch_if_missing = 0;\n+\t\t\tsetenv(NO_LAZY_FETCH_ENVIRONMENT, \"1\", 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n \t\t} else if (!strcmp(cmd, \"--no-replace-objects\")) {\n \t\t\tdisable_replace_refs();\n \t\t\tsetenv(NO_REPLACE_OBJECTS_ENVIRONMENT, \"1\", 1);\ndiff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh\nindex 5b7bee888d..59629cea1f 100755\n--- a/t/t0410-partial-clone.sh\n+++ b/t/t0410-partial-clone.sh\n@@ -665,6 +665,15 @@ test_expect_success 'lazy-fetch when accessing object not in the_repository' '\n \tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n \tgrep \"[?]$FILE_HASH\" out &&\n \n+\t# The no-lazy-fetch mechanism prevents Git from fetching\n+\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\tgit -C partial.git cat-file -e \"$FILE_HASH\" &&\n+\ttest_must_fail git --no-lazy-fetch -C partial.git cat-file -e \"$FILE_HASH\" &&\n+\n+\t# Sanity check that the file is still missing\n+\tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n+\tgrep \"[?]$FILE_HASH\" out &&\n+\n \tgit -C full cat-file -s \"$FILE_HASH\" >expect &&\n \ttest-tool partial-clone object-info partial.git \"$FILE_HASH\" >actual &&\n \ttest_cmp expect actual &&\n-- \n2.44.0-rc1-17-g3e0d3cd5c7\n\n"},{"id":"488819","messageId":"owlyh6i882k4.fsf@fine.c.googlers.com","threadId":"60878","inReplyTo":"xmqqr0hcglpk.fsf_-_@gitster.g","subject":"Re: [PATCH] git: extend --no-lazy-fetch to work across subprocesses","fromName":"Linus Arver","fromEmail":"linusa@google.com","sentAt":"2024-02-16T22:30:35Z","receivedAt":"2024-02-16T22:30:38Z","isPatch":true,"sender":{"key":"linus@ucla.edu","avatar":null},"body":"\nFWIW, I see some typos. Otherwise this patch along with your \"git:\ndocument GIT_NO_REPLACE_OBJECTS environment variable\" one both LGTM for\nwording/readability. I must defer to Peff and others for \"is this patch\nactually doing the right thing?\" ;).\n\nJunio C Hamano <gitster@pobox.com> writes:\n\n> Modeling after how the `--no-replace-objects` option is made usable\n> across subprocess spawning (e.g., cURL based remote helpers are\n> spawned as a separate process while running \"git fetch\"), allow the\n> `--no-lazy-fetch` option to be passed across process boundary.\n\ns/boundary/boundaries\n\n> Do not model how the value of GIT_NO_REPLACE_OBJECTS environment\n> variable is ignored, though.  Just use the usual git_env_bool() to\n> allow \"export GIT_NO_LAZY_FETCH=0\" and \"unset GIT_NO_LAZY_FETCH\"\n> to be equivalents.\n\ns/equivalents/equivalent\n"},{"id":"488821","messageId":"xmqq7cj4ggir.fsf@gitster.g","threadId":"60878","inReplyTo":"owlyh6i882k4.fsf@fine.c.googlers.com","subject":"Re: [PATCH] git: extend --no-lazy-fetch to work across subprocesses","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-16T23:01:48Z","receivedAt":"2024-02-16T23:01:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Arver <linusa@google.com> writes:\n\n> FWIW, I see some typos. Otherwise this patch along with your \"git:\n> document GIT_NO_REPLACE_OBJECTS environment variable\" one both LGTM for\n> wording/readability. I must defer to Peff and others for \"is this patch\n> actually doing the right thing?\" ;).\n>\n> Junio C Hamano <gitster@pobox.com> writes:\n>\n>> Modeling after how the `--no-replace-objects` option is made usable\n>> across subprocess spawning (e.g., cURL based remote helpers are\n>> spawned as a separate process while running \"git fetch\"), allow the\n>> `--no-lazy-fetch` option to be passed across process boundary.\n>\n> s/boundary/boundaries\n\nRight; thanks.\n\n>\n>> Do not model how the value of GIT_NO_REPLACE_OBJECTS environment\n>> variable is ignored, though.  Just use the usual git_env_bool() to\n>> allow \"export GIT_NO_LAZY_FETCH=0\" and \"unset GIT_NO_LAZY_FETCH\"\n>> to be equivalents.\n>\n> s/equivalents/equivalent\n\nI meant to say that these two are \"equivalents\" (noun, plural).\n\nI can rephrase to\n\n\tJust use git_env_bool() to make \"export GIT_NO_LAZY_FETCH=0\"\n\tan equivalent to \"unset GIT_NO_LAZY_FETCH\".\n\nthough, of course.\n"},{"id":"488832","messageId":"owlybk8g80lr.fsf@fine.c.googlers.com","threadId":"60878","inReplyTo":"xmqq7cj4ggir.fsf@gitster.g","subject":"Re: [PATCH] git: extend --no-lazy-fetch to work across subprocesses","fromName":"Linus Arver","fromEmail":"linusa@google.com","sentAt":"2024-02-16T23:12:48Z","receivedAt":"2024-02-16T23:12:50Z","isPatch":true,"sender":{"key":"linus@ucla.edu","avatar":null},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n>>> Do not model how the value of GIT_NO_REPLACE_OBJECTS environment\n>>> variable is ignored, though.  Just use the usual git_env_bool() to\n>>> allow \"export GIT_NO_LAZY_FETCH=0\" and \"unset GIT_NO_LAZY_FETCH\"\n>>> to be equivalents.\n>>\n>> s/equivalents/equivalent\n>\n> I meant to say that these two are \"equivalents\" (noun, plural).\n\nAh, I see.\n\n> I can rephrase to\n>\n> \tJust use git_env_bool() to make \"export GIT_NO_LAZY_FETCH=0\"\n> \tan equivalent to \"unset GIT_NO_LAZY_FETCH\".\n>\n> though, of course.\n\nSGTM either way. Thanks.\n"},{"id":"488840","messageId":"20240217052932.GC539459@coredump.intra.peff.net","threadId":"60878","inReplyTo":"xmqq1q9cl3xv.fsf@gitster.g","subject":"Re: [PATCH] git: --no-lazy-fetch option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-02-17T05:29:32Z","receivedAt":"2024-02-17T05:29:34Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Feb 16, 2024 at 09:22:20AM -0800, Junio C Hamano wrote:\n\n> Here is a preliminary clean-up only for Documentation.  Will not be\n> queuing before the final, but just so that I won't forget.\n\nI think this is an improvement, but two small comments...\n\n> ------- >8 ------------- >8 ------------- >8 -------\n> Subject: [PATCH] git: document GIT_NO_REPLACE_OBJECTS environment variable\n> \n> This variable is used as the primary way to disable the object\n> replacement mechanism, with the \"--no-replace-objects\" command line\n> option as an end-user visible way to set it, but has not been\n> documented.\n> \n> The original reason why it was left undocumented might be because it\n> was meant as an internal implementation detail, but the thing is,\n> that our tests use the environment variable directly without the\n> command line option, and there certainly are folks who learned its\n> use from there, making it impossible to deprecate or change its\n> behaviour by now.\n\nI agree that we should document these sorts of variables; they really\nare part of the public interface since it's so easy for users to see\nthem in their own scripts, aliases, etc, when we set them.\n\nBut in fact do document this environment variable in git-replace(1), and\nhave for a long time. So I don't think we need to even make claims about\nits undocumented state. :)\n\n>  --no-replace-objects::\n> -\tDo not use replacement refs to replace Git objects. See\n> -\tlinkgit:git-replace[1] for more information.\n> +\tDo not use replacement refs to replace Git objects.\n> +\tThis is equivalent to exporting the `GIT_NO_REPLACE_OBJECTS`\n> +\tenvironment variable with any value.\n> +\tSee linkgit:git-replace[1] for more information.\n\nThe text both before and after your patch links to git-replace, which\ncovers the variable. So I think the \"before\" state is actually not that\nbad. But I still think mentioning it explicitly is better still.\n\nHowever, should it get an entry in the \"ENVIRONMENT VARIABLES\" section?\nWe cover stuff there like GIT_LITERAL_PATHSPECS, which is triggered in\nthe same way.\n\n> Add documentation and note that for this variable, unlike many\n> boolean-looking environment variables, only the presence matters,\n> not what value it is set to.\n\nYuck. IMHO depending on GIT_NO_REPLACE_OBJECTS=0 is somewhat crazy, and\nwe could consider the current behavior a bug. It's probably not _that_\nbig a deal either way (because I would not expect anybody to set it that\nway in the first place). But I wonder if being consistent across\nvariables trumps retaining weird historical compatibility for such a\nfar-fetched case. I dunno. I guess this is https://xkcd.com/1172/. :)\n\n-Peff\n"},{"id":"488841","messageId":"20240217054048.GD539459@coredump.intra.peff.net","threadId":"60878","inReplyTo":"xmqqr0hcglpk.fsf_-_@gitster.g","subject":"Re: [PATCH] git: extend --no-lazy-fetch to work across subprocesses","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-02-17T05:40:48Z","receivedAt":"2024-02-17T05:40:50Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Feb 16, 2024 at 01:09:43PM -0800, Junio C Hamano wrote:\n\n> Modeling after how the `--no-replace-objects` option is made usable\n> across subprocess spawning (e.g., cURL based remote helpers are\n> spawned as a separate process while running \"git fetch\"), allow the\n> `--no-lazy-fetch` option to be passed across process boundary.\n> \n> Do not model how the value of GIT_NO_REPLACE_OBJECTS environment\n> variable is ignored, though.  Just use the usual git_env_bool() to\n> allow \"export GIT_NO_LAZY_FETCH=0\" and \"unset GIT_NO_LAZY_FETCH\"\n> to be equivalents.\n\nNice. I wondered if we would need to hide fetch_if_missing behind a\nfunction that lazy-evaluates the environment variable, but it looks like\nsetup_git_env() is central enough to cover us here.\n\n> diff --git a/Documentation/git.txt b/Documentation/git.txt\n> index 2f1cb3ef4e..be2829003d 100644\n> --- a/Documentation/git.txt\n> +++ b/Documentation/git.txt\n> @@ -183,6 +183,8 @@ If you just want to run git as if it was started in `<path>` then use\n>  \tDo not fetch missing objects from the promisor remote on\n>  \tdemand.  Useful together with `git cat-file -e <object>` to\n>  \tsee if the object is locally available.\n> +\tThis is equivalent to setting the `GIT_NO_LAZY_FETCH`\n> +\tenvironment variable to `1`.\n\nAs with the other patch, I'd suggest adding an entry to the list of\nenvironment variables later in the manpage.\n\n> --- a/environment.h\n> +++ b/environment.h\n> @@ -35,6 +35,7 @@ const char *getenv_safe(struct strvec *argv, const char *name);\n>  #define EXEC_PATH_ENVIRONMENT \"GIT_EXEC_PATH\"\n>  #define CEILING_DIRECTORIES_ENVIRONMENT \"GIT_CEILING_DIRECTORIES\"\n>  #define NO_REPLACE_OBJECTS_ENVIRONMENT \"GIT_NO_REPLACE_OBJECTS\"\n> +#define NO_LAZY_FETCH_ENVIRONMENT \"GIT_NO_LAZY_FETCH\"\n>  #define GIT_REPLACE_REF_BASE_ENVIRONMENT \"GIT_REPLACE_REF_BASE\"\n>  #define GITATTRIBUTES_FILE \".gitattributes\"\n>  #define INFOATTRIBUTES_FILE \"info/attributes\"\n\nA small nit, but maybe worth keeping the two replace-related variables\nnext to each other, rather than sticking the new one in the middle?\n\n> diff --git a/git.c b/git.c\n> index 28e8bf7497..d11d4dc77b 100644\n> --- a/git.c\n> +++ b/git.c\n> @@ -189,6 +189,9 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n>  \t\t\t\t*envchanged = 1;\n>  \t\t} else if (!strcmp(cmd, \"--no-lazy-fetch\")) {\n>  \t\t\tfetch_if_missing = 0;\n> +\t\t\tsetenv(NO_LAZY_FETCH_ENVIRONMENT, \"1\", 1);\n> +\t\t\tif (envchanged)\n> +\t\t\t\t*envchanged = 1;\n>  \t\t} else if (!strcmp(cmd, \"--no-replace-objects\")) {\n>  \t\t\tdisable_replace_refs();\n>  \t\t\tsetenv(NO_REPLACE_OBJECTS_ENVIRONMENT, \"1\", 1);\n\nI _suspect_ this makes the fetch_if_missing call redundant, since we\nshould be parsing these before doing any repo setup that would trigger\nthe code that reads the environment variable.\n\nThis should probably also be xsetenv(), though as you can see in the\ncontext we are not very consistent about using it. :) But certainly if\nwe failed to set it I would prefer to see an error rather than\naccidentally lazy-fetching.\n\n> diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh\n> index 5b7bee888d..59629cea1f 100755\n> --- a/t/t0410-partial-clone.sh\n> +++ b/t/t0410-partial-clone.sh\n> @@ -665,6 +665,15 @@ test_expect_success 'lazy-fetch when accessing object not in the_repository' '\n>  \tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n>  \tgrep \"[?]$FILE_HASH\" out &&\n>  \n> +\t# The no-lazy-fetch mechanism prevents Git from fetching\n> +\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n> +\t\tgit -C partial.git cat-file -e \"$FILE_HASH\" &&\n> +\ttest_must_fail git --no-lazy-fetch -C partial.git cat-file -e \"$FILE_HASH\" &&\n> +\n> +\t# Sanity check that the file is still missing\n> +\tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n> +\tgrep \"[?]$FILE_HASH\" out &&\n\nOK, we exercise it by setting the variable directly. A more interesting\none might be:\n\n  git -c alias.foo='!git cat-file' --no-lazy-fetch ...\n\nwhich should fail without the patch.\n\nI also wondered why we were testing the direct-builtin invocation here,\nsince we are building on top of the original --no-lazy-fetch patch, but\nit looks like it is because that patch didn't add any tests at all). I\nthink they would make more sense as a single patch, but I guess the\noriginal is already in 'next' (though I suppose it is about to be\nrewound). I am OK with it either way.\n\n-Peff\n"},{"id":"489459","messageId":"xmqqfrxexx15.fsf@gitster.g","threadId":"60878","inReplyTo":"20240217054048.GD539459@coredump.intra.peff.net","subject":"Re: [PATCH] git: extend --no-lazy-fetch to work across subprocesses","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-27T06:04:54Z","receivedAt":"2024-02-27T06:04:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n>> diff --git a/Documentation/git.txt b/Documentation/git.txt\n>> index 2f1cb3ef4e..be2829003d 100644\n>> --- a/Documentation/git.txt\n>> +++ b/Documentation/git.txt\n>> @@ -183,6 +183,8 @@ If you just want to run git as if it was started in `<path>` then use\n>>  \tDo not fetch missing objects from the promisor remote on\n>>  \tdemand.  Useful together with `git cat-file -e <object>` to\n>>  \tsee if the object is locally available.\n>> +\tThis is equivalent to setting the `GIT_NO_LAZY_FETCH`\n>> +\tenvironment variable to `1`.\n>\n> As with the other patch, I'd suggest adding an entry to the list of\n> environment variables later in the manpage.\n\nThanks, done.\n\nWe'll have to make a separate patch to add GIT_NO_REPLACE_OBJECTS to\nthe list, by the way (#leftoverbits), which I used as a model to\nfind what needs to be updated.\n\n>> --- a/environment.h\n>> +++ b/environment.h\n>> @@ -35,6 +35,7 @@ const char *getenv_safe(struct strvec *argv, const char *name);\n>>  #define EXEC_PATH_ENVIRONMENT \"GIT_EXEC_PATH\"\n>>  #define CEILING_DIRECTORIES_ENVIRONMENT \"GIT_CEILING_DIRECTORIES\"\n>>  #define NO_REPLACE_OBJECTS_ENVIRONMENT \"GIT_NO_REPLACE_OBJECTS\"\n>> +#define NO_LAZY_FETCH_ENVIRONMENT \"GIT_NO_LAZY_FETCH\"\n>>  #define GIT_REPLACE_REF_BASE_ENVIRONMENT \"GIT_REPLACE_REF_BASE\"\n>>  #define GITATTRIBUTES_FILE \".gitattributes\"\n>>  #define INFOATTRIBUTES_FILE \"info/attributes\"\n>\n> A small nit, but maybe worth keeping the two replace-related variables\n> next to each other, rather than sticking the new one in the middle?\n\nOK.\n\n>> diff --git a/git.c b/git.c\n>> index 28e8bf7497..d11d4dc77b 100644\n>> --- a/git.c\n>> +++ b/git.c\n>> @@ -189,6 +189,9 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n>>  \t\t\t\t*envchanged = 1;\n>>  \t\t} else if (!strcmp(cmd, \"--no-lazy-fetch\")) {\n>>  \t\t\tfetch_if_missing = 0;\n>> +\t\t\tsetenv(NO_LAZY_FETCH_ENVIRONMENT, \"1\", 1);\n>> +\t\t\tif (envchanged)\n>> +\t\t\t\t*envchanged = 1;\n>>  \t\t} else if (!strcmp(cmd, \"--no-replace-objects\")) {\n>>  \t\t\tdisable_replace_refs();\n>>  \t\t\tsetenv(NO_REPLACE_OBJECTS_ENVIRONMENT, \"1\", 1);\n>\n> I _suspect_ this makes the fetch_if_missing call redundant, since we\n> should be parsing these before doing any repo setup that would trigger\n> the code that reads the environment variable.\n\nTrue.  Again, we'd need to clean-up the NO_REPLACE_OBJECTS codepath\nas well---the disable_replace_refs() call should also be redundant,\nwhich I mimicked to add the no-lazy-fetch codepath (#leftoverbits).\n\n> This should probably also be xsetenv(), though as you can see in the\n> context we are not very consistent about using it. :) But certainly if\n> we failed to set it I would prefer to see an error rather than\n> accidentally lazy-fetching.\n\nI'd probably leave it outside the scope of this series; as nobody\nuses xsetenv() in git.c currently, it would be a clean-up topic on\nits own (#leftoverbits).\n\n>> diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh\n>> index 5b7bee888d..59629cea1f 100755\n>> --- a/t/t0410-partial-clone.sh\n>> +++ b/t/t0410-partial-clone.sh\n>> @@ -665,6 +665,15 @@ test_expect_success 'lazy-fetch when accessing object not in the_repository' '\n>>  \tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n>>  \tgrep \"[?]$FILE_HASH\" out &&\n>>  \n>> +\t# The no-lazy-fetch mechanism prevents Git from fetching\n>> +\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n>> +\t\tgit -C partial.git cat-file -e \"$FILE_HASH\" &&\n>> +\ttest_must_fail git --no-lazy-fetch -C partial.git cat-file -e \"$FILE_HASH\" &&\n>> +\n>> +\t# Sanity check that the file is still missing\n>> +\tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n>> +\tgrep \"[?]$FILE_HASH\" out &&\n>\n> OK, we exercise it by setting the variable directly. A more interesting\n> one might be:\n>\n>   git -c alias.foo='!git cat-file' --no-lazy-fetch ...\n>\n> which should fail without the patch.\n\nTrue, again.  I'll test all three variants (i.e. environment,\ncommand line option to \"git\", forced subprocess turning command line\noption to \"git\" into environment).\n\nThanks.\n\n\n\n----- >8 --------- >8 --------- >8 --------- >8 -----\nSubject: [PATCH v2 3/3] git: extend --no-lazy-fetch to work across subprocesses\n\nModeling after how the `--no-replace-objects` option is made usable\nacross subprocess spawning (e.g., cURL based remote helpers are\nspawned as a separate process while running \"git fetch\"), allow the\n`--no-lazy-fetch` option to be passed across process boundaries.\n\nDo not model how the value of GIT_NO_REPLACE_OBJECTS environment\nvariable is ignored, though.  Just use the usual git_env_bool() to\nallow \"export GIT_NO_LAZY_FETCH=0\" and \"unset GIT_NO_LAZY_FETCH\"\nto be equivalents.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\nRange-diff:\n1:  00e202b679 ! 1:  e0764f2e21 git: extend --no-lazy-fetch to work across subprocesses\n    @@ Documentation/git.txt: If you just want to run git as if it was started in `<pat\n      \n      --literal-pathspecs::\n      \tTreat pathspecs literally (i.e. no globbing, no pathspec magic).\n    +@@ Documentation/git.txt: for full details.\n    + \tSetting this Boolean environment variable to true will cause Git to treat all\n    + \tpathspecs as case-insensitive.\n    + \n    ++`GIT_NO_LAZY_FETCH`::\n    ++\tSetting this Boolean environment variable to true tells Git\n    ++\tnot to lazily fetch missing objects from the promisor remote\n    ++\ton demand.\n    ++\n    + `GIT_REFLOG_ACTION`::\n    + \tWhen a ref is updated, reflog entries are created to keep\n    + \ttrack of the reason why the ref was updated (which is\n     \n      ## environment.c ##\n     @@ environment.c: const char * const local_repo_env[] = {\n    @@ environment.c: void setup_git_env(const char *git_dir)\n     \n      ## environment.h ##\n     @@ environment.h: const char *getenv_safe(struct strvec *argv, const char *name);\n    - #define EXEC_PATH_ENVIRONMENT \"GIT_EXEC_PATH\"\n      #define CEILING_DIRECTORIES_ENVIRONMENT \"GIT_CEILING_DIRECTORIES\"\n      #define NO_REPLACE_OBJECTS_ENVIRONMENT \"GIT_NO_REPLACE_OBJECTS\"\n    -+#define NO_LAZY_FETCH_ENVIRONMENT \"GIT_NO_LAZY_FETCH\"\n      #define GIT_REPLACE_REF_BASE_ENVIRONMENT \"GIT_REPLACE_REF_BASE\"\n    ++#define NO_LAZY_FETCH_ENVIRONMENT \"GIT_NO_LAZY_FETCH\"\n      #define GITATTRIBUTES_FILE \".gitattributes\"\n      #define INFOATTRIBUTES_FILE \"info/attributes\"\n    + #define ATTRIBUTE_MACRO_PREFIX \"[attr]\"\n     \n      ## git.c ##\n     @@ git.c: static int handle_options(const char ***argv, int *argc, int *envchanged)\n    @@ t/t0410-partial-clone.sh: test_expect_success 'lazy-fetch when accessing object\n     +\t# The no-lazy-fetch mechanism prevents Git from fetching\n     +\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n     +\t\tgit -C partial.git cat-file -e \"$FILE_HASH\" &&\n    ++\n    ++\t# The same with command line option to \"git\"\n     +\ttest_must_fail git --no-lazy-fetch -C partial.git cat-file -e \"$FILE_HASH\" &&\n     +\n    ++\t# The same, forcing a subprocess via an alias\n    ++\ttest_must_fail git --no-lazy-fetch -C partial.git \\\n    ++\t\t-c alias.foo=\"!git cat-file\" foo -e \"$FILE_HASH\" &&\n    ++\n     +\t# Sanity check that the file is still missing\n     +\tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n     +\tgrep \"[?]$FILE_HASH\" out &&\n\n Documentation/git.txt    |  7 +++++++\n environment.c            |  4 ++++\n environment.h            |  1 +\n git.c                    |  3 +++\n t/t0410-partial-clone.sh | 15 +++++++++++++++\n 5 files changed, 30 insertions(+)\n\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex 2f1cb3ef4e..6fbaa9dd2b 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -183,6 +183,8 @@ If you just want to run git as if it was started in `<path>` then use\n \tDo not fetch missing objects from the promisor remote on\n \tdemand.  Useful together with `git cat-file -e <object>` to\n \tsee if the object is locally available.\n+\tThis is equivalent to setting the `GIT_NO_LAZY_FETCH`\n+\tenvironment variable to `1`.\n \n --literal-pathspecs::\n \tTreat pathspecs literally (i.e. no globbing, no pathspec magic).\n@@ -896,6 +898,11 @@ for full details.\n \tSetting this Boolean environment variable to true will cause Git to treat all\n \tpathspecs as case-insensitive.\n \n+`GIT_NO_LAZY_FETCH`::\n+\tSetting this Boolean environment variable to true tells Git\n+\tnot to lazily fetch missing objects from the promisor remote\n+\ton demand.\n+\n `GIT_REFLOG_ACTION`::\n \tWhen a ref is updated, reflog entries are created to keep\n \ttrack of the reason why the ref was updated (which is\ndiff --git a/environment.c b/environment.c\nindex 9e37bf58c0..afad78a3f8 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -136,6 +136,7 @@ const char * const local_repo_env[] = {\n \tGRAFT_ENVIRONMENT,\n \tINDEX_ENVIRONMENT,\n \tNO_REPLACE_OBJECTS_ENVIRONMENT,\n+\tNO_LAZY_FETCH_ENVIRONMENT,\n \tGIT_REPLACE_REF_BASE_ENVIRONMENT,\n \tGIT_PREFIX_ENVIRONMENT,\n \tGIT_SHALLOW_FILE_ENVIRONMENT,\n@@ -207,6 +208,9 @@ void setup_git_env(const char *git_dir)\n \tshallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);\n \tif (shallow_file)\n \t\tset_alternate_shallow_file(the_repository, shallow_file, 0);\n+\n+\tif (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))\n+\t\tfetch_if_missing = 0;\n }\n \n int is_bare_repository(void)\ndiff --git a/environment.h b/environment.h\nindex e5351c9dd9..5cec19cecc 100644\n--- a/environment.h\n+++ b/environment.h\n@@ -36,6 +36,7 @@ const char *getenv_safe(struct strvec *argv, const char *name);\n #define CEILING_DIRECTORIES_ENVIRONMENT \"GIT_CEILING_DIRECTORIES\"\n #define NO_REPLACE_OBJECTS_ENVIRONMENT \"GIT_NO_REPLACE_OBJECTS\"\n #define GIT_REPLACE_REF_BASE_ENVIRONMENT \"GIT_REPLACE_REF_BASE\"\n+#define NO_LAZY_FETCH_ENVIRONMENT \"GIT_NO_LAZY_FETCH\"\n #define GITATTRIBUTES_FILE \".gitattributes\"\n #define INFOATTRIBUTES_FILE \"info/attributes\"\n #define ATTRIBUTE_MACRO_PREFIX \"[attr]\"\ndiff --git a/git.c b/git.c\nindex 28e8bf7497..d11d4dc77b 100644\n--- a/git.c\n+++ b/git.c\n@@ -189,6 +189,9 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n \t\t\t\t*envchanged = 1;\n \t\t} else if (!strcmp(cmd, \"--no-lazy-fetch\")) {\n \t\t\tfetch_if_missing = 0;\n+\t\t\tsetenv(NO_LAZY_FETCH_ENVIRONMENT, \"1\", 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n \t\t} else if (!strcmp(cmd, \"--no-replace-objects\")) {\n \t\t\tdisable_replace_refs();\n \t\t\tsetenv(NO_REPLACE_OBJECTS_ENVIRONMENT, \"1\", 1);\ndiff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh\nindex 5b7bee888d..c282851af7 100755\n--- a/t/t0410-partial-clone.sh\n+++ b/t/t0410-partial-clone.sh\n@@ -665,6 +665,21 @@ test_expect_success 'lazy-fetch when accessing object not in the_repository' '\n \tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n \tgrep \"[?]$FILE_HASH\" out &&\n \n+\t# The no-lazy-fetch mechanism prevents Git from fetching\n+\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\tgit -C partial.git cat-file -e \"$FILE_HASH\" &&\n+\n+\t# The same with command line option to \"git\"\n+\ttest_must_fail git --no-lazy-fetch -C partial.git cat-file -e \"$FILE_HASH\" &&\n+\n+\t# The same, forcing a subprocess via an alias\n+\ttest_must_fail git --no-lazy-fetch -C partial.git \\\n+\t\t-c alias.foo=\"!git cat-file\" foo -e \"$FILE_HASH\" &&\n+\n+\t# Sanity check that the file is still missing\n+\tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n+\tgrep \"[?]$FILE_HASH\" out &&\n+\n \tgit -C full cat-file -s \"$FILE_HASH\" >expect &&\n \ttest-tool partial-clone object-info partial.git \"$FILE_HASH\" >actual &&\n \ttest_cmp expect actual &&\n-- \n2.44.0-35-ga2082dbdd3\n\n"},{"id":"489465","messageId":"20240227074903.GD3263678@coredump.intra.peff.net","threadId":"60878","inReplyTo":"xmqqfrxexx15.fsf@gitster.g","subject":"Re: [PATCH] git: extend --no-lazy-fetch to work across subprocesses","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-02-27T07:49:03Z","receivedAt":"2024-02-27T07:49:10Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Feb 26, 2024 at 10:04:54PM -0800, Junio C Hamano wrote:\n\n> ----- >8 --------- >8 --------- >8 --------- >8 -----\n> Subject: [PATCH v2 3/3] git: extend --no-lazy-fetch to work across subprocesses\n> [...]\n\nThis looks pretty reasonable to me, and the lines you drew for\n#leftoverbits all seemed liked good spots.\n\nThe only thing I noticed in the patch was this (which I think is not\neven new in this round):\n\n> diff --git a/environment.c b/environment.c\n> index 9e37bf58c0..afad78a3f8 100644\n> --- a/environment.c\n> +++ b/environment.c\n> @@ -136,6 +136,7 @@ const char * const local_repo_env[] = {\n>  \tGRAFT_ENVIRONMENT,\n>  \tINDEX_ENVIRONMENT,\n>  \tNO_REPLACE_OBJECTS_ENVIRONMENT,\n> +\tNO_LAZY_FETCH_ENVIRONMENT,\n>  \tGIT_REPLACE_REF_BASE_ENVIRONMENT,\n>  \tGIT_PREFIX_ENVIRONMENT,\n>  \tGIT_SHALLOW_FILE_ENVIRONMENT,\n\nThis will clear the environment variable when we move between\nrepositories. I can see an argument for it, and certainly that's how\nGIT_NO_REPLACE_OBJECTS works.\n\nBut I can also see an argument that this is not a property of the\n_repository_, but of the request. For example, if I run \"git\n--no-lazy-fetch log\" and we cross into a submodule to do a diff, should\nthat submodule also avoid lazy-fetching? I'd think so, but I think your\npatch would restore the defaults when we \"enter\" the submodule repo.\n\nThere's some prior art there, I think, in how GIT_CEILING_DIRECTORIES\nworks, or even something like \"git --literal-pathspecs\", neither of\nwhich appear in local_repo_env.\n\n-Peff\n"},{"id":"489543","messageId":"xmqq1q8xx38i.fsf@gitster.g","threadId":"60878","inReplyTo":"20240227074903.GD3263678@coredump.intra.peff.net","subject":"Re: [PATCH] git: extend --no-lazy-fetch to work across subprocesses","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-02-27T16:48:29Z","receivedAt":"2024-02-27T16:48:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> There's some prior art there, I think, in how GIT_CEILING_DIRECTORIES\n> works, or even something like \"git --literal-pathspecs\", neither of\n> which appear in local_repo_env.\n\nUnlike GIT_CEILING_DIRECTORIES that is more or less permanent and a\npart of configuring an everyday environment for real work, I view\nthis --no-lazy-fetch thing as solely a debugging aid.  A repository\nwith promisor wouldn't be able to \"fill the gap\" by forbidding\non-demand fetching of promised objects while running say \"git fetch\"\nfrom elsewhere and it just will die with \"some objects we are\nsupposed to have are missing from this repository\".\n\nSo I do not have a strong opinion either way, if it is more\nconvenient to propagate the request out to other repositories when\nwe run processes in two or more repositories (e.g. \"git clone\n--local\"), or if it is more convenient to make sure that the request\nis limited to the target repository.  Here is a version without the\nlocal_repo_env[] change.\n\n----- >8 --------- >8 --------- >8 --------- >8 --------- >8 -----\nSubject: [PATCH v3 3/3] git: extend --no-lazy-fetch to work across subprocesses\n\nModeling after how the `--no-replace-objects` option is made usable\nacross subprocess spawning (e.g., cURL based remote helpers are\nspawned as a separate process while running \"git fetch\"), allow the\n`--no-lazy-fetch` option to be passed across process boundaries.\n\nDo not model how the value of GIT_NO_REPLACE_OBJECTS environment\nvariable is ignored, though.  Just use the usual git_env_bool() to\nallow \"export GIT_NO_LAZY_FETCH=0\" and \"unset GIT_NO_LAZY_FETCH\"\nto be equivalents.\n\nAlso do not model how the request is not propagated to subprocesses\nwe spawn (e.g. \"git clone --local\" that spawns a new process to work\nin the origin repository, while the original one working in the\nnewly created one) by the \"--no-replace-objects\" option, as this \"do\nnot lazily fetch from the promisor\" is more about a per-request\ndebugging aid, not \"this repository's promisor should not be relied\nupon\" property specific to a repository.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\nRange-diff:\n1:  5fa7ddbf68 ! 1:  ea40bb9a1d git: extend --no-lazy-fetch to work across subprocesses\n    @@ Commit message\n         allow \"export GIT_NO_LAZY_FETCH=0\" and \"unset GIT_NO_LAZY_FETCH\"\n         to be equivalents.\n     \n    +    Also do not model how the request is not propagated to subprocesses\n    +    we spawn (e.g. \"git clone --local\" that spawns a new process to work\n    +    in the origin repository, while the original one working in the\n    +    newly created one) by the \"--no-replace-objects\" option, as this \"do\n    +    not lazily fetch from the promisor\" is more about a per-request\n    +    debugging aid, not \"this repository's promisor should not be relied\n    +    upon\" property specific to a repository.\n    +\n         Signed-off-by: Junio C Hamano <gitster@pobox.com>\n     \n      ## Documentation/git.txt ##\n    @@ Documentation/git.txt: for full details.\n      \ttrack of the reason why the ref was updated (which is\n     \n      ## environment.c ##\n    -@@ environment.c: const char * const local_repo_env[] = {\n    - \tGRAFT_ENVIRONMENT,\n    - \tINDEX_ENVIRONMENT,\n    - \tNO_REPLACE_OBJECTS_ENVIRONMENT,\n    -+\tNO_LAZY_FETCH_ENVIRONMENT,\n    - \tGIT_REPLACE_REF_BASE_ENVIRONMENT,\n    - \tGIT_PREFIX_ENVIRONMENT,\n    - \tGIT_SHALLOW_FILE_ENVIRONMENT,\n     @@ environment.c: void setup_git_env(const char *git_dir)\n      \tshallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);\n      \tif (shallow_file)\n\n Documentation/git.txt    |  7 +++++++\n environment.c            |  3 +++\n environment.h            |  1 +\n git.c                    |  3 +++\n t/t0410-partial-clone.sh | 15 +++++++++++++++\n 5 files changed, 29 insertions(+)\n\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex b1f754e84b..a517d94a7a 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -183,6 +183,8 @@ If you just want to run git as if it was started in `<path>` then use\n \tDo not fetch missing objects from the promisor remote on\n \tdemand.  Useful together with `git cat-file -e <object>` to\n \tsee if the object is locally available.\n+\tThis is equivalent to setting the `GIT_NO_LAZY_FETCH`\n+\tenvironment variable to `1`.\n \n --literal-pathspecs::\n \tTreat pathspecs literally (i.e. no globbing, no pathspec magic).\n@@ -900,6 +902,11 @@ for full details.\n \tSetting this Boolean environment variable to true will cause Git to treat all\n \tpathspecs as case-insensitive.\n \n+`GIT_NO_LAZY_FETCH`::\n+\tSetting this Boolean environment variable to true tells Git\n+\tnot to lazily fetch missing objects from the promisor remote\n+\ton demand.\n+\n `GIT_REFLOG_ACTION`::\n \tWhen a ref is updated, reflog entries are created to keep\n \ttrack of the reason why the ref was updated (which is\ndiff --git a/environment.c b/environment.c\nindex 9e37bf58c0..0ae5bbd4a1 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -207,6 +207,9 @@ void setup_git_env(const char *git_dir)\n \tshallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);\n \tif (shallow_file)\n \t\tset_alternate_shallow_file(the_repository, shallow_file, 0);\n+\n+\tif (git_env_bool(NO_LAZY_FETCH_ENVIRONMENT, 0))\n+\t\tfetch_if_missing = 0;\n }\n \n int is_bare_repository(void)\ndiff --git a/environment.h b/environment.h\nindex e5351c9dd9..5cec19cecc 100644\n--- a/environment.h\n+++ b/environment.h\n@@ -36,6 +36,7 @@ const char *getenv_safe(struct strvec *argv, const char *name);\n #define CEILING_DIRECTORIES_ENVIRONMENT \"GIT_CEILING_DIRECTORIES\"\n #define NO_REPLACE_OBJECTS_ENVIRONMENT \"GIT_NO_REPLACE_OBJECTS\"\n #define GIT_REPLACE_REF_BASE_ENVIRONMENT \"GIT_REPLACE_REF_BASE\"\n+#define NO_LAZY_FETCH_ENVIRONMENT \"GIT_NO_LAZY_FETCH\"\n #define GITATTRIBUTES_FILE \".gitattributes\"\n #define INFOATTRIBUTES_FILE \"info/attributes\"\n #define ATTRIBUTE_MACRO_PREFIX \"[attr]\"\ndiff --git a/git.c b/git.c\nindex 28e8bf7497..d11d4dc77b 100644\n--- a/git.c\n+++ b/git.c\n@@ -189,6 +189,9 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n \t\t\t\t*envchanged = 1;\n \t\t} else if (!strcmp(cmd, \"--no-lazy-fetch\")) {\n \t\t\tfetch_if_missing = 0;\n+\t\t\tsetenv(NO_LAZY_FETCH_ENVIRONMENT, \"1\", 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n \t\t} else if (!strcmp(cmd, \"--no-replace-objects\")) {\n \t\t\tdisable_replace_refs();\n \t\t\tsetenv(NO_REPLACE_OBJECTS_ENVIRONMENT, \"1\", 1);\ndiff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh\nindex 5b7bee888d..c282851af7 100755\n--- a/t/t0410-partial-clone.sh\n+++ b/t/t0410-partial-clone.sh\n@@ -665,6 +665,21 @@ test_expect_success 'lazy-fetch when accessing object not in the_repository' '\n \tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n \tgrep \"[?]$FILE_HASH\" out &&\n \n+\t# The no-lazy-fetch mechanism prevents Git from fetching\n+\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\tgit -C partial.git cat-file -e \"$FILE_HASH\" &&\n+\n+\t# The same with command line option to \"git\"\n+\ttest_must_fail git --no-lazy-fetch -C partial.git cat-file -e \"$FILE_HASH\" &&\n+\n+\t# The same, forcing a subprocess via an alias\n+\ttest_must_fail git --no-lazy-fetch -C partial.git \\\n+\t\t-c alias.foo=\"!git cat-file\" foo -e \"$FILE_HASH\" &&\n+\n+\t# Sanity check that the file is still missing\n+\tgit -C partial.git rev-list --objects --missing=print HEAD >out &&\n+\tgrep \"[?]$FILE_HASH\" out &&\n+\n \tgit -C full cat-file -s \"$FILE_HASH\" >expect &&\n \ttest-tool partial-clone object-info partial.git \"$FILE_HASH\" >actual &&\n \ttest_cmp expect actual &&\n-- \n2.44.0-35-ga2082dbdd3\n\n\n"},{"id":"490167","messageId":"20240307095638.GC2650063@coredump.intra.peff.net","threadId":"60878","inReplyTo":"xmqq1q8xx38i.fsf@gitster.g","subject":"Re: [PATCH] git: extend --no-lazy-fetch to work across subprocesses","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-03-07T09:56:38Z","receivedAt":"2024-03-07T09:56:40Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Feb 27, 2024 at 08:48:29AM -0800, Junio C Hamano wrote:\n\n> Jeff King <peff@peff.net> writes:\n> \n> > There's some prior art there, I think, in how GIT_CEILING_DIRECTORIES\n> > works, or even something like \"git --literal-pathspecs\", neither of\n> > which appear in local_repo_env.\n> \n> Unlike GIT_CEILING_DIRECTORIES that is more or less permanent and a\n> part of configuring an everyday environment for real work, I view\n> this --no-lazy-fetch thing as solely a debugging aid.  A repository\n> with promisor wouldn't be able to \"fill the gap\" by forbidding\n> on-demand fetching of promised objects while running say \"git fetch\"\n> from elsewhere and it just will die with \"some objects we are\n> supposed to have are missing from this repository\".\n> \n> So I do not have a strong opinion either way, if it is more\n> convenient to propagate the request out to other repositories when\n> we run processes in two or more repositories (e.g. \"git clone\n> --local\"), or if it is more convenient to make sure that the request\n> is limited to the target repository.  Here is a version without the\n> local_repo_env[] change.\n\nYeah, GIT_CEILING_DIRECTORIES is maybe a bad example. But I do think\nLITERAL_PATHSPECS is a better one, and the submodule-fetch example I\ngave would be genuinely surprising if it behaved differently than the\nsuperproject, I'd think.\n\nI do agree this is probably going to mostly be a debugging aid, so it\nmight not matter much. But once in the wild these things tend to take on\na life of their own. ;)\n\n> ----- >8 --------- >8 --------- >8 --------- >8 --------- >8 -----\n> Subject: [PATCH v3 3/3] git: extend --no-lazy-fetch to work across subprocesses\n\nSo anyway, this version seems good to me.\n\n-Peff\n"},{"id":"490212","messageId":"xmqqjzmdlr43.fsf@gitster.g","threadId":"60878","inReplyTo":"20240307095638.GC2650063@coredump.intra.peff.net","subject":"Re: [PATCH] git: extend --no-lazy-fetch to work across subprocesses","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-03-07T20:33:00Z","receivedAt":"2024-03-07T20:33:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n>> So I do not have a strong opinion either way, if it is more\n>> convenient to propagate the request out to other repositories when\n>> we run processes in two or more repositories (e.g. \"git clone\n>> --local\"), or if it is more convenient to make sure that the request\n>> is limited to the target repository.  Here is a version without the\n>> local_repo_env[] change.\n>\n> Yeah, GIT_CEILING_DIRECTORIES is maybe a bad example. But I do think\n> LITERAL_PATHSPECS is a better one, and the submodule-fetch example I\n> gave would be genuinely surprising if it behaved differently than the\n> superproject, I'd think.\n>\n> I do agree this is probably going to mostly be a debugging aid, so it\n> might not matter much. But once in the wild these things tend to take on\n> a life of their own. ;)\n>\n>> ----- >8 --------- >8 --------- >8 --------- >8 --------- >8 -----\n>> Subject: [PATCH v3 3/3] git: extend --no-lazy-fetch to work across subprocesses\n>\n> So anyway, this version seems good to me.\n\nThanks.\n"},{"id":"490283","messageId":"owlya5n8416t.fsf@fine.c.googlers.com","threadId":"60878","inReplyTo":"20240217052932.GC539459@coredump.intra.peff.net","subject":"Re: [PATCH] git: --no-lazy-fetch option","fromName":"Linus Arver","fromEmail":"linusa@google.com","sentAt":"2024-03-09T01:57:14Z","receivedAt":"2024-03-09T01:57:16Z","isPatch":true,"sender":{"key":"linus@ucla.edu","avatar":null},"body":"Jeff King <peff@peff.net> writes:\n\n> On Fri, Feb 16, 2024 at 09:22:20AM -0800, Junio C Hamano wrote:\n>\n> [...]\n>> Add documentation and note that for this variable, unlike many\n>> boolean-looking environment variables, only the presence matters,\n>> not what value it is set to.\n>\n> Yuck. IMHO depending on GIT_NO_REPLACE_OBJECTS=0 is somewhat crazy, and\n> we could consider the current behavior a bug. It's probably not _that_\n> big a deal either way (because I would not expect anybody to set it that\n> way in the first place). But I wonder if being consistent across\n> variables trumps retaining weird historical compatibility for such a\n> far-fetched case. I dunno. I guess this is https://xkcd.com/1172/. :)\n\nI totally agree with your take on this. Would such cleanup patches\n(e.g., changing the behavior of GIT_NO_REPLACE_OBJECTS=0 to be \"false\"\ninstead of \"true\") be acceptable as #leftoverbits?\n"}]}