{"thread":{"id":"56696","subject":"[RFC PATCH 0/2] Conditional config includes based on remote URL","startedAt":"2021-10-12T22:57:32Z","lastAt":"2022-01-25T13:29:04Z","messageCount":87,"participants":["Jonathan Tan","Jeff King","Junio C Hamano","brian m. carlson","Ævar Arnfjörð Bjarmason","Emily Shaffer","Glen Choo","Elijah Newren","Johannes Schindelin"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"438613","messageId":"cover.1634077795.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":null,"subject":"[RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-12T22:57:21Z","receivedAt":"2021-10-12T22:57:32Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Previously [1], I sent a patch set for remote-suggested configs that are\ntransmitted when fetching, but there were some security concerns. Here\nis another way that remote repo administators can provide recommended\nconfigs - through conditionally included files based on the configured\nremote. Git itself neither transmits nor prompts for these files, which\nhopefully reduces people's concerns.\n\nMore information is in the commit message of patch 2.\n\n[1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n\nJonathan Tan (2):\n config: make git_config_include() static\n config: include file if remote URL matches a glob\n\n config.c | 80 ++++++++++++++++++++++++++++++++++++++++++-----\n config.h | 37 +++-------------------\n t/t1300-config.sh | 27 ++++++++++++++++\n 3 files changed, 103 insertions(+), 41 deletions(-)\n\n-- \n2.33.0.882.g93a45727a2-goog\n\n"},{"id":"438615","messageId":"179df6d6adfe460de7413e1fb1dff6bce185ae0e.1634077795.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"[RFC PATCH 1/2] config: make git_config_include() static","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-12T22:57:22Z","receivedAt":"2021-10-12T22:57:38Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"It is not used from outside the file in which it is declared.\n\nSigned-off-by: Jonathan Tan \n---\n config.c | 12 +++++++++++-\n config.h | 37 ++++---------------------------------\n 2 files changed, 15 insertions(+), 34 deletions(-)\n\ndiff --git a/config.c b/config.c\nindex 2edf835262..365d57833b 100644\n--- a/config.c\n+++ b/config.c\n@@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n \treturn conf->u.buf.pos;\n }\n \n+struct config_include_data {\n+\tint depth;\n+\tconfig_fn_t fn;\n+\tvoid *data;\n+\tconst struct config_options *opts;\n+};\n+#define CONFIG_INCLUDE_INIT { 0 }\n+\n+static int git_config_include(const char *var, const char *value, void *data);\n+\n #define MAX_INCLUDE_DEPTH 10\n static const char include_depth_advice[] = N_(\n \"exceeded maximum include depth (%d) while including\\n\"\n@@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n-int git_config_include(const char *var, const char *value, void *data)\n+static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n \tconst char *cond, *key;\ndiff --git a/config.h b/config.h\nindex 147f5e0490..b11b0be09a 100644\n--- a/config.h\n+++ b/config.h\n@@ -126,6 +126,8 @@ int git_default_config(const char *, const char *, void *);\n /**\n * Read a specific file in git-config format.\n * This function takes the same callback and data parameters as `git_config`.\n+ *\n+ * Unlike git_config(), this function does not respect includes.\n */\n int git_config_from_file(config_fn_t fn, const char *, void *);\n \n@@ -158,6 +160,8 @@ void read_very_early_config(config_fn_t cb, void *data);\n * will first feed the user-wide one to the callback, and then the\n * repo-specific one; by overwriting, the higher-priority repo-specific\n * value is left at the end).\n+ *\n+ * Unlike git_config_from_file(), this function respects includes.\n */\n void git_config(config_fn_t fn, void *);\n \n@@ -339,39 +343,6 @@ const char *current_config_origin_type(void);\n const char *current_config_name(void);\n int current_config_line(void);\n \n-/**\n- * Include Directives\n- * ------------------\n- *\n- * By default, the config parser does not respect include directives.\n- * However, a caller can use the special `git_config_include` wrapper\n- * callback to support them. To do so, you simply wrap your \"real\" callback\n- * function and data pointer in a `struct config_include_data`, and pass\n- * the wrapper to the regular config-reading functions. For example:\n- *\n- * -------------------------------------------\n- * int read_file_with_include(const char *file, config_fn_t fn, void *data)\n- * {\n- * struct config_include_data inc = CONFIG_INCLUDE_INIT;\n- * inc.fn = fn;\n- * inc.data = data;\n- * return git_config_from_file(git_config_include, file, &inc);\n- * }\n- * -------------------------------------------\n- *\n- * `git_config` respects includes automatically. The lower-level\n- * `git_config_from_file` does not.\n- *\n- */\n-struct config_include_data {\n-\tint depth;\n-\tconfig_fn_t fn;\n-\tvoid *data;\n-\tconst struct config_options *opts;\n-};\n-#define CONFIG_INCLUDE_INIT { 0 }\n-int git_config_include(const char *name, const char *value, void *data);\n-\n /*\n * Match and parse a config key of the form:\n *\n-- \n2.33.0.882.g93a45727a2-goog\n\n"},{"id":"438614","messageId":"967680d27aa7a2709e528ff989a9dd534886efba.1634077795.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"[RFC PATCH 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-12T22:57:23Z","receivedAt":"2021-10-12T22:57:39Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This is a feature that supports config file inclusion conditional on\nwhether the repo has a remote with a URL that matches a glob.\n\nSimilar to my previous work on remote-suggested hooks [1], the main\nmotivation is to allow remote repo administrators to provide recommended\nconfigs in a way that can be consumed more easily (e.g. through a\npackage installable by a package manager). But this can also be used by,\nsay, an individual that wants certain configs to apply to a certain set\nof local repos but not others.\n\nI marked this as RFC because there are some design points that need to\nbe resolved:\n\n - The existing \"include\" and \"includeIf\" instructions are executed\n immediately, whereas in order to be useful, the execution of\n \"includeIf hasremoteurl\" needs to be delayed until all config files\n are read. Are there better ways to do this?\n\n - Is the conditionally-included file allowed to have its own\n \"include{,If}\" instructions? I'm thinking that we should forbid it\n because, for example, if we had 4 files as follows: A includes B and\n C includes D, and we include A and C in our main config (in that\n order), it wouldn't be clear whether B (because A was first included)\n or C (because we should execute everything at the same depth first)\n should be executed first. (In this patch, I didn't do anything about\n includes.)\n\n - A small one: the exact format of the glob. I probably will treat the\n URL like a path.\n\n[1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n\nSigned-off-by: Jonathan Tan \n---\n config.c | 70 +++++++++++++++++++++++++++++++++++++++++------\n t/t1300-config.sh | 27 ++++++++++++++++++\n 2 files changed, 89 insertions(+), 8 deletions(-)\n\ndiff --git a/config.c b/config.c\nindex 365d57833b..448509d549 100644\n--- a/config.c\n+++ b/config.c\n@@ -125,8 +125,20 @@ struct config_include_data {\n \tconfig_fn_t fn;\n \tvoid *data;\n \tconst struct config_options *opts;\n+\n+\t/*\n+\t * All remote URLs discovered when reading all config files.\n+\t */\n+\tstruct string_list remote_urls;\n+\n+\t/*\n+\t * All \"includeif.hasremoteurl:\" entries. The item is the URL glob and the\n+\t * util is the path (must be freed).\n+\t */\n+\tstruct string_list include_url_glob_to_path;\n };\n-#define CONFIG_INCLUDE_INIT { 0 }\n+#define CONFIG_INCLUDE_INIT { .remote_urls = STRING_LIST_INIT_DUP, \\\n+\t.include_url_glob_to_path = STRING_LIST_INIT_DUP }\n \n static int git_config_include(const char *var, const char *value, void *data);\n \n@@ -319,10 +331,18 @@ static int include_condition_is_true(const struct config_options *opts,\n static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n \tconst char *cond, *key;\n \tsize_t cond_len;\n \tint ret;\n \n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tstring_list_append(&inc->remote_urls, value);\n+\n \t/*\n \t * Pass along all values, including \"include\" directives; this makes it\n \t * possible to query information on the includes themselves.\n@@ -335,9 +355,18 @@ static int git_config_include(const char *var, const char *value, void *data)\n \t\tret = handle_path_include(value, inc);\n \n \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n-\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n-\t !strcmp(key, \"path\"))\n-\t\tret = handle_path_include(value, inc);\n+\t cond && !strcmp(key, \"path\")) {\n+\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &cond,\n+\t\t\t\t &cond_len)) {\n+\t\t\tstruct string_list_item *item = string_list_append_nodup(\n+\t\t\t\t&inc->include_url_glob_to_path,\n+\t\t\t\txmemdupz(cond, cond_len));\n+\t\t\titem->util = xstrdup(value);\n+\t\t\tret = 0;\n+\t\t} else if (include_condition_is_true(inc->opts, cond, cond_len)) {\n+\t\t\tret = handle_path_include(value, inc);\n+\t\t}\n+\t}\n \n \treturn ret;\n }\n@@ -1951,6 +1980,8 @@ int config_with_options(config_fn_t fn, void *data,\n \t\t\tconst struct config_options *opts)\n {\n \tstruct config_include_data inc = CONFIG_INCLUDE_INIT;\n+\tint ret;\n+\tstruct string_list_item *glob_item;\n \n \tif (opts->respect_includes) {\n \t\tinc.fn = fn;\n@@ -1968,17 +1999,40 @@ int config_with_options(config_fn_t fn, void *data,\n \t * regular lookup sequence.\n \t */\n \tif (config_source && config_source->use_stdin) {\n-\t\treturn git_config_from_stdin(fn, data);\n+\t\tret = git_config_from_stdin(fn, data);\n \t} else if (config_source && config_source->file) {\n-\t\treturn git_config_from_file(fn, config_source->file, data);\n+\t\tret = git_config_from_file(fn, config_source->file, data);\n \t} else if (config_source && config_source->blob) {\n \t\tstruct repository *repo = config_source->repo ?\n \t\t\tconfig_source->repo : the_repository;\n-\t\treturn git_config_from_blob_ref(fn, repo, config_source->blob,\n+\t\tret = git_config_from_blob_ref(fn, repo, config_source->blob,\n \t\t\t\t\t\tdata);\n+\t} else {\n+\t\tret = do_git_config_sequence(opts, fn, data);\n+\t}\n+\n+\tfor_each_string_list_item(glob_item, &inc.include_url_glob_to_path) {\n+\t\tstruct strbuf pattern = STRBUF_INIT;\n+\t\tstruct string_list_item *url_item;\n+\t\tint found = 0;\n+\n+\t\tstrbuf_addstr(&pattern, glob_item->string);\n+\t\tadd_trailing_starstar_for_dir(&pattern);\n+\t\tfor_each_string_list_item(url_item, &inc.remote_urls) {\n+\t\t\tif (!wildmatch(pattern.buf, url_item->string, WM_PATHNAME)) {\n+\t\t\t\tfound = 1;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t\tstrbuf_release(&pattern);\n+\t\tif (found) {\n+\t\t\thandle_path_include(glob_item->util, &inc);\n+\t\t}\n \t}\n \n-\treturn do_git_config_sequence(opts, fn, data);\n+\tstring_list_clear(&inc.remote_urls, 0);\n+\tstring_list_clear(&inc.include_url_glob_to_path, 1);\n+\treturn ret;\n }\n \n static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)\ndiff --git a/t/t1300-config.sh b/t/t1300-config.sh\nindex 9ff46f3b04..4803155f89 100755\n--- a/t/t1300-config.sh\n+++ b/t/t1300-config.sh\n@@ -2387,4 +2387,31 @@ test_expect_success '--get and --get-all with --fixed-value' '\n \ttest_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n '\n \n+test_expect_success 'includeIf.hasremoteurl' '\n+\ttest_create_repo hasremoteurlTest &&\n+\n+\tcat >\"$(pwd)\"/include-this <<-\\EOF &&\n+\t[user]\n+\t\tthis = this-is-included\n+\tEOF\n+\tcat >\"$(pwd)\"/dont-include-that <<-\\EOF &&\n+\t[user]\n+\t\tthat = that-is-not-included\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasremoteurl:foo\"]\n+\t\tpath = \"$(pwd)/include-this\"\n+\t[includeIf \"hasremoteurl:bar\"]\n+\t\tpath = \"$(pwd)/dont-include-that\"\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\tEOF\n+\n+\techo this-is-included >expect-this &&\n+\tgit -C hasremoteurlTest config --get user.this >actual-this &&\n+\ttest_cmp expect-this actual-this &&\n+\n+\ttest_must_fail git -C hasremoteurlTest config --get user.that\n+'\n+\n test_done\n-- \n2.33.0.882.g93a45727a2-goog\n\n"},{"id":"438618","messageId":"YWYVM8dzC/w8t7vk@coredump.intra.peff.net","threadId":"56696","inReplyTo":"179df6d6adfe460de7413e1fb1dff6bce185ae0e.1634077795.git.jonathantanmy@google.com","subject":"Re: [RFC PATCH 1/2] config: make git_config_include() static","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-10-12T23:07:31Z","receivedAt":"2021-10-12T23:07:34Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 12, 2021 at 03:57:22PM -0700, Jonathan Tan wrote:\n\n> It is not used from outside the file in which it is declared.\n\nMakes sense. We used to use it from builtin/config.c, but that went away\nin e895589883 (git-config: use git_config_with_options, 2012-10-23).\n\n> diff --git a/config.h b/config.h\n> index 147f5e0490..b11b0be09a 100644\n> --- a/config.h\n> +++ b/config.h\n> @@ -126,6 +126,8 @@ int git_default_config(const char *, const char *, void *);\n> /**\n> * Read a specific file in git-config format.\n> * This function takes the same callback and data parameters as `git_config`.\n> + *\n> + * Unlike git_config(), this function does not respect includes.\n> */\n\nBreaking out the relevant caller-facing parts of the documentation like\nthis is a nice touch.\n\nAnd the rest of the patch looks good to me.\n\n-Peff\n"},{"id":"438619","messageId":"xmqq1r4p3jw4.fsf@gitster.g","threadId":"56696","inReplyTo":"179df6d6adfe460de7413e1fb1dff6bce185ae0e.1634077795.git.jonathantanmy@google.com","subject":"Re: [RFC PATCH 1/2] config: make git_config_include() static","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-10-12T23:26:03Z","receivedAt":"2021-10-12T23:26:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> It is not used from outside the file in which it is declared.\n>\n> Signed-off-by: Jonathan Tan \n> ---\n> config.c | 12 +++++++++++-\n> config.h | 37 ++++---------------------------------\n> 2 files changed, 15 insertions(+), 34 deletions(-)\n\nNice.\n"},{"id":"438620","messageId":"YWYafdpemaiAjvUV@coredump.intra.peff.net","threadId":"56696","inReplyTo":"967680d27aa7a2709e528ff989a9dd534886efba.1634077795.git.jonathantanmy@google.com","subject":"Re: [RFC PATCH 2/2] config: include file if remote URL matches a glob","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-10-12T23:30:05Z","receivedAt":"2021-10-12T23:30:09Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 12, 2021 at 03:57:23PM -0700, Jonathan Tan wrote:\n\n> This is a feature that supports config file inclusion conditional on\n> whether the repo has a remote with a URL that matches a glob.\n> \n> Similar to my previous work on remote-suggested hooks [1], the main\n> motivation is to allow remote repo administrators to provide recommended\n> configs in a way that can be consumed more easily (e.g. through a\n> package installable by a package manager). But this can also be used by,\n> say, an individual that wants certain configs to apply to a certain set\n> of local repos but not others.\n\nOK. I was a little wary after reading the subject that this would be\n\"when we are using such a URL\", which is full of all kinds of odd corner\ncases. But if it is \"a remote is defined with a matching URL\" that makes\nit a property of the repository, not the operation.\n\nI think in general this kind of feature is currently served by just\ncorrelating filesystem paths with their function. So with your patch I\ncould do:\n\n [includeIf \"hasremoteurl:https://myjob.example.com\"]\n path = foo.conf\n\nBut in general, I'd imagine most people put their repository in ~/work\nor similar, and just do:\n\n [includeIf \"gitdir:~/work\"]\n path = foo.conf\n\n(and of course you can imagine more subdivisions as necessary). So I\nfind the use-case only sort-of compelling. In general, I'm in favor of\nadding new includeIf directions even if they're only moderately\nconvenient. But this one is rather sticky, because it is dependent on\nother config keys being defined. So it introduces a new and complicated\nordering issue. Is it worth it? Maybe I'm not being imaginative enough\nin seeing the use cases.\n\n> I marked this as RFC because there are some design points that need to\n> be resolved:\n> \n> - The existing \"include\" and \"includeIf\" instructions are executed\n> immediately, whereas in order to be useful, the execution of\n> \"includeIf hasremoteurl\" needs to be delayed until all config files\n> are read. Are there better ways to do this?\n\nNote that this violates the \"as if they had been found at the location\nof the include directive\" rule which we advertise to users. I'd imagine\nthat most of the time it doesn't matter, but this is a pretty big\nexception we'll need to document.\n\nJust brainstorming some alternatives:\n\n - We could stop the world while we are parsing and do a _new_ parse\n that just looks at the remote config (in fact, this is the natural\n thing if you were consulting the regular remote.c code for the list\n of remotes, because it does its own config parse).\n\n That does mean that the remote-conditional includes cannot\n themselves define new remotes. But I think that is already the case\n with your patch (and violating that gets you into weird circular\n problems).\n\n - We could simply document that if you want to depend on conditional\n includes based on a particular remote.*.url existing, then that\n remote config must appear earlier in the sequence.\n\n This is a bit ugly, because I'm sure it will bite somebody\n eventually. But at the same time, it resolves all of the weird\n timing issues, and does so in a way that will be easy to match if we\n have any other config dependencies.\n\n> - Is the conditionally-included file allowed to have its own\n> \"include{,If}\" instructions? I'm thinking that we should forbid it\n> because, for example, if we had 4 files as follows: A includes B and\n> C includes D, and we include A and C in our main config (in that\n> order), it wouldn't be clear whether B (because A was first included)\n> or C (because we should execute everything at the same depth first)\n> should be executed first. (In this patch, I didn't do anything about\n> includes.)\n\nI'd say that A would expand B at the moment it is parsed, by the usual\nas-if rule. If it has a recursive includeIf on remotes, then my head may\nexplode. I'd argue that we should refuse to do recursive remote-ifs in\nthat case (though all of this is a consequence of the after-the-fact\nparsing; I'd much prefer one of the alternatives I gave earlier).\n\n> - A small one: the exact format of the glob. I probably will treat the\n> URL like a path.\n\nYou might want to use the matcher from urlmatch.[ch], which understands\nthings like wildcards. Of course remote \"URLs\" are not always real\nsyntactically valid URLs, which may make that awkward.\n\nBarring that the usual fnmatch glob is probably our best bet.\n\n> @@ -319,10 +331,18 @@ static int include_condition_is_true(const struct config_options *opts,\n> static int git_config_include(const char *var, const char *value, void *data)\n> {\n> \tstruct config_include_data *inc = data;\n> +\tconst char *remote_name;\n> +\tsize_t remote_name_len;\n> \tconst char *cond, *key;\n> \tsize_t cond_len;\n> \tint ret;\n> \n> +\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n> +\t\t\t &key) &&\n> +\t remote_name &&\n> +\t !strcmp(key, \"url\"))\n> +\t\tstring_list_append(&inc->remote_urls, value);\n\nSo we make a copy of every remote name on the off chance that somebody\nhas an includeIf which looks at it. That feels wasteful, though in\npractice it's probably not that big a deal.\n\nBy doing the config parsing ourselves here we're missing out on any\nother forms of remote, like .git/remotes. Those are old and not widely\nused, and I'd be OK with skipping them. But we should clearly document\nthat this is matching remote.*.url, not any of the other mechanisms.\n\n> [...]\n\nI only lightly read the rest of the patch. I didn't see anything\nobviously wrong, but I think the goal at this point is figuring out the\ndesign.\n\n-Peff\n"},{"id":"438621","messageId":"xmqqk0ih24a0.fsf@gitster.g","threadId":"56696","inReplyTo":"967680d27aa7a2709e528ff989a9dd534886efba.1634077795.git.jonathantanmy@google.com","subject":"Re: [RFC PATCH 2/2] config: include file if remote URL matches a glob","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-10-12T23:48:39Z","receivedAt":"2021-10-12T23:48:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> I marked this as RFC because there are some design points that need to\n> be resolved:\n>\n> - The existing \"include\" and \"includeIf\" instructions are executed\n> immediately, whereas in order to be useful, the execution of\n> \"includeIf hasremoteurl\" needs to be delayed until all config files\n> are read. Are there better ways to do this?\n\nAn interesting chicken-and-egg problem. Even if an included\nconfiguration file does not have further \"include\", you may discover\nthere are more remotes, which may add new includes to fire from the\ntop-level configuration file.\n\nWhat if we have multiple remotes? Is it a sufficient match for only\none of them match what is mentioned in the includeIf condition?\nShould all of them must match the pattern instead? Majority,\nperhaps? Something else?\n\n> - Is the conditionally-included file allowed to have its own\n> \"include{,If}\" instructions? I'm thinking that we should forbid it\n> because, for example, if we had 4 files as follows: A includes B and\n> C includes D, and we include A and C in our main config (in that\n> order), it wouldn't be clear whether B (because A was first included)\n> or C (because we should execute everything at the same depth first)\n> should be executed first. (In this patch, I didn't do anything about\n> includes.)\n\nInteresting. The order of real inclusion obviously would affect the\noutcome of the \"last one wins\" rule. And this does not have to be\nlimited to this \"hasremote\" condition, so we need to design it with\na bit of care.\n\nWould it be possible for a newly included file to invalidate an\nearlier condition that was used to decide whether to include another\nfile or not? If not, then you can take a two-pass approach where\nthe first pass is used to decide solely to discover which\nconditionally included files are taken, clear the slate and the\nparse these files in the textual order. In the case of your example\nabove, the early part of the primary config would be the first to be\nread, then comes A's early part, then comes B in its entirety, then\nthe rest of A, and then the middle part of the primary config, then\nC's early part, then D, and then the rest of C,... you got the idea.\n\nIf it is possible for an included file to invalidate a condition we\nhave already evaluated to make a decision, it would become messy.\nFor example, we might decide to include another file based on the\nvalue we discovered for a config variable:\n\n === .git/config ===\n [my] variable\n [your] variable = false\n\n [includeIf \"configEq:my.variable==true\"]\n file = fileA\n\nbut the included file may override the condition, e.g.\n\n === fileA ===\n [my] variable = false\n [your] variable = true\n\nand applying the \"last one wins\" rule becomes messy. I do not\noffhand know what these\n\n $ git config --bool my.variable\n $ git config --bool your.variable\n\nshould say, and do not have a good explanation for possible\noutcomes.\n\nMaybe the above example can serve as a way to guide us when we\ndesign the types of conditionals we allow in includeIf. This\nexample tells us that it is probably a terrible idea to allow using\nvalues of configuration variables as part of \"includeIf\" condition.\n\nThere may lead to similar \"'hasremoteurl' makes a well-behaved\ncondition, because [remote] are additive and not 'last-one-wins',\nbut we cannot add 'lacksremoteurl' as a condition, because a file we\ndecide to include based on a 'lacks' predicate may invalidate the\n'lacks' condition by defining such a remote\" design decisions you'd\nneed to make around the URLs of the remotes defined for the\nrepository.\n\n\n"},{"id":"438623","messageId":"YWYsbGHwQZbEuTCa@camp.crustytoothpaste.net","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"Re: [RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2021-10-13T00:46:36Z","receivedAt":"2021-10-13T00:47:27Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2021-10-12 at 22:57:21, Jonathan Tan wrote:\n> Previously [1], I sent a patch set for remote-suggested configs that are\n> transmitted when fetching, but there were some security concerns. Here\n> is another way that remote repo administators can provide recommended\n> configs - through conditionally included files based on the configured\n> remote. Git itself neither transmits nor prompts for these files, which\n> hopefully reduces people's concerns.\n> \n> More information is in the commit message of patch 2.\n\nI won't go into the details of the patches, since I'm a little low on\ntime at the moment, but I think from what I've seen of the cover letter\nand the commit messages, this approach is much better from a security\nperspective and, provided we can get the kinks mentioned downthread\nironed out, I'd be happy to see this merged.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"438643","messageId":"87sfx5ib3h.fsf@evledraar.gmail.com","threadId":"56696","inReplyTo":"179df6d6adfe460de7413e1fb1dff6bce185ae0e.1634077795.git.jonathantanmy@google.com","subject":"Re: [RFC PATCH 1/2] config: make git_config_include() static","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-13T08:26:11Z","receivedAt":"2021-10-13T08:27:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Oct 12 2021, Jonathan Tan wrote:\n\n> It is not used from outside the file in which it is declared.\n>\n> Signed-off-by: Jonathan Tan \n> ---\n> config.c | 12 +++++++++++-\n> config.h | 37 ++++---------------------------------\n> 2 files changed, 15 insertions(+), 34 deletions(-)\n>\n> diff --git a/config.c b/config.c\n> index 2edf835262..365d57833b 100644\n> --- a/config.c\n> +++ b/config.c\n> @@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n> \treturn conf->u.buf.pos;\n> }\n> \n> +struct config_include_data {\n> +\tint depth;\n> +\tconfig_fn_t fn;\n> +\tvoid *data;\n> +\tconst struct config_options *opts;\n> +};\n> +#define CONFIG_INCLUDE_INIT { 0 }\n> +\n> +static int git_config_include(const char *var, const char *value, void *data);\n\nCan't we just move the function here?\n\n> #define MAX_INCLUDE_DEPTH 10\n> static const char include_depth_advice[] = N_(\n> \"exceeded maximum include depth (%d) while including\\n\"\n> @@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n> \treturn 0;\n> }\n> \n> -int git_config_include(const char *var, const char *value, void *data)\n> +static int git_config_include(const char *var, const char *value, void *data)\n\n...and avoid the forward declaration?\n\nI've seen that in a few places, making the diff smaller here doesn't\nseem worth it v.s. maintaining the definition in two places.\n"},{"id":"438665","messageId":"xmqqk0igj1vf.fsf@gitster.g","threadId":"56696","inReplyTo":"87sfx5ib3h.fsf@evledraar.gmail.com","subject":"Re: [RFC PATCH 1/2] config: make git_config_include() static","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-10-13T17:00:52Z","receivedAt":"2021-10-13T17:00:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Tue, Oct 12 2021, Jonathan Tan wrote:\n>\n>> It is not used from outside the file in which it is declared.\n>>\n>> Signed-off-by: Jonathan Tan \n>> ---\n>> config.c | 12 +++++++++++-\n>> config.h | 37 ++++---------------------------------\n>> 2 files changed, 15 insertions(+), 34 deletions(-)\n>>\n>> diff --git a/config.c b/config.c\n>> index 2edf835262..365d57833b 100644\n>> --- a/config.c\n>> +++ b/config.c\n>> @@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n>> \treturn conf->u.buf.pos;\n>> }\n>> \n>> +struct config_include_data {\n>> +\tint depth;\n>> +\tconfig_fn_t fn;\n>> +\tvoid *data;\n>> +\tconst struct config_options *opts;\n>> +};\n>> +#define CONFIG_INCLUDE_INIT { 0 }\n>> +\n>> +static int git_config_include(const char *var, const char *value, void *data);\n>\n> Can't we just move the function here?\n>\n>> #define MAX_INCLUDE_DEPTH 10\n>> static const char include_depth_advice[] = N_(\n>> \"exceeded maximum include depth (%d) while including\\n\"\n>> @@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n>> \treturn 0;\n>> }\n>> \n>> -int git_config_include(const char *var, const char *value, void *data)\n>> +static int git_config_include(const char *var, const char *value, void *data)\n>\n> ...and avoid the forward declaration?\n>\n> I've seen that in a few places, making the diff smaller here doesn't\n> seem worth it v.s. maintaining the definition in two places.\n\nSounds good. If we are moving things around anyway, it is probably\na good time to do that, too ;-)\n"},{"id":"438672","messageId":"20211013181353.83505-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"xmqqk0igj1vf.fsf@gitster.g","subject":"Re: [RFC PATCH 1/2] config: make git_config_include() static","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-13T18:13:53Z","receivedAt":"2021-10-13T18:14:10Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> > ...and avoid the forward declaration?\n> >\n> > I've seen that in a few places, making the diff smaller here doesn't\n> > seem worth it v.s. maintaining the definition in two places.\n> \n> Sounds good. If we are moving things around anyway, it is probably\n> a good time to do that, too ;-)\n\nOK, I'll do this.\n"},{"id":"438673","messageId":"20211013181743.84126-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"YWYsbGHwQZbEuTCa@camp.crustytoothpaste.net","subject":"Re: [RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-13T18:17:43Z","receivedAt":"2021-10-13T18:18:29Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> I won't go into the details of the patches, since I'm a little low on\n> time at the moment, but I think from what I've seen of the cover letter\n> and the commit messages, this approach is much better from a security\n> perspective and, provided we can get the kinks mentioned downthread\n> ironed out, I'd be happy to see this merged.\n\nThanks - I really appreciate this note. Thanks also for all your\nthoughts up to now about the security perspective.\n"},{"id":"438674","messageId":"20211013183341.85761-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"YWYafdpemaiAjvUV@coredump.intra.peff.net","subject":"Re: [RFC PATCH 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-13T18:33:41Z","receivedAt":"2021-10-13T18:33:59Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> OK. I was a little wary after reading the subject that this would be\n> \"when we are using such a URL\", which is full of all kinds of odd corner\n> cases. But if it is \"a remote is defined with a matching URL\" that makes\n> it a property of the repository, not the operation.\n> \n> I think in general this kind of feature is currently served by just\n> correlating filesystem paths with their function. So with your patch I\n> could do:\n> \n> [includeIf \"hasremoteurl:https://myjob.example.com\"]\n> path = foo.conf\n> \n> But in general, I'd imagine most people put their repository in ~/work\n> or similar, and just do:\n> \n> [includeIf \"gitdir:~/work\"]\n> path = foo.conf\n> \n> (and of course you can imagine more subdivisions as necessary). So I\n> find the use-case only sort-of compelling. In general, I'm in favor of\n> adding new includeIf directions even if they're only moderately\n> convenient. But this one is rather sticky, because it is dependent on\n> other config keys being defined. So it introduces a new and complicated\n> ordering issue. Is it worth it? Maybe I'm not being imaginative enough\n> in seeing the use cases.\n\nMy main use case is for a remote repo administrator to offer a\nrecommended config to anyone who clones that repo. For this, I don't\nthink we can prescribe a local directory structure (e.g. \"~/work\")\nwithout being too restrictive or broad (that is, if the user ends up\ncreating a repo that so happens to match our glob but did not intend the\nconfig to apply to it).\n\nI did bring up the idea that an individual could use this to have config\nin one place that affects a subset of remotes, but you're right that\nthey could just do this by putting repositories at different places in\nthe filesystem.\n\n> > I marked this as RFC because there are some design points that need to\n> > be resolved:\n> > \n> > - The existing \"include\" and \"includeIf\" instructions are executed\n> > immediately, whereas in order to be useful, the execution of\n> > \"includeIf hasremoteurl\" needs to be delayed until all config files\n> > are read. Are there better ways to do this?\n> \n> Note that this violates the \"as if they had been found at the location\n> of the include directive\" rule which we advertise to users. I'd imagine\n> that most of the time it doesn't matter, but this is a pretty big\n> exception we'll need to document.\n\nYes, that's true. Another thing I just thought of is to add a new\n\"deferIncludeIf\" which makes clear the different semantics (deferred\ninclude, and perhaps not allow recursive includes).\n\n> Just brainstorming some alternatives:\n> \n> - We could stop the world while we are parsing and do a _new_ parse\n> that just looks at the remote config (in fact, this is the natural\n> thing if you were consulting the regular remote.c code for the list\n> of remotes, because it does its own config parse).\n> \n> That does mean that the remote-conditional includes cannot\n> themselves define new remotes. But I think that is already the case\n> with your patch (and violating that gets you into weird circular\n> problems).\n\nHmm...yes, having a special-case rule that such an included file cannot\ndefine new remotes would be complex.\n\n> - We could simply document that if you want to depend on conditional\n> includes based on a particular remote.*.url existing, then that\n> remote config must appear earlier in the sequence.\n> \n> This is a bit ugly, because I'm sure it will bite somebody\n> eventually. But at the same time, it resolves all of the weird\n> timing issues, and does so in a way that will be easy to match if we\n> have any other config dependencies.\n\nMy main issue with this is that different config files are read at\ndifferent times, and the repo config (that usually contains the remote)\nis read last.\n\n> > - Is the conditionally-included file allowed to have its own\n> > \"include{,If}\" instructions? I'm thinking that we should forbid it\n> > because, for example, if we had 4 files as follows: A includes B and\n> > C includes D, and we include A and C in our main config (in that\n> > order), it wouldn't be clear whether B (because A was first included)\n> > or C (because we should execute everything at the same depth first)\n> > should be executed first. (In this patch, I didn't do anything about\n> > includes.)\n> \n> I'd say that A would expand B at the moment it is parsed, by the usual\n> as-if rule. If it has a recursive includeIf on remotes, then my head may\n> explode. I'd argue that we should refuse to do recursive remote-ifs in\n> that case (though all of this is a consequence of the after-the-fact\n> parsing; I'd much prefer one of the alternatives I gave earlier).\n\nIf we can't expand in place, I would say that any recursive includes\nshould be refused. But as you said, we could still think about whether\nin-place expansion can be done before addressing this question.\n\n> > - A small one: the exact format of the glob. I probably will treat the\n> > URL like a path.\n> \n> You might want to use the matcher from urlmatch.[ch], which understands\n> things like wildcards. Of course remote \"URLs\" are not always real\n> syntactically valid URLs, which may make that awkward.\n> \n> Barring that the usual fnmatch glob is probably our best bet.\n\nOK.\n\n> So we make a copy of every remote name on the off chance that somebody\n> has an includeIf which looks at it. That feels wasteful, though in\n> practice it's probably not that big a deal.\n> \n> By doing the config parsing ourselves here we're missing out on any\n> other forms of remote, like .git/remotes. Those are old and not widely\n> used, and I'd be OK with skipping them. But we should clearly document\n> that this is matching remote.*.url, not any of the other mechanisms.\n\nSounds good.\n\n> I only lightly read the rest of the patch. I didn't see anything\n> obviously wrong, but I think the goal at this point is figuring out the\n> design.\n\nYes, that's right.\n"},{"id":"438685","messageId":"20211013195245.93145-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"xmqqk0ih24a0.fsf@gitster.g","subject":"Re: [RFC PATCH 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-13T19:52:45Z","receivedAt":"2021-10-13T19:52:52Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Jonathan Tan writes:\n> \n> > I marked this as RFC because there are some design points that need to\n> > be resolved:\n> >\n> > - The existing \"include\" and \"includeIf\" instructions are executed\n> > immediately, whereas in order to be useful, the execution of\n> > \"includeIf hasremoteurl\" needs to be delayed until all config files\n> > are read. Are there better ways to do this?\n> \n> An interesting chicken-and-egg problem. Even if an included\n> configuration file does not have further \"include\", you may discover\n> there are more remotes, which may add new includes to fire from the\n> top-level configuration file.\n\nThat's true. We might need to say that such conditional includes are\nbased only on what happened during the main config parsing.\n\n> What if we have multiple remotes? Is it a sufficient match for only\n> one of them match what is mentioned in the includeIf condition?\n> Should all of them must match the pattern instead? Majority,\n> perhaps? Something else?\n\nI think at least one remote should match.\n\n> > - Is the conditionally-included file allowed to have its own\n> > \"include{,If}\" instructions? I'm thinking that we should forbid it\n> > because, for example, if we had 4 files as follows: A includes B and\n> > C includes D, and we include A and C in our main config (in that\n> > order), it wouldn't be clear whether B (because A was first included)\n> > or C (because we should execute everything at the same depth first)\n> > should be executed first. (In this patch, I didn't do anything about\n> > includes.)\n> \n> Interesting. The order of real inclusion obviously would affect the\n> outcome of the \"last one wins\" rule. And this does not have to be\n> limited to this \"hasremote\" condition, so we need to design it with\n> a bit of care.\n> \n> Would it be possible for a newly included file to invalidate an\n> earlier condition that was used to decide whether to include another\n> file or not? If not, then you can take a two-pass approach where\n> the first pass is used to decide solely to discover which\n> conditionally included files are taken, clear the slate and the\n> parse these files in the textual order. In the case of your example\n> above, the early part of the primary config would be the first to be\n> read, then comes A's early part, then comes B in its entirety, then\n> the rest of A, and then the middle part of the primary config, then\n> C's early part, then D, and then the rest of C,... you got the idea.\n>\n> If it is possible for an included file to invalidate a condition we\n> have already evaluated to make a decision, it would become messy.\n> For example, we might decide to include another file based on the\n> value we discovered for a config variable:\n> \n> === .git/config ===\n> [my] variable\n> [your] variable = false\n> \n> [includeIf \"configEq:my.variable==true\"]\n> file = fileA\n> \n> but the included file may override the condition, e.g.\n> \n> === fileA ===\n> [my] variable = false\n> [your] variable = true\n> \n> and applying the \"last one wins\" rule becomes messy. I do not\n> offhand know what these\n> \n> $ git config --bool my.variable\n> $ git config --bool your.variable\n> \n> should say, and do not have a good explanation for possible\n> outcomes.\n\nIn this case, it makes sense to me to think that files are included\nentirely or not at all, so my.variable would be false and your.variable\nwould be true. I guess the tricky part is something like:\n\n === .git/config ===\n [my] variable = true\n [your] variable = false\n [includeIf \"configEq:my.variable==true\"]\n file = fileA\n [includeIf \"configEq:my.variable==false\"]\n file = fileB\n === fileA ===\n my.variable = false\n === fileB ===\n your.variable = true\n\nand what my.variable and your.variable would end up being.\n\n> Maybe the above example can serve as a way to guide us when we\n> design the types of conditionals we allow in includeIf. This\n> example tells us that it is probably a terrible idea to allow using\n> values of configuration variables as part of \"includeIf\" condition.\n\nHmm...well, remote.foo.url is a configuration variable. I think that the\ntwo-pass approach you describe would work if we prohibit subsequent\ninclusions.\n\n> There may lead to similar \"'hasremoteurl' makes a well-behaved\n> condition, because [remote] are additive and not 'last-one-wins',\n> but we cannot add 'lacksremoteurl' as a condition, because a file we\n> decide to include based on a 'lacks' predicate may invalidate the\n> 'lacks' condition by defining such a remote\" design decisions you'd\n> need to make around the URLs of the remotes defined for the\n> repository.\n\nAnd if we implement two-pass with no subsequent inclusions, \"lacks\"\nwould work the same way.\n"},{"id":"439005","messageId":"20211018204803.75088-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"Re: [RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-18T20:48:03Z","receivedAt":"2021-10-18T20:48:12Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"After some in-office discussion, here are the alternatives as I see it:\n\n (1) Introduce a \"includeAfterIf\" (or \"deferIncludeIf\", or some other\n name) command that is executed after all config files are read. (If\n there are multiple, they are executed in order of appearance.)\n Files included by this mechanism cannot directly or indirectly\n contain another \"includeAfterIf\". This is the same as what was\n introduced in this patch set, except for the name of the directive.\n\n (2) Leave the name as \"includeIf\", and when it is encountered with a\n remote-URL condition: continue parsing the config files, skipping\n all \"includeIf hasRemoteUrl\", only looking for remote.*.url. After\n that, resume the reading of config files at the first \"includeIf\n hasRemoteUrl\", using the prior remote.*.url information gathered to\n determine which files to include when \"includeIf hasRemoteUrl\" is\n encountered. Files included by this mechanism cannot contain any\n \"remote.*.url\" variables.\n\nIn all cases, the include is executed if at least one remote URL\nmatches.\n\nThere are other ideas including:\n\n (3) remote.*.url must appear before a \"includeIf hasRemoteUrl\" that\n wants to match it. (But this doesn't fit our use case, in which a\n repo config has the URL but a system or user config has the\n include.)\n\n (4) \"includeIf hasRemoteUrl\" triggers a search of the repo config just\n for remote.*.url. (I think this out-of-order config search is more\n complicated than (2), though.)\n\nFor (2), I think that prohibiting \"remote.*.url\" from any \"includeIf\nhasRemoteUrl\" files sidesteps questions like \"what happens when an\nincluded file overrides the URL that made us include this file in the\nfirst place\" or \"what happens if an included file includes a\nremote.*.url that validates or invalidates a prior or subsequent file\",\nbecause now that cannot happen at all. My main concern with this\nprohibition was that if we were to introduce another similar condition\n(say, one based on remote names), what would happen? But I think this is\nsolvable - make the prohibitions based only on all the conditions that\nthe actually used, so if the user only uses conditions on remote URLs,\nthen the user can still set refspecs (for example), even after the\nremote-name-condition feature is introduced in Git.\n\nFor (1), it is simpler in concept (and also in implementation, I think).\nThe user just needs to know that certain includes are on-the-spot and\ncertain includes (the ones with \"after\" in the name) are deferred - in\nparticular, if a config variable isn't the value they expect, they'll\nneed to check that it wasn't introduced in an includeAfterIf file. (And\nthe user also needs to figure out that if they want to override such a\nvariable, they'll need to make their own includeAfterIf with an\nalways-true condition.)\n\nFrom the prior replies, I think that people will be more interested in\n(2) as it preserves the \"last config wins\" rule, and I'm inclined to go\nfor (2) too. I'll see if others have any other opinions, and if not I'll\nsee how the implementation of (2) will look like.\n"},{"id":"439326","messageId":"YXIsNIoxhADuM1Mi@google.com","threadId":"56696","inReplyTo":"20211018204803.75088-1-jonathantanmy@google.com","subject":"Re: [RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-10-22T03:12:52Z","receivedAt":"2021-10-22T03:13:00Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Oct 18, 2021 at 01:48:03PM -0700, Jonathan Tan wrote:\n> \n> After some in-office discussion, here are the alternatives as I see it:\n> \n> (1) Introduce a \"includeAfterIf\" (or \"deferIncludeIf\", or some other\n> name) command that is executed after all config files are read. (If\n> there are multiple, they are executed in order of appearance.)\n> Files included by this mechanism cannot directly or indirectly\n> contain another \"includeAfterIf\". This is the same as what was\n> introduced in this patch set, except for the name of the directive.\n> \n> (2) Leave the name as \"includeIf\", and when it is encountered with a\n> remote-URL condition: continue parsing the config files, skipping\n> all \"includeIf hasRemoteUrl\", only looking for remote.*.url. After\n> that, resume the reading of config files at the first \"includeIf\n> hasRemoteUrl\", using the prior remote.*.url information gathered to\n> determine which files to include when \"includeIf hasRemoteUrl\" is\n> encountered. Files included by this mechanism cannot contain any\n> \"remote.*.url\" variables.\n> \n> In all cases, the include is executed if at least one remote URL\n> matches.\n> \n> There are other ideas including:\n> \n> (3) remote.*.url must appear before a \"includeIf hasRemoteUrl\" that\n> wants to match it. (But this doesn't fit our use case, in which a\n> repo config has the URL but a system or user config has the\n> include.)\n> \n> (4) \"includeIf hasRemoteUrl\" triggers a search of the repo config just\n> for remote.*.url. (I think this out-of-order config search is more\n> complicated than (2), though.)\n> \n> For (2), I think that prohibiting \"remote.*.url\" from any \"includeIf\n> hasRemoteUrl\" files sidesteps questions like \"what happens when an\n> included file overrides the URL that made us include this file in the\n> first place\" or \"what happens if an included file includes a\n> remote.*.url that validates or invalidates a prior or subsequent file\",\n> because now that cannot happen at all. My main concern with this\n> prohibition was that if we were to introduce another similar condition\n> (say, one based on remote names), what would happen? But I think this is\n> solvable - make the prohibitions based only on all the conditions that\n> the actually used, so if the user only uses conditions on remote URLs,\n> then the user can still set refspecs (for example), even after the\n> remote-name-condition feature is introduced in Git.\n> \n> For (1), it is simpler in concept (and also in implementation, I think).\n> The user just needs to know that certain includes are on-the-spot and\n> certain includes (the ones with \"after\" in the name) are deferred - in\n> particular, if a config variable isn't the value they expect, they'll\n> need to check that it wasn't introduced in an includeAfterIf file. (And\n> the user also needs to figure out that if they want to override such a\n> variable, they'll need to make their own includeAfterIf with an\n> always-true condition.)\n> \n> From the prior replies, I think that people will be more interested in\n> (2) as it preserves the \"last config wins\" rule, and I'm inclined to go\n> for (2) too. I'll see if others have any other opinions, and if not I'll\n> see how the implementation of (2) will look like.\n\nAnother concern which came up for me in a private conversation today -\n\nHow difficult will it be for users to override this include directive if\nit is set somewhere outside of their control? For example:\n\n/etc/gitconfig:\n[includeIf hasRemoteUrl.https://example.com/example.git] // or whatever\n path = /etc/some-special-config\n\nWill it be possible for a user to \"un-include\" /etc/some-special-config\nthemselves?\n\nI don't think this should change your patch much - if my understanding\nis correct, we also don't have a way to \"un-include\" existing include or\nincludeIf directives made outside of the user's control. But I wonder if\nit'd be useful to think about some way to do that. Maybe we can teach\nthe config parse how to include a config file in reverse? Maybe we need\na \"neverInclude\" directive? Food for thought, anyway.\n\nSorry, but I won't have time to take a look at the rest of this series\ntil next week.\n\n - Emily\n\n"},{"id":"439544","messageId":"211025.86lf2hgsmj.gmgdl@evledraar.gmail.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"Re: [RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-25T13:03:20Z","receivedAt":"2021-10-25T13:07:36Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Oct 12 2021, Jonathan Tan wrote:\n\nI tried sending the below (sans some last minute spellchecking now)\naround October 19th, but for some reason it didn't make it\non-list. Trying again now, apologies for [near-]duplicates, if any (I\nelaborated a bit at the end just now).\n\n> Previously [1], I sent a patch set for remote-suggested configs that are\n> transmitted when fetching, but there were some security concerns. Here\n> is another way that remote repo administators can provide recommended\n> configs - through conditionally included files based on the configured\n> remote. Git itself neither transmits nor prompts for these files, which\n> hopefully reduces people's concerns.\n\nI had some concerns about the specifics of the implementation/what\nseemed to be tailoring it a bit too closely to one use-case[1][2], not\ninherently with the idea (although I think e.g. for brian that more\nclosely reflects his thoughts).\n\nAnyway, just saying that aside from this RFC I don't think we were at\nthe point of really fleshing out what this would look like, and there\nbeing some hard \"no\", so I think that idea could still be pursued.\n\nOn this proposal: this also applies globally to all history, but I don't\nhave the same concern with that as the 1=1 mapping of remote-suggested\nhooks, our path includes work that way, after all.\n\nI think it would be nice if you could think about if/how this and the\n\"onbranch\" include would work together though to serve the general case\nbetter.\n\nAlso if you have a repo with N remotes each where \"origin\" tracks URLs\nat git.example.com, and you add a \"dev\" tracking dev.example.com, will\nthe config apply if you're say on a branch tracking the \"live\" server,\nif you've said \"include this for repos matching dev.example.com?\n\nArguably that's what you want, but perhaps something that those more\nused to the centralized workflows wouldn't consider as being unintuitive\nfor users who might want to add this config only for their main \"origin\"\nremote. We don't really have a way of marking that special-ness though,\nexcept maybe checkout.defaultRemote.\n\nI'm also still somewhat mystified at how this would better serve your\nuserbase than the path-based included, i.e. the selling point of the\nremote-suggested configuration was that it would Just Work.\n\nBut for this the users would either need to setup the config themselves\nfor your remote, but that would be easier than pro-actively cloning in\n\"work\" or whatever? I guess, just wondering if I'm missing something.\n\nOr if it's a partly-automated system where some automation is dropping\nin a /etc/gitconfig.d/google-remote-config-include I wonder if this\nwhole thing wouldn't be better for users with such special-needs if we\njust supported an \"early config hook\".\n\ni.e. similar to how we read trace2 config from /etc/gitconfig early, we\ncould start picking up a hook that just so happens to conform to the\nconfig schema Emily's config-based hooks use.\n\nSo the /etc/gitconfig would have say:\n\n hook.ourConfigThingy.command=/usr/bin/googly-git-config\n hook.ourConfigThingy.event=include-config\n\nThat hook would just produce a config snippet to be included on STDOUT.\n\nSince it's an arbitrary external command it would nicely get around any\nchicken and egg problems in git itself, it could run \"git remote -v\",\ninspect the equivalent of an \"onbranch\" etc. etc, then just dynamically\nproduce config-to-be-included.\n\nPlease don't take this as some objection to your current proposal, just\na thought on something that might entirely bypass odd edge cases and\narbitrary limitations associated with doing this all in the \"main\"\nprocess on-the-fly.\n\nThe special-ness with that one would need to be that we'd say it\nwouldn't have the normal \"last set wins\" semantics, or maybe we could do\nthat and just note that we saw it, and execute the \"include\" when we\ndetect the end of the full config parsing (I'm not familiar enough with\nthose bits to say where that is).\n\nBoth of those seem easier than dealing with any chicken & egg problems\nin parsing the config stream itself, since such a hook could just invoke\n\"git remote -v\" and the like itself, after e.g. setting some environment\nvariable of its own to guard against its own recursion (or we'd do it\nfor it for such hooks...).\n\n1. https://lore.kernel.org/git/87k0mn2dd3.fsf@evledraar.gmail.com/\n2. https://lore.kernel.org/git/87o8awvglr.fsf@evledraar.gmail.com/\n"},{"id":"439577","messageId":"20211025185356.1232635-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"211025.86lf2hgsmj.gmgdl@evledraar.gmail.com","subject":"Re: [RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-25T18:53:56Z","receivedAt":"2021-10-25T18:54:02Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> I had some concerns about the specifics of the implementation/what\n> seemed to be tailoring it a bit too closely to one use-case[1][2], not\n> inherently with the idea (although I think e.g. for brian that more\n> closely reflects his thoughts).\n> \n> Anyway, just saying that aside from this RFC I don't think we were at\n> the point of really fleshing out what this would look like, and there\n> being some hard \"no\", so I think that idea could still be pursued.\n\nWhich idea specifically do you think could still be pursued?\n\n> On this proposal: this also applies globally to all history, but I don't\n> have the same concern with that as the 1=1 mapping of remote-suggested\n> hooks, our path includes work that way, after all.\n> \n> I think it would be nice if you could think about if/how this and the\n> \"onbranch\" include would work together though to serve the general case\n> better.\n> \n> Also if you have a repo with N remotes each where \"origin\" tracks URLs\n> at git.example.com, and you add a \"dev\" tracking dev.example.com, will\n> the config apply if you're say on a branch tracking the \"live\" server,\n> if you've said \"include this for repos matching dev.example.com?\n\nRight now, the feature is only dependent on remote URLs configured\nthrough remote.?.url. It wouldn't work with \"onbranch\" because there's\nno way to combine conditions (and I have no plans to do that). I think\nthat if you have something that you want depending on which branch\nyou're on, you can just use the existing \"onbranch\" feature.\n\n> Arguably that's what you want, but perhaps something that those more\n> used to the centralized workflows wouldn't consider as being unintuitive\n> for users who might want to add this config only for their main \"origin\"\n> remote. We don't really have a way of marking that special-ness though,\n> except maybe checkout.defaultRemote.\n\nWhat do you mean by adding a config for a specific remote?\n\n> I'm also still somewhat mystified at how this would better serve your\n> userbase than the path-based included, i.e. the selling point of the\n> remote-suggested configuration was that it would Just Work.\n> \n> But for this the users would either need to setup the config themselves\n> for your remote, but that would be easier than pro-actively cloning in\n> \"work\" or whatever? I guess, just wondering if I'm missing something.\n> \n> Or if it's a partly-automated system where some automation is dropping\n> in a /etc/gitconfig.d/google-remote-config-include \n\nYes, the config is meant to be handled e.g. through a package manager\nlike apt. We don't want to prescribe directory structures like \"work\",\nwhich is why the include is conditional upon the remote URL.\n\nEven if the user pro-actively clones into \"work\", the user still needs\nto set up the conditional config, so I don't see how that is a net\nbenefit.\n\n> I wonder if this\n> whole thing wouldn't be better for users with such special-needs if we\n> just supported an \"early config hook\".\n> \n> i.e. similar to how we read trace2 config from /etc/gitconfig early, we\n> could start picking up a hook that just so happens to conform to the\n> config schema Emily's config-based hooks use.\n> \n> So the /etc/gitconfig would have say:\n> \n> hook.ourConfigThingy.command=/usr/bin/googly-git-config\n> hook.ourConfigThingy.event=include-config\n> \n> That hook would just produce a config snippet to be included on STDOUT.\n> \n> Since it's an arbitrary external command it would nicely get around any\n> chicken and egg problems in git itself, it could run \"git remote -v\",\n> inspect the equivalent of an \"onbranch\" etc. etc, then just dynamically\n> produce config-to-be-included.\n\nI see that later on, you suggest an environment variable to guard\nagainst recursion.\n\nOne thing is that if there are multiple such hooks, each one won't be\nable to see what the other hooks have produced.\n\nIf the feature you described already existed in Git, I think I could use\nthat, but if we're deciding between implementing the config hook you\ndescribe versus something with more constraints, I think the one I\nproposed is better for now. Some design points that have already been\ndiscussed are whether setting a config during processing of an included\nfile would then invalidate the include and also the order of operations,\nboth of which would be much more difficult to control with config hooks.\n\n> Please don't take this as some objection to your current proposal, just\n> a thought on something that might entirely bypass odd edge cases and\n> arbitrary limitations associated with doing this all in the \"main\"\n> process on-the-fly.\n> \n> The special-ness with that one would need to be that we'd say it\n> wouldn't have the normal \"last set wins\" semantics, or maybe we could do\n> that and just note that we saw it, and execute the \"include\" when we\n> detect the end of the full config parsing (I'm not familiar enough with\n> those bits to say where that is).\n\nThe \"last set\" would be those set by the hooks, so yes, a user would\nneed to know to make their own hook in order to override anything set by\nthe hooks. The end of the full config parsing is in\nconfig_with_options().\n\n> Both of those seem easier than dealing with any chicken & egg problems\n> in parsing the config stream itself, since such a hook could just invoke\n> \"git remote -v\" and the like itself, after e.g. setting some environment\n> variable of its own to guard against its own recursion (or we'd do it\n> for it for such hooks...).\n> \n> 1. https://lore.kernel.org/git/87k0mn2dd3.fsf@evledraar.gmail.com/\n> 2. https://lore.kernel.org/git/87o8awvglr.fsf@evledraar.gmail.com/\n"},{"id":"439635","messageId":"211026.86bl3c13pw.gmgdl@evledraar.gmail.com","threadId":"56696","inReplyTo":"20211025185356.1232635-1-jonathantanmy@google.com","subject":"Re: [RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-10-26T10:12:12Z","receivedAt":"2021-10-26T10:26:57Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Oct 25 2021, Jonathan Tan wrote:\n\n>> I had some concerns about the specifics of the implementation/what\n>> seemed to be tailoring it a bit too closely to one use-case[1][2], not\n>> inherently with the idea (although I think e.g. for brian that more\n>> closely reflects his thoughts).\n>> \n>> Anyway, just saying that aside from this RFC I don't think we were at\n>> the point of really fleshing out what this would look like, and there\n>> being some hard \"no\", so I think that idea could still be pursued.\n>\n> Which idea specifically do you think could still be pursued?\n\nI meant the whole in-repo .gitconfig. I.e. to the extent that you're\nsubmitting this as an alternative to that because of the negative\nfeedback on that RFC.\n\n>> On this proposal: this also applies globally to all history, but I don't\n>> have the same concern with that as the 1=1 mapping of remote-suggested\n>> hooks, our path includes work that way, after all.\n>> \n>> I think it would be nice if you could think about if/how this and the\n>> \"onbranch\" include would work together though to serve the general case\n>> better.\n>> \n>> Also if you have a repo with N remotes each where \"origin\" tracks URLs\n>> at git.example.com, and you add a \"dev\" tracking dev.example.com, will\n>> the config apply if you're say on a branch tracking the \"live\" server,\n>> if you've said \"include this for repos matching dev.example.com?\n>\n> Right now, the feature is only dependent on remote URLs configured\n> through remote.?.url. It wouldn't work with \"onbranch\" because there's\n> no way to combine conditions (and I have no plans to do that). I think\n> that if you have something that you want depending on which branch\n> you're on, you can just use the existing \"onbranch\" feature.\n\nI mean with this and the below...\n\n>> Arguably that's what you want, but perhaps something that those more\n>> used to the centralized workflows wouldn't consider as being unintuitive\n>> for users who might want to add this config only for their main \"origin\"\n>> remote. We don't really have a way of marking that special-ness though,\n>> except maybe checkout.defaultRemote.\n>\n> What do you mean by adding a config for a specific remote?\n\n...what happens if you add a google.com remote for a repository that\n\"lives\" on github.com. I.e. are the semantics \"match any remote\", or\n\"match the 'primary' remote (origin?\" etc.\n\n>> I'm also still somewhat mystified at how this would better serve your\n>> userbase than the path-based included, i.e. the selling point of the\n>> remote-suggested configuration was that it would Just Work.\n>> \n>> But for this the users would either need to setup the config themselves\n>> for your remote, but that would be easier than pro-actively cloning in\n>> \"work\" or whatever? I guess, just wondering if I'm missing something.\n>> \n>> Or if it's a partly-automated system where some automation is dropping\n>> in a /etc/gitconfig.d/google-remote-config-include \n>\n> Yes, the config is meant to be handled e.g. through a package manager\n> like apt. We don't want to prescribe directory structures like \"work\",\n> which is why the include is conditional upon the remote URL.\n>\n> Even if the user pro-actively clones into \"work\", the user still needs\n> to set up the conditional config, so I don't see how that is a net\n> benefit.\n\nAh, that explains it. I assumed both cases would be ones where the user\nwould need to manually enable the 'configuration' (or cloning to a given\nsubdir).\n\n>> I wonder if this\n>> whole thing wouldn't be better for users with such special-needs if we\n>> just supported an \"early config hook\".\n>> \n>> i.e. similar to how we read trace2 config from /etc/gitconfig early, we\n>> could start picking up a hook that just so happens to conform to the\n>> config schema Emily's config-based hooks use.\n>> \n>> So the /etc/gitconfig would have say:\n>> \n>> hook.ourConfigThingy.command=/usr/bin/googly-git-config\n>> hook.ourConfigThingy.event=include-config\n>> \n>> That hook would just produce a config snippet to be included on STDOUT.\n>> \n>> Since it's an arbitrary external command it would nicely get around any\n>> chicken and egg problems in git itself, it could run \"git remote -v\",\n>> inspect the equivalent of an \"onbranch\" etc. etc, then just dynamically\n>> produce config-to-be-included.\n>\n> I see that later on, you suggest an environment variable to guard\n> against recursion.\n>\n> One thing is that if there are multiple such hooks, each one won't be\n> able to see what the other hooks have produced.\n\nYes, although aside from this hook that's a general caveat with the\nproposed config-based hooks, I think if you need a hook that does that\n(whether it's this, or pre-receive etc.) our answer is \"put it in your\nown wrapper\".\n\n> If the feature you described already existed in Git, I think I could use\n> that, but if we're deciding between implementing the config hook you\n> describe versus something with more constraints, I think the one I\n> proposed is better for now. Some design points that have already been\n> discussed are whether setting a config during processing of an included\n> file would then invalidate the include and also the order of operations,\n> both of which would be much more difficult to control with config hooks.\n\nI suggested it because maybe it would be a lot simpler, i.e. we don't\nneed such a feature to be aware of remote config at all, or having to\n\"read forward\" to find it, maybe it would be more complex. I haven't\ntried to implement it.\n\n>> Please don't take this as some objection to your current proposal, just\n>> a thought on something that might entirely bypass odd edge cases and\n>> arbitrary limitations associated with doing this all in the \"main\"\n>> process on-the-fly.\n>> \n>> The special-ness with that one would need to be that we'd say it\n>> wouldn't have the normal \"last set wins\" semantics, or maybe we could do\n>> that and just note that we saw it, and execute the \"include\" when we\n>> detect the end of the full config parsing (I'm not familiar enough with\n>> those bits to say where that is).\n>\n> The \"last set\" would be those set by the hooks, so yes, a user would\n> need to know to make their own hook in order to override anything set by\n> the hooks. The end of the full config parsing is in\n> config_with_options().\n\nOn the \"user would need to know\" that's the same if it's config? I.e. in\neither case it would be in /etc/gitconfig or whatever shipped by the\n*.deb package.\n\nAnyway, I really just meant this as a suggestion, and one that might\nmake things simpler. If you don't think it makes sense...\n"},{"id":"439752","messageId":"YXk6nLJTZn1ilsf6@coredump.intra.peff.net","threadId":"56696","inReplyTo":"20211013183341.85761-1-jonathantanmy@google.com","subject":"Re: [RFC PATCH 2/2] config: include file if remote URL matches a glob","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-10-27T11:40:12Z","receivedAt":"2021-10-27T11:40:14Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Oct 13, 2021 at 11:33:41AM -0700, Jonathan Tan wrote:\n\n> > But in general, I'd imagine most people put their repository in ~/work\n> > or similar, and just do:\n> > \n> > [includeIf \"gitdir:~/work\"]\n> > path = foo.conf\n> > \n> > (and of course you can imagine more subdivisions as necessary). So I\n> > find the use-case only sort-of compelling. In general, I'm in favor of\n> > adding new includeIf directions even if they're only moderately\n> > convenient. But this one is rather sticky, because it is dependent on\n> > other config keys being defined. So it introduces a new and complicated\n> > ordering issue. Is it worth it? Maybe I'm not being imaginative enough\n> > in seeing the use cases.\n> \n> My main use case is for a remote repo administrator to offer a\n> recommended config to anyone who clones that repo. For this, I don't\n> think we can prescribe a local directory structure (e.g. \"~/work\")\n> without being too restrictive or broad (that is, if the user ends up\n> creating a repo that so happens to match our glob but did not intend the\n> config to apply to it).\n\nYeah, I agree that it's not quite as turnkey if you have to assume\nsomething about the user's directory structure. On the other hand, they\nhave to decide to put the included config file somewhere, too, so it\nseems like you need to give the user \"do something like this\"\ninstructions rather than purely something they can copy and paste.\n\nI dunno. I guess you can assume they'll put it in ~/.gitconfig-foo or\nsimilar, and come up with copy-and-pastable directions from that.\n\nI agree that the \"match the remote\" rule makes things _more_ convenient.\nMostly I was just wondering if it changed things enough to merit the\ncomplications it introduces. I'm not sure I have an answer, and clearly\nit's pretty subjective.\n\n> > Just brainstorming some alternatives:\n> > \n> > - We could stop the world while we are parsing and do a _new_ parse\n> > that just looks at the remote config (in fact, this is the natural\n> > thing if you were consulting the regular remote.c code for the list\n> > of remotes, because it does its own config parse).\n> > \n> > That does mean that the remote-conditional includes cannot\n> > themselves define new remotes. But I think that is already the case\n> > with your patch (and violating that gets you into weird circular\n> > problems).\n> \n> Hmm...yes, having a special-case rule that such an included file cannot\n> define new remotes would be complex.\n\nI think that's mostly true of your \"defer\" system, too, unless you keep\napplying it recursively. The rule is slightly different there: it's not\n\"you can't define new remotes\", but rather \"you can't do a\nremote-conditional include based on a remote included by\nremote-conditional\".\n\n> > - We could simply document that if you want to depend on conditional\n> > includes based on a particular remote.*.url existing, then that\n> > remote config must appear earlier in the sequence.\n> > \n> > This is a bit ugly, because I'm sure it will bite somebody\n> > eventually. But at the same time, it resolves all of the weird\n> > timing issues, and does so in a way that will be easy to match if we\n> > have any other config dependencies.\n> \n> My main issue with this is that different config files are read at\n> different times, and the repo config (that usually contains the remote)\n> is read last.\n\nAh, right. I was thinking of the definitions within a single file, but\nyou're right that the common case would be having the include in\n~/.gitconfig, and the remotes defined in $GIT_DIR/config. So yeah, any\nordering constraint like that is a non-starter, I'd think.\n\n-Peff\n"},{"id":"439753","messageId":"YXk+TRnndNZkdsGF@coredump.intra.peff.net","threadId":"56696","inReplyTo":"20211018204803.75088-1-jonathantanmy@google.com","subject":"Re: [RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-10-27T11:55:57Z","receivedAt":"2021-10-27T11:56:09Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Oct 18, 2021 at 01:48:03PM -0700, Jonathan Tan wrote:\n\n> (1) Introduce a \"includeAfterIf\" (or \"deferIncludeIf\", or some other\n> name) command that is executed after all config files are read. (If\n> there are multiple, they are executed in order of appearance.)\n> Files included by this mechanism cannot directly or indirectly\n> contain another \"includeAfterIf\". This is the same as what was\n> introduced in this patch set, except for the name of the directive.\n\nI think this works in terms of having self-consistent rules that make\nsense. But deferring things does introduce new complications in terms of\noverrides, because we rely on last-one-wins. Emily asked elsewhere about\noverriding the inclusion of a file. We don't have a way to do that now,\nand I think it would be tricky to add. But what about overriding a\nsingle variable?\n\nRight now this works:\n\n git config --global foo.bar true\n git config --local foo.bar false\n\nto give you \"false\". But imagining there was a world of deferred config,\nthen:\n\n git config --file ~/.gitconfig-foo foo.bar true\n git config --global deferInclude.path .gitconfig-foo\n git config --local foo.bar false\n\ngives \"true\". We'd read .gitconfig-foo after everything else, overriding\nthe repo-level config.\n\nIf the deferred includes were processed at the end of each individual\nfile, that would solve that. You're still left with the slight oddness\nthat a deferred include may override options within the same file that\ncome after it, but that's inherent to the \"defer\" concept, and the\nanswer is probably \"don't do that\". It's only when it crosses file\nboundaries (which are explicitly ordered by priority) that it really\nhurts.\n\n> (2) Leave the name as \"includeIf\", and when it is encountered with a\n> remote-URL condition: continue parsing the config files, skipping\n> all \"includeIf hasRemoteUrl\", only looking for remote.*.url. After\n> that, resume the reading of config files at the first \"includeIf\n> hasRemoteUrl\", using the prior remote.*.url information gathered to\n> determine which files to include when \"includeIf hasRemoteUrl\" is\n> encountered. Files included by this mechanism cannot contain any\n> \"remote.*.url\" variables.\n\nI think doing this as \"continue parsing\" and \"resume\" is hard to do.\nBecause you can't look at other non-remote.*.url entries here (otherwise\nyou'd see them out of order). So you have to either:\n\n - complete the parse, stashing all the other variables away, and then\n resolve the include, and then look at all the stashed variables as\n if you were parsing them anew.\n\n - teach our config parser how to save and restore state, including\n both intra-file state and the progress across the set of files\n\nI think it's much easier if you think of it as \"start a new config parse\nthat does not respect hasRemoteURL\". And the easiest way to do that is\nto just let remote.c's existing git_config() start that parse (probably\nby calling git_config_with_options() and telling it \"don't respect\nhasRemoteURL includes\"). You may also need to teach the config parser to\nbe reentrant. We did some work on that a while ago, pushing the state\nint config_source which functions as a stack, but I don't offhand know\nif you can call git_config() from within a config callback.\n\n> There are other ideas including:\n> \n> (3) remote.*.url must appear before a \"includeIf hasRemoteUrl\" that\n> wants to match it. (But this doesn't fit our use case, in which a\n> repo config has the URL but a system or user config has the\n> include.)\n\nYeah, I agree this won't work.\n\n> (4) \"includeIf hasRemoteUrl\" triggers a search of the repo config just\n> for remote.*.url. (I think this out-of-order config search is more\n> complicated than (2), though.)\n\nI think this is what I described above, and actually is less\ncomplicated. ;)\n\n-Peff\n"},{"id":"439796","messageId":"20211027172358.2227660-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"YXk6nLJTZn1ilsf6@coredump.intra.peff.net","subject":"Re: [RFC PATCH 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-27T17:23:58Z","receivedAt":"2021-10-27T17:24:05Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Yeah, I agree that it's not quite as turnkey if you have to assume\n> something about the user's directory structure. On the other hand, they\n> have to decide to put the included config file somewhere, too, so it\n> seems like you need to give the user \"do something like this\"\n> instructions rather than purely something they can copy and paste.\n\nThey can copy and paste instructions to add a package repository (e.g.\nby editing /etc/apt/sources.list) and then install a package.\n\n> I dunno. I guess you can assume they'll put it in ~/.gitconfig-foo or\n> similar, and come up with copy-and-pastable directions from that.\n> \n> I agree that the \"match the remote\" rule makes things _more_ convenient.\n> Mostly I was just wondering if it changed things enough to merit the\n> complications it introduces. I'm not sure I have an answer, and clearly\n> it's pretty subjective.\n\nI am almost done with the implementation, so maybe the community could\nlook at it and concretely see the extent of the complication.\n\n> > > Just brainstorming some alternatives:\n> > > \n> > > - We could stop the world while we are parsing and do a _new_ parse\n> > > that just looks at the remote config (in fact, this is the natural\n> > > thing if you were consulting the regular remote.c code for the list\n> > > of remotes, because it does its own config parse).\n> > > \n> > > That does mean that the remote-conditional includes cannot\n> > > themselves define new remotes. But I think that is already the case\n> > > with your patch (and violating that gets you into weird circular\n> > > problems).\n> > \n> > Hmm...yes, having a special-case rule that such an included file cannot\n> > define new remotes would be complex.\n> \n> I think that's mostly true of your \"defer\" system, too, unless you keep\n> applying it recursively. The rule is slightly different there: it's not\n> \"you can't define new remotes\", but rather \"you can't do a\n> remote-conditional include based on a remote included by\n> remote-conditional\".\n\nI was thinking that deferred includes cannot themselves have other\ndeferred includes and that which deferred includes are included would be\ncomputed only once, and those would be the only rules. (But I guess this\nis moot now - we're not doing this approach.)\n\n> > > - We could simply document that if you want to depend on conditional\n> > > includes based on a particular remote.*.url existing, then that\n> > > remote config must appear earlier in the sequence.\n> > > \n> > > This is a bit ugly, because I'm sure it will bite somebody\n> > > eventually. But at the same time, it resolves all of the weird\n> > > timing issues, and does so in a way that will be easy to match if we\n> > > have any other config dependencies.\n> > \n> > My main issue with this is that different config files are read at\n> > different times, and the repo config (that usually contains the remote)\n> > is read last.\n> \n> Ah, right. I was thinking of the definitions within a single file, but\n> you're right that the common case would be having the include in\n> ~/.gitconfig, and the remotes defined in $GIT_DIR/config. So yeah, any\n> ordering constraint like that is a non-starter, I'd think.\n> \n> -Peff\n\nYeah. Thanks for continuing to take a look at this.\n"},{"id":"439799","messageId":"20211027175259.2230232-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"YXk+TRnndNZkdsGF@coredump.intra.peff.net","subject":"Re: [RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-27T17:52:59Z","receivedAt":"2021-10-27T17:53:04Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> On Mon, Oct 18, 2021 at 01:48:03PM -0700, Jonathan Tan wrote:\n> \n> > (1) Introduce a \"includeAfterIf\" (or \"deferIncludeIf\", or some other\n> > name) command that is executed after all config files are read. (If\n> > there are multiple, they are executed in order of appearance.)\n> > Files included by this mechanism cannot directly or indirectly\n> > contain another \"includeAfterIf\". This is the same as what was\n> > introduced in this patch set, except for the name of the directive.\n> \n> I think this works in terms of having self-consistent rules that make\n> sense. But deferring things does introduce new complications in terms of\n> overrides, because we rely on last-one-wins. Emily asked elsewhere about\n> overriding the inclusion of a file. We don't have a way to do that now,\n> and I think it would be tricky to add. But what about overriding a\n> single variable?\n> \n> Right now this works:\n> \n> git config --global foo.bar true\n> git config --local foo.bar false\n> \n> to give you \"false\". But imagining there was a world of deferred config,\n> then:\n> \n> git config --file ~/.gitconfig-foo foo.bar true\n> git config --global deferInclude.path .gitconfig-foo\n> git config --local foo.bar false\n> \n> gives \"true\". We'd read .gitconfig-foo after everything else, overriding\n> the repo-level config.\n> \n> If the deferred includes were processed at the end of each individual\n> file, that would solve that. You're still left with the slight oddness\n> that a deferred include may override options within the same file that\n> come after it, but that's inherent to the \"defer\" concept, and the\n> answer is probably \"don't do that\". It's only when it crosses file\n> boundaries (which are explicitly ordered by priority) that it really\n> hurts.\n\nThis would indeed solve the issue of the user needing to know the trick\nto override variables set by deferred includes. But this wouldn't solve\nour primary use case in which a system-level config defines a\nconditional include but the repo config defines the URL, I think.\n\n> > (2) Leave the name as \"includeIf\", and when it is encountered with a\n> > remote-URL condition: continue parsing the config files, skipping\n> > all \"includeIf hasRemoteUrl\", only looking for remote.*.url. After\n> > that, resume the reading of config files at the first \"includeIf\n> > hasRemoteUrl\", using the prior remote.*.url information gathered to\n> > determine which files to include when \"includeIf hasRemoteUrl\" is\n> > encountered. Files included by this mechanism cannot contain any\n> > \"remote.*.url\" variables.\n> \n> I think doing this as \"continue parsing\" and \"resume\" is hard to do.\n> Because you can't look at other non-remote.*.url entries here (otherwise\n> you'd see them out of order). So you have to either:\n> \n> - complete the parse, stashing all the other variables away, and then\n> resolve the include, and then look at all the stashed variables as\n> if you were parsing them anew.\n> \n> - teach our config parser how to save and restore state, including\n> both intra-file state and the progress across the set of files\n\nI am implementing something similar to your first approach (stashing\nthings). It's almost done so hopefully we'll have something concrete to\ndiscuss soon.\n\n> I think it's much easier if you think of it as \"start a new config parse\n> that does not respect hasRemoteURL\". And the easiest way to do that is\n> to just let remote.c's existing git_config() start that parse (probably\n> by calling git_config_with_options() and telling it \"don't respect\n> hasRemoteURL includes\"). You may also need to teach the config parser to\n> be reentrant. We did some work on that a while ago, pushing the state\n> int config_source which functions as a stack, but I don't offhand know\n> if you can call git_config() from within a config callback.\n\nBesides the reentrancy (which may be difficult, as there are some global\nvariables, but from a glance, some code seems to take care to save and\nrestore them, so it may already be reentrant or not too difficult to\nmake reentrant), we would have to bubble down the config (struct\ngit_config_source and struct config_options) into all the places that\ncould potentially start the parse and also have a place to store the\nURLs we get. If we're already going to stash URLs, it may be easier to\nstash the variables instead.\n\n> > There are other ideas including:\n> > \n> > (3) remote.*.url must appear before a \"includeIf hasRemoteUrl\" that\n> > wants to match it. (But this doesn't fit our use case, in which a\n> > repo config has the URL but a system or user config has the\n> > include.)\n> \n> Yeah, I agree this won't work.\n> \n> > (4) \"includeIf hasRemoteUrl\" triggers a search of the repo config just\n> > for remote.*.url. (I think this out-of-order config search is more\n> > complicated than (2), though.)\n> \n> I think this is what I described above, and actually is less\n> complicated. ;)\n> \n> -Peff\n\nWell, let me finish up (2), and let's see.\n"},{"id":"439830","messageId":"YXm3dtOy3FwR7FUG@coredump.intra.peff.net","threadId":"56696","inReplyTo":"20211027175259.2230232-1-jonathantanmy@google.com","subject":"Re: [RFC PATCH 0/2] Conditional config includes based on remote URL","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-10-27T20:32:54Z","receivedAt":"2021-10-27T20:32:58Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Oct 27, 2021 at 10:52:59AM -0700, Jonathan Tan wrote:\n\n> > If the deferred includes were processed at the end of each individual\n> > file, that would solve that. You're still left with the slight oddness\n> > that a deferred include may override options within the same file that\n> > come after it, but that's inherent to the \"defer\" concept, and the\n> > answer is probably \"don't do that\". It's only when it crosses file\n> > boundaries (which are explicitly ordered by priority) that it really\n> > hurts.\n> \n> This would indeed solve the issue of the user needing to know the trick\n> to override variables set by deferred includes. But this wouldn't solve\n> our primary use case in which a system-level config defines a\n> conditional include but the repo config defines the URL, I think.\n\nDoh, of course. I forgot that was the whole point of the defer. ;)\n\n> I am implementing something similar to your first approach (stashing\n> things). It's almost done so hopefully we'll have something concrete to\n> discuss soon.\n\nSounds good.\n\n-Peff\n"},{"id":"440028","messageId":"cover.1635527389.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"[WIP v2 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-29T17:31:08Z","receivedAt":"2021-10-29T17:31:16Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Here's a version that implements the URL lookahead. Compared to an\napproach in which we rerun config in order to just get the remote URLs,\nI think that the main benefit of this is that we don't read from disk\ntwice. One inelegant thing is that we now have essentially two caches of\nconfig variables - the one in struct repository (struct config_set\n*config) and the one newly introduced by this patch set. (Although I\nthink that many commands don't use the cache in struct repository,\ninstead just reading the config in one pass.)\n\n(Of course, there is also the possibility that we should have the remote\nrepo administrator provide the config in another way.)\n\nI have marked this WIP. This patch set is mostly done, except for the\nfollowing:\n - Prohibiting remote.?.url from any files included directly or\n indirectly by a URL-conditional include.\n - Checking that memory everywhere is freed when no longer needed.\n - Documentation (as mentioned in the NEEDSWORK comment in patch 2).\n - Tests that check what the glob matches and doesn't match.\n\nNo range-diff included because this version is substantially different.\n\nJonathan Tan (2):\n config: make git_config_include() static\n config: include file if remote URL matches a glob\n\n config.c | 142 ++++++++++++++++++++++++++++++++++++++++++----\n config.h | 37 ++----------\n t/t1300-config.sh | 60 ++++++++++++++++++++\n 3 files changed, 194 insertions(+), 45 deletions(-)\n\n-- \n2.33.1.1089.g2158813163f-goog\n\n"},{"id":"440029","messageId":"3879e94c822421774001c823635f06235f50bfde.1635527389.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1635527389.git.jonathantanmy@google.com","subject":"[WIP v2 1/2] config: make git_config_include() static","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-29T17:31:09Z","receivedAt":"2021-10-29T17:31:18Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"It is not used from outside the file in which it is declared.\n\nSigned-off-by: Jonathan Tan \n---\n config.c | 12 +++++++++++-\n config.h | 37 ++++---------------------------------\n 2 files changed, 15 insertions(+), 34 deletions(-)\n\ndiff --git a/config.c b/config.c\nindex 2dcbe901b6..94ad5ce913 100644\n--- a/config.c\n+++ b/config.c\n@@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n \treturn conf->u.buf.pos;\n }\n \n+struct config_include_data {\n+\tint depth;\n+\tconfig_fn_t fn;\n+\tvoid *data;\n+\tconst struct config_options *opts;\n+};\n+#define CONFIG_INCLUDE_INIT { 0 }\n+\n+static int git_config_include(const char *var, const char *value, void *data);\n+\n #define MAX_INCLUDE_DEPTH 10\n static const char include_depth_advice[] = N_(\n \"exceeded maximum include depth (%d) while including\\n\"\n@@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n-int git_config_include(const char *var, const char *value, void *data)\n+static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n \tconst char *cond, *key;\ndiff --git a/config.h b/config.h\nindex f119de0130..48a5e472ca 100644\n--- a/config.h\n+++ b/config.h\n@@ -126,6 +126,8 @@ int git_default_config(const char *, const char *, void *);\n /**\n * Read a specific file in git-config format.\n * This function takes the same callback and data parameters as `git_config`.\n+ *\n+ * Unlike git_config(), this function does not respect includes.\n */\n int git_config_from_file(config_fn_t fn, const char *, void *);\n \n@@ -158,6 +160,8 @@ void read_very_early_config(config_fn_t cb, void *data);\n * will first feed the user-wide one to the callback, and then the\n * repo-specific one; by overwriting, the higher-priority repo-specific\n * value is left at the end).\n+ *\n+ * Unlike git_config_from_file(), this function respects includes.\n */\n void git_config(config_fn_t fn, void *);\n \n@@ -338,39 +342,6 @@ const char *current_config_origin_type(void);\n const char *current_config_name(void);\n int current_config_line(void);\n \n-/**\n- * Include Directives\n- * ------------------\n- *\n- * By default, the config parser does not respect include directives.\n- * However, a caller can use the special `git_config_include` wrapper\n- * callback to support them. To do so, you simply wrap your \"real\" callback\n- * function and data pointer in a `struct config_include_data`, and pass\n- * the wrapper to the regular config-reading functions. For example:\n- *\n- * -------------------------------------------\n- * int read_file_with_include(const char *file, config_fn_t fn, void *data)\n- * {\n- * struct config_include_data inc = CONFIG_INCLUDE_INIT;\n- * inc.fn = fn;\n- * inc.data = data;\n- * return git_config_from_file(git_config_include, file, &inc);\n- * }\n- * -------------------------------------------\n- *\n- * `git_config` respects includes automatically. The lower-level\n- * `git_config_from_file` does not.\n- *\n- */\n-struct config_include_data {\n-\tint depth;\n-\tconfig_fn_t fn;\n-\tvoid *data;\n-\tconst struct config_options *opts;\n-};\n-#define CONFIG_INCLUDE_INIT { 0 }\n-int git_config_include(const char *name, const char *value, void *data);\n-\n /*\n * Match and parse a config key of the form:\n *\n-- \n2.33.1.1089.g2158813163f-goog\n\n"},{"id":"440030","messageId":"3c0d51ee39b8e149b5be57b8cd3f8cd403fe49c9.1635527389.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1635527389.git.jonathantanmy@google.com","subject":"[WIP v2 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-10-29T17:31:10Z","receivedAt":"2021-10-29T17:31:25Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This is a feature that supports config file inclusion conditional on\nwhether the repo has a remote with a URL that matches a glob.\n\nSimilar to my previous work on remote-suggested hooks [1], the main\nmotivation is to allow remote repo administrators to provide recommended\nconfigs in a way that can be consumed more easily (e.g. through a\npackage installable by a package manager).\n\nNEEDSWORK: The way this works is that if we see such an include, we\nshunt all subsequent configs into a stash (while looking for URLs), then\nprocess the stash. In particular, this means that more memory is needed,\nand the nature of error reporting changes (currently, if a callback\nreturns nonzero for a variable, processing halts immediately, but with\nthis patch, all the config might be read from disk before the callback\neven sees the variable). I'll need to expand on this and write a\ndocumentation section.\n\nOne alternative is to rerun the config parsing mechanism upon noticing\nthe first URL-conditional include in order to find all URLs. This would\nrequire the config files to be read from disk twice, though.\n\n[1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n\nSigned-off-by: Jonathan Tan \n---\n config.c | 132 +++++++++++++++++++++++++++++++++++++++++-----\n t/t1300-config.sh | 60 +++++++++++++++++++++\n 2 files changed, 180 insertions(+), 12 deletions(-)\n\ndiff --git a/config.c b/config.c\nindex 94ad5ce913..63a37e0a5d 100644\n--- a/config.c\n+++ b/config.c\n@@ -120,13 +120,30 @@ static long config_buf_ftell(struct config_source *conf)\n \treturn conf->u.buf.pos;\n }\n \n+struct stashed_var {\n+\tchar *var;\n+\tchar *value;\n+\tint depth;\n+\n+\tchar *url;\n+};\n+\n struct config_include_data {\n \tint depth;\n \tconfig_fn_t fn;\n \tvoid *data;\n \tconst struct config_options *opts;\n+\n+\t/*\n+\t * All remote URLs discovered when reading all config files.\n+\t */\n+\tstruct string_list remote_urls;\n+\n+\tstruct stashed_var *stashed;\n+\tsize_t stashed_nr, stashed_alloc;\n+\tint current_stash_depth;\n };\n-#define CONFIG_INCLUDE_INIT { 0 }\n+#define CONFIG_INCLUDE_INIT { .remote_urls = STRING_LIST_INIT_DUP }\n \n static int git_config_include(const char *var, const char *value, void *data);\n \n@@ -316,28 +333,110 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n+static int execute_stashed(struct config_include_data *inc)\n+{\n+\tsize_t i = 0;\n+\twhile (i < inc->stashed_nr) {\n+\t\tint ret = inc->fn(inc->stashed[i].var, inc->stashed[i].value,\n+\t\t\t\t inc->data);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\n+\t\t/*\n+\t\t * If it is an include, skip to next entry of the same depth if\n+\t\t * the URL doesn't match\n+\t\t */\n+\t\tif (inc->stashed[i].url) {\n+\t\t\tstruct strbuf pattern = STRBUF_INIT;\n+\t\t\tstruct string_list_item *url_item;\n+\t\t\tint found = 0;\n+\n+\t\t\tstrbuf_addstr(&pattern, inc->stashed[i].url);\n+\t\t\tadd_trailing_starstar_for_dir(&pattern);\n+\t\t\tfor_each_string_list_item(url_item, &inc->remote_urls) {\n+\t\t\t\tif (!wildmatch(pattern.buf, url_item->string,\n+\t\t\t\t\t WM_PATHNAME)) {\n+\t\t\t\t\tfound = 1;\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t\tstrbuf_release(&pattern);\n+\t\t\tif (found) {\n+\t\t\t\ti++;\n+\t\t\t} else {\n+\t\t\t\tint depth = inc->stashed[i].depth;\n+\n+\t\t\t\ti++;\n+\t\t\t\twhile (i < inc->stashed_nr &&\n+\t\t\t\t inc->stashed[i].depth != depth)\n+\t\t\t\t\ti++;\n+\t\t\t}\n+\t\t} else {\n+\t\t\ti++;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n \tconst char *cond, *key;\n \tsize_t cond_len;\n-\tint ret;\n+\tint ret = 0;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tstring_list_append(&inc->remote_urls, value);\n \n \t/*\n \t * Pass along all values, including \"include\" directives; this makes it\n \t * possible to query information on the includes themselves.\n \t */\n-\tret = inc->fn(var, value, inc->data);\n-\tif (ret < 0)\n-\t\treturn ret;\n+\tif (inc->stashed_nr || starts_with(var, \"includeif.hasremoteurl:\")) {\n+\t\tstruct stashed_var *last;\n+\n+\t\t/*\n+\t\t * Start or continue using the stash. (A false positive on\n+\t\t * \"includeif.hasremoteurl:?.path\" is fine here - this just\n+\t\t * means that some config variables unnecessarily go through\n+\t\t * the stash before being passed to the callback.)\n+\t\t */\n+\t\tALLOC_GROW_BY(inc->stashed, inc->stashed_nr, 1,\n+\t\t\t inc->stashed_alloc);\n+\t\tlast = &inc->stashed[inc->stashed_nr - 1];\n+\t\tlast->var = xstrdup(var);\n+\t\tlast->value = xstrdup(value);\n+\t\tlast->depth = inc->current_stash_depth;\n+\t} else {\n+\t\tret = inc->fn(var, value, inc->data);\n+\t\tif (ret < 0)\n+\t\t\treturn ret;\n+\t}\n \n \tif (!strcmp(var, \"include.path\"))\n \t\tret = handle_path_include(value, inc);\n \n \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n-\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n-\t !strcmp(key, \"path\"))\n-\t\tret = handle_path_include(value, inc);\n+\t cond && !strcmp(key, \"path\")) {\n+\t\tconst char *url;\n+\t\tsize_t url_len;\n+\n+\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &url,\n+\t\t\t\t &url_len)) {\n+\t\t\tinc->stashed[inc->stashed_nr - 1].url =\n+\t\t\t\txmemdupz(url, url_len);\n+\t\t\tinc->current_stash_depth++;\n+\t\t\tret = handle_path_include(value, inc);\n+\t\t\tinc->current_stash_depth--;\n+\t\t} else if (include_condition_is_true(inc->opts, cond, cond_len)) {\n+\t\t\tret = handle_path_include(value, inc);\n+\t\t}\n+\t}\n \n \treturn ret;\n }\n@@ -1933,6 +2032,7 @@ int config_with_options(config_fn_t fn, void *data,\n \t\t\tconst struct config_options *opts)\n {\n \tstruct config_include_data inc = CONFIG_INCLUDE_INIT;\n+\tint ret;\n \n \tif (opts->respect_includes) {\n \t\tinc.fn = fn;\n@@ -1950,17 +2050,25 @@ int config_with_options(config_fn_t fn, void *data,\n \t * regular lookup sequence.\n \t */\n \tif (config_source && config_source->use_stdin) {\n-\t\treturn git_config_from_stdin(fn, data);\n+\t\tret = git_config_from_stdin(fn, data);\n \t} else if (config_source && config_source->file) {\n-\t\treturn git_config_from_file(fn, config_source->file, data);\n+\t\tret = git_config_from_file(fn, config_source->file, data);\n \t} else if (config_source && config_source->blob) {\n \t\tstruct repository *repo = config_source->repo ?\n \t\t\tconfig_source->repo : the_repository;\n-\t\treturn git_config_from_blob_ref(fn, repo, config_source->blob,\n+\t\tret = git_config_from_blob_ref(fn, repo, config_source->blob,\n \t\t\t\t\t\tdata);\n+\t} else {\n+\t\tret = do_git_config_sequence(opts, fn, data);\n \t}\n \n-\treturn do_git_config_sequence(opts, fn, data);\n+\tif (inc.stashed_nr) {\n+\t\texecute_stashed(&inc);\n+\t\tinc.stashed_nr = 0;\n+\t}\n+\n+\tstring_list_clear(&inc.remote_urls, 0);\n+\treturn ret;\n }\n \n static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)\ndiff --git a/t/t1300-config.sh b/t/t1300-config.sh\nindex 9ff46f3b04..ea15f7fd46 100755\n--- a/t/t1300-config.sh\n+++ b/t/t1300-config.sh\n@@ -2387,4 +2387,64 @@ test_expect_success '--get and --get-all with --fixed-value' '\n \ttest_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n '\n \n+test_expect_success 'includeIf.hasremoteurl' '\n+\ttest_create_repo hasremoteurlTest &&\n+\n+\tcat >\"$(pwd)\"/include-this <<-\\EOF &&\n+\t[user]\n+\t\tthis = this-is-included\n+\tEOF\n+\tcat >\"$(pwd)\"/dont-include-that <<-\\EOF &&\n+\t[user]\n+\t\tthat = that-is-not-included\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasremoteurl:foo\"]\n+\t\tpath = \"$(pwd)/include-this\"\n+\t[includeIf \"hasremoteurl:bar\"]\n+\t\tpath = \"$(pwd)/dont-include-that\"\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\tEOF\n+\n+\techo this-is-included >expect-this &&\n+\tgit -C hasremoteurlTest config --get user.this >actual-this &&\n+\ttest_cmp expect-this actual-this &&\n+\n+\ttest_must_fail git -C hasremoteurlTest config --get user.that\n+'\n+\n+test_expect_success 'includeIf.hasremoteurl respects last-config-wins' '\n+\ttest_create_repo hasremoteurlTestOverride &&\n+\n+\tcat >\"$(pwd)\"/include-two-three <<-\\EOF &&\n+\t[user]\n+\t\ttwo = included-config\n+\t\tthree = included-config\n+\tEOF\n+\tcat >>hasremoteurlTestOverride/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\t[user]\n+\t\tone = main-config\n+\t\ttwo = main-config\n+\t[includeIf \"hasremoteurl:foo\"]\n+\t\tpath = \"$(pwd)/include-two-three\"\n+\t[user]\n+\t\tthree = main-config\n+\tEOF\n+\n+\techo main-config >expect-main-config &&\n+\techo included-config >expect-included-config &&\n+\n+\tgit -C hasremoteurlTestOverride config --get user.one >actual &&\n+\ttest_cmp expect-main-config actual &&\n+\n+\tgit -C hasremoteurlTestOverride config --get user.two >actual &&\n+\ttest_cmp expect-included-config actual &&\n+\n+\tgit -C hasremoteurlTestOverride config --get user.three >actual &&\n+\ttest_cmp expect-main-config actual\n+'\n+\n test_done\n-- \n2.33.1.1089.g2158813163f-goog\n\n"},{"id":"440548","messageId":"YYWJ57gFiSGIQI7j@google.com","threadId":"56696","inReplyTo":"3879e94c822421774001c823635f06235f50bfde.1635527389.git.jonathantanmy@google.com","subject":"Re: [WIP v2 1/2] config: make git_config_include() static","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-11-05T19:45:43Z","receivedAt":"2021-11-05T19:45:51Z","isPatch":false,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Fri, Oct 29, 2021 at 10:31:09AM -0700, Jonathan Tan wrote:\n> \n> It is not used from outside the file in which it is declared.\n> \n> Signed-off-by: Jonathan Tan \n> ---\n> config.c | 12 +++++++++++-\n> config.h | 37 ++++---------------------------------\n> 2 files changed, 15 insertions(+), 34 deletions(-)\n> \n> diff --git a/config.c b/config.c\n> index 2dcbe901b6..94ad5ce913 100644\n> --- a/config.c\n> +++ b/config.c\n> @@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n> \treturn conf->u.buf.pos;\n> }\n> \n> +struct config_include_data {\n> +\tint depth;\n> +\tconfig_fn_t fn;\n> +\tvoid *data;\n> +\tconst struct config_options *opts;\n> +};\n> +#define CONFIG_INCLUDE_INIT { 0 }\n> +\n> +static int git_config_include(const char *var, const char *value, void *data);\n> +\n> #define MAX_INCLUDE_DEPTH 10\n> static const char include_depth_advice[] = N_(\n> \"exceeded maximum include depth (%d) while including\\n\"\n> @@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n> \treturn 0;\n> }\n> \n> -int git_config_include(const char *var, const char *value, void *data)\n> +static int git_config_include(const char *var, const char *value, void *data)\n> {\n> \tstruct config_include_data *inc = data;\n> \tconst char *cond, *key;\n> diff --git a/config.h b/config.h\n> index f119de0130..48a5e472ca 100644\n> --- a/config.h\n> +++ b/config.h\n> @@ -126,6 +126,8 @@ int git_default_config(const char *, const char *, void *);\n> /**\n> * Read a specific file in git-config format.\n> * This function takes the same callback and data parameters as `git_config`.\n> + *\n> + * Unlike git_config(), this function does not respect includes.\n> */\n> int git_config_from_file(config_fn_t fn, const char *, void *);\n> \n> @@ -158,6 +160,8 @@ void read_very_early_config(config_fn_t cb, void *data);\n> * will first feed the user-wide one to the callback, and then the\n> * repo-specific one; by overwriting, the higher-priority repo-specific\n> * value is left at the end).\n> + *\n> + * Unlike git_config_from_file(), this function respects includes.\n> */\n> void git_config(config_fn_t fn, void *);\n> \n> @@ -338,39 +342,6 @@ const char *current_config_origin_type(void);\n> const char *current_config_name(void);\n> int current_config_line(void);\n> \n> -/**\n> - * Include Directives\n> - * ------------------\n> - *\n> - * By default, the config parser does not respect include directives.\n> - * However, a caller can use the special `git_config_include` wrapper\n> - * callback to support them. To do so, you simply wrap your \"real\" callback\n> - * function and data pointer in a `struct config_include_data`, and pass\n> - * the wrapper to the regular config-reading functions. For example:\n> - *\n> - * -------------------------------------------\n> - * int read_file_with_include(const char *file, config_fn_t fn, void *data)\n> - * {\n> - * struct config_include_data inc = CONFIG_INCLUDE_INIT;\n> - * inc.fn = fn;\n> - * inc.data = data;\n> - * return git_config_from_file(git_config_include, file, &inc);\n> - * }\n> - * -------------------------------------------\n> - *\n> - * `git_config` respects includes automatically. The lower-level\n> - * `git_config_from_file` does not.\n> - *\n> - */\n\nIt is a shame to lose the comprehensive usage documentation. Can we move\nit into the source near the static definition instead?\n\n> -struct config_include_data {\n> -\tint depth;\n> -\tconfig_fn_t fn;\n> -\tvoid *data;\n> -\tconst struct config_options *opts;\n> -};\n> -#define CONFIG_INCLUDE_INIT { 0 }\n> -int git_config_include(const char *name, const char *value, void *data);\n\nI wondered why we even had this here, if we were only calling\n'git_config_include()' from config.c. The last time this definition was\ntouched was when it was moved out of cache.h (e67a57fc518) and the time\nbefore that was when it was introduced in 2012 (9b25a0b52e0). At the\ntime of its introduction it was only called in config.c, anyways. So I\nguess it is just a matter of history.\n\nIt's still a WIP so I won't leave a reviewed-by line, but this patch\nlooks fine.\n\n - Emily\n\n> -\n> /*\n> * Match and parse a config key of the form:\n> *\n> -- \n> 2.33.1.1089.g2158813163f-goog\n> \n"},{"id":"440549","messageId":"YYWTCcAljHQRTJQ/@google.com","threadId":"56696","inReplyTo":"3c0d51ee39b8e149b5be57b8cd3f8cd403fe49c9.1635527389.git.jonathantanmy@google.com","subject":"Re: [WIP v2 2/2] config: include file if remote URL matches a glob","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-11-05T20:24:41Z","receivedAt":"2021-11-05T20:24:49Z","isPatch":false,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Fri, Oct 29, 2021 at 10:31:10AM -0700, Jonathan Tan wrote:\n> \n> This is a feature that supports config file inclusion conditional on\n> whether the repo has a remote with a URL that matches a glob.\n> \n> Similar to my previous work on remote-suggested hooks [1], the main\n> motivation is to allow remote repo administrators to provide recommended\n> configs in a way that can be consumed more easily (e.g. through a\n> package installable by a package manager).\n\nTo expand a little more on this:\n\nAt Google we ship /etc/gitconfig, as well as /usr/share/git-core/. Our\n/etc/gitconfig looks basically like:\n\n [include]\n path = /usr/share/git-core/gitconfig\n path = /usr/share/git-core/some-specific-config\n path = /usr/share/git-core/other-specific-config\n\nJonathan's WIP allows us to append lines to /etc/gitconfig sort of like\n\n [includeIf \"hasRemoteUrl:https://internal-google/big-project\"]\n path = /usr/share/big-project/gitconfig\n\nThat's approach #1 to shipping a config, which we might use for a\nproject that makes up a significant portion of our userbase. We ship\n(and own) the /etc/gitconfig; BigProject team ships and owns their own\ngitconfig; everybody internally who works on BigProject, whether it's\njust once to fix a small thing or every day as their main job, gets the\nrelevant configs for BigProject.\n\nApproach #2 I think is also still a useful one, and maybe more\ninteresting outside of Google:\n\nWhen I run 'sudo apt install big-oss-project-devkit', a few things\nhappen:\n1. /usr/share/big-oss-project/gitconfig appears\n2. `git config --global \\\n\t\t'includeIf.hasRemoteUrl:https://github/big-oss-project/*' \\\n\t\t'/usr/share/big-oss-project/gitconfig'` is run\n3. whatever other special tools, scripts, etc. are installed\n\nThat way regardless of which project I'm working on -\nbig-oss-project/translation, big-oss-project/docs,\nbig-oss-project/big-oss-project - I still get configs and style checkers\nand whatever else.\n\nWith this approach #2, it's still possible for someone to do a drive-by\ncontribution without ever running 'apt install big-oss-project-devkit',\nso it's not quite as strong a recommendation as the former\n\"remote-suggested-hooks\" topic. User would still want to take a look at\nthe README for big-oss-project to learn they're supposed to be\ninstalling that package ahead of time. But it's still a oneshot setup\nfor nice things like partial clone filters, maybe sparsity filters,\nmaybe config-based hooks, etc., especially if big-oss-project already\nwas shipping some project-specific tooling (like maybe a special\ndebugger or a docker image or I don't know).\n\nThe nice thing about 'hasRemoteUrl' in this case is that we don't need\nto know the location of the user's big-oss-project/ checkout on disk. We\ncan set that config globally and they can checkout big-oss-project as\nmany times and as many places as they wish. It wouldn't be possible to\nship configs via a package manager or other automated script without it.\n\n> \n> NEEDSWORK: The way this works is that if we see such an include, we\n> shunt all subsequent configs into a stash (while looking for URLs), then\n> process the stash. In particular, this means that more memory is needed,\n> and the nature of error reporting changes (currently, if a callback\n> returns nonzero for a variable, processing halts immediately, but with\n> this patch, all the config might be read from disk before the callback\n> even sees the variable). I'll need to expand on this and write a\n> documentation section.\n\nHm. I'm not so sure about making another structure for storing config\ninto memory, because we already do that during the regular config parse\n(to make things like git_config_get_string() fast). Can we not re-walk\nthe in-memory config at the end of the normal parse, rather than reading\nfrom disk twice?\n\nI think git_config()/repo_config() callback even does that for you for free...?\n\n2304 void repo_config(struct repository *repo, config_fn_t fn, void\n*data)\n2305 {\n2306 git_config_check_init(repo);\n2307 configset_iter(repo->config, fn, data);\n2308 }\n\n> \n> One alternative is to rerun the config parsing mechanism upon noticing\n> the first URL-conditional include in order to find all URLs. This would\n> require the config files to be read from disk twice, though.\n\nWhat's the easiest way to \"try it and see\", to add tooling and find out\nwhether the config files would be reopened during the second parse?\nBecause I suspect that we won't actually reopen those files, due to the\nconfig cache.\n\nSo couldn't we do something like....\n\npass #1:\n if (include)\n if (not hasRemoteUrl)\n open up path & parse\n put config into in-memory cache normally\npass #2: (and this pass would need to be added to repo_config() probably)\n if (include)\n if (hasRemoteUrl)\n open up path & parse\n insert in-order into in-memory cache\n don't touch existing configs otherwise\n\nI think it's in practice similar to the approach you're using (getting\naround the weird ordering with a cache in memory), but we could reuse\nthe existing config cache rather than creating a new and different one.\n\n - Emily\n"},{"id":"440568","messageId":"211106.8635o9hogz.gmgdl@evledraar.gmail.com","threadId":"56696","inReplyTo":"YYWTCcAljHQRTJQ/@google.com","subject":"Re: [WIP v2 2/2] config: include file if remote URL matches a glob","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-11-06T04:41:38Z","receivedAt":"2021-11-06T04:56:01Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Nov 05 2021, Emily Shaffer wrote:\n\n> On Fri, Oct 29, 2021 at 10:31:10AM -0700, Jonathan Tan wrote:\n> [...]\n>> \n>> One alternative is to rerun the config parsing mechanism upon noticing\n>> the first URL-conditional include in order to find all URLs. This would\n>> require the config files to be read from disk twice, though.\n>\n> What's the easiest way to \"try it and see\", to add tooling and find out\n> whether the config files would be reopened during the second parse?\n> Because I suspect that we won't actually reopen those files, due to the\n> config cache.\n\nstrace -f?\n\n> So couldn't we do something like....\n>\n> pass #1:\n> if (include)\n> if (not hasRemoteUrl)\n> open up path & parse\n> put config into in-memory cache normally\n> pass #2: (and this pass would need to be added to repo_config() probably)\n> if (include)\n> if (hasRemoteUrl)\n> open up path & parse\n> insert in-order into in-memory cache\n> don't touch existing configs otherwise\n>\n> I think it's in practice similar to the approach you're using (getting\n> around the weird ordering with a cache in memory), but we could reuse\n> the existing config cache rather than creating a new and different one.\n\nI don't know enough to say if this two-step approach is better (although\nI'm slightly biased in that direction, since it seems simpler), but this\njust seems like premature optimization.\n\nI.e. let's just read the files twice, they'll be in the OS's FS cache,\nwhich is unlikely to be a bottleneck for the amount of files involved.\n\nThat being said we do have exactly this cache already. See [1] and\n3c8687a73ee (add `config_set` API for caching config-like files,\n2014-07-28).\n\nBut I think that was added due to *very* frequent re-parsing of the\nentire config every time someone needed a config variable, not due to\nthe I/O overhead (but I may be wrong).\n\nSo if we've got 100 config variables we need and 10 config files then\n10*100 is probably starting to hurt, but if for whatever reason we\nneeded 2*10 here that's probably no big deal, and in any case would only\nhappen if this new include mechanism was in play.\n\n1. https://lore.kernel.org/git/1404631162-18556-1-git-send-email-tanayabh@gmail.com/ \n\n"},{"id":"440709","messageId":"20211109002255.1110653-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"YYWTCcAljHQRTJQ/@google.com","subject":"Re: [WIP v2 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-09T00:22:55Z","receivedAt":"2021-11-09T00:22:59Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> To expand a little more on this:\n\n[snip]\n\n> The nice thing about 'hasRemoteUrl' in this case is that we don't need\n> to know the location of the user's big-oss-project/ checkout on disk. We\n> can set that config globally and they can checkout big-oss-project as\n> many times and as many places as they wish. It wouldn't be possible to\n> ship configs via a package manager or other automated script without it.\n\nAh, thanks for the elaboration!\n\n> > NEEDSWORK: The way this works is that if we see such an include, we\n> > shunt all subsequent configs into a stash (while looking for URLs), then\n> > process the stash. In particular, this means that more memory is needed,\n> > and the nature of error reporting changes (currently, if a callback\n> > returns nonzero for a variable, processing halts immediately, but with\n> > this patch, all the config might be read from disk before the callback\n> > even sees the variable). I'll need to expand on this and write a\n> > documentation section.\n> \n> Hm. I'm not so sure about making another structure for storing config\n> into memory, because we already do that during the regular config parse\n> (to make things like git_config_get_string() fast). Can we not re-walk\n> the in-memory config at the end of the normal parse, rather than reading\n> from disk twice?\n> \n> I think git_config()/repo_config() callback even does that for you for free...?\n\nThe main thing is that we wouldn't know if an entry would have been\noverridden by a value from an includeif.hasremoteurl or not.\n\n> What's the easiest way to \"try it and see\", to add tooling and find out\n> whether the config files would be reopened during the second parse?\n> Because I suspect that we won't actually reopen those files, due to the\n> config cache.\n\nAs Ævar says, strace should work. The hard part is implementing the\nrecursive config parse, but it looks like the way to go, so I'll try it\nand see how it goes.\n\n[1] https://lore.kernel.org/git/211106.8635o9hogz.gmgdl@evledraar.gmail.com/\n\n> So couldn't we do something like....\n> \n> pass #1:\n> if (include)\n> if (not hasRemoteUrl)\n> open up path & parse\n> put config into in-memory cache normally\n> pass #2: (and this pass would need to be added to repo_config() probably)\n> if (include)\n> if (hasRemoteUrl)\n> open up path & parse\n> insert in-order into in-memory cache\n> don't touch existing configs otherwise\n>\n> I think it's in practice similar to the approach you're using (getting\n> around the weird ordering with a cache in memory), but we could reuse\n> the existing config cache rather than creating a new and different one.\n\nWhat do you mean by \"insert in-order\"? If you mean figuring out which\nvariables would be overridden (and for multi-valued variables, what\norder to put all the values in), I think that's the hard part.\n\nAnother thing is that at the point where we read the config\n(config_with_options()), we have a callback, so we would need to make\nsure that we're writing to the in-memory cache in the first place (as\nopposed to passing a callback that does something else). That might be\ndoable by changing the API, but in ay case, I'll try the recursive\nconfig parse first.\n"},{"id":"440710","messageId":"20211109002536.1111372-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"211106.8635o9hogz.gmgdl@evledraar.gmail.com","subject":"Re: [WIP v2 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-09T00:25:36Z","receivedAt":"2021-11-09T00:25:40Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> > What's the easiest way to \"try it and see\", to add tooling and find out\n> > whether the config files would be reopened during the second parse?\n> > Because I suspect that we won't actually reopen those files, due to the\n> > config cache.\n> \n> strace -f?\n\nThanks - this might work.\n\n> > So couldn't we do something like....\n> >\n> > pass #1:\n> > if (include)\n> > if (not hasRemoteUrl)\n> > open up path & parse\n> > put config into in-memory cache normally\n> > pass #2: (and this pass would need to be added to repo_config() probably)\n> > if (include)\n> > if (hasRemoteUrl)\n> > open up path & parse\n> > insert in-order into in-memory cache\n> > don't touch existing configs otherwise\n> >\n> > I think it's in practice similar to the approach you're using (getting\n> > around the weird ordering with a cache in memory), but we could reuse\n> > the existing config cache rather than creating a new and different one.\n> \n> I don't know enough to say if this two-step approach is better (although\n> I'm slightly biased in that direction, since it seems simpler), but this\n> just seems like premature optimization.\n> \n> I.e. let's just read the files twice, they'll be in the OS's FS cache,\n> which is unlikely to be a bottleneck for the amount of files involved.\n\nOK - let me try this.\n\n> That being said we do have exactly this cache already. See [1] and\n> 3c8687a73ee (add `config_set` API for caching config-like files,\n> 2014-07-28).\n> \n> But I think that was added due to *very* frequent re-parsing of the\n> entire config every time someone needed a config variable, not due to\n> the I/O overhead (but I may be wrong).\n> \n> So if we've got 100 config variables we need and 10 config files then\n> 10*100 is probably starting to hurt, but if for whatever reason we\n> needed 2*10 here that's probably no big deal, and in any case would only\n> happen if this new include mechanism was in play.\n> \n> 1. https://lore.kernel.org/git/1404631162-18556-1-git-send-email-tanayabh@gmail.com/ \n\nThis might not work for the reasons I described in my reply to Emily\n[1]. I'll try the read-twice version first.\n\n[1] https://lore.kernel.org/git/20211109002255.1110653-1-jonathantanmy@google.com/\n"},{"id":"441236","messageId":"cover.1637020610.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"[PATCH v3 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-16T00:00:31Z","receivedAt":"2021-11-16T03:41:38Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Here's a version that starts a second traversal of the config file read\nwhen encountering the first remote URL. No user-visible changes from v2,\nbut hopefully the algorithm is simpler.\n\nI've also added the user-facing documentation and a test of what the\nglob pattern should match.\n\nSome people have suggested avoiding the forward declaration in patch 1,\nbut I found that there are 2 functions that call each other, so the\nforward declaration cannot be avoided.\n\nJonathan Tan (2):\n config: make git_config_include() static\n config: include file if remote URL matches a glob\n\n Documentation/config.txt | 11 ++++\n config.c | 133 ++++++++++++++++++++++++++++++++++++---\n config.h | 44 ++++---------\n t/t1300-config.sh | 100 +++++++++++++++++++++++++++++\n 4 files changed, 246 insertions(+), 42 deletions(-)\n\n-- \n2.34.0.rc1.387.gb447b232ab-goog\n\n"},{"id":"441237","messageId":"b2dcae03ed3016f96a5368db925fa3736c3d7c58.1637020610.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1637020610.git.jonathantanmy@google.com","subject":"[PATCH v3 1/2] config: make git_config_include() static","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-16T00:00:32Z","receivedAt":"2021-11-16T03:41:42Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"It is not used from outside the file in which it is declared.\n\nSigned-off-by: Jonathan Tan \n---\n config.c | 12 +++++++++++-\n config.h | 37 ++++---------------------------------\n 2 files changed, 15 insertions(+), 34 deletions(-)\n\ndiff --git a/config.c b/config.c\nindex 2dcbe901b6..94ad5ce913 100644\n--- a/config.c\n+++ b/config.c\n@@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n \treturn conf->u.buf.pos;\n }\n \n+struct config_include_data {\n+\tint depth;\n+\tconfig_fn_t fn;\n+\tvoid *data;\n+\tconst struct config_options *opts;\n+};\n+#define CONFIG_INCLUDE_INIT { 0 }\n+\n+static int git_config_include(const char *var, const char *value, void *data);\n+\n #define MAX_INCLUDE_DEPTH 10\n static const char include_depth_advice[] = N_(\n \"exceeded maximum include depth (%d) while including\\n\"\n@@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n-int git_config_include(const char *var, const char *value, void *data)\n+static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n \tconst char *cond, *key;\ndiff --git a/config.h b/config.h\nindex f119de0130..48a5e472ca 100644\n--- a/config.h\n+++ b/config.h\n@@ -126,6 +126,8 @@ int git_default_config(const char *, const char *, void *);\n /**\n * Read a specific file in git-config format.\n * This function takes the same callback and data parameters as `git_config`.\n+ *\n+ * Unlike git_config(), this function does not respect includes.\n */\n int git_config_from_file(config_fn_t fn, const char *, void *);\n \n@@ -158,6 +160,8 @@ void read_very_early_config(config_fn_t cb, void *data);\n * will first feed the user-wide one to the callback, and then the\n * repo-specific one; by overwriting, the higher-priority repo-specific\n * value is left at the end).\n+ *\n+ * Unlike git_config_from_file(), this function respects includes.\n */\n void git_config(config_fn_t fn, void *);\n \n@@ -338,39 +342,6 @@ const char *current_config_origin_type(void);\n const char *current_config_name(void);\n int current_config_line(void);\n \n-/**\n- * Include Directives\n- * ------------------\n- *\n- * By default, the config parser does not respect include directives.\n- * However, a caller can use the special `git_config_include` wrapper\n- * callback to support them. To do so, you simply wrap your \"real\" callback\n- * function and data pointer in a `struct config_include_data`, and pass\n- * the wrapper to the regular config-reading functions. For example:\n- *\n- * -------------------------------------------\n- * int read_file_with_include(const char *file, config_fn_t fn, void *data)\n- * {\n- * struct config_include_data inc = CONFIG_INCLUDE_INIT;\n- * inc.fn = fn;\n- * inc.data = data;\n- * return git_config_from_file(git_config_include, file, &inc);\n- * }\n- * -------------------------------------------\n- *\n- * `git_config` respects includes automatically. The lower-level\n- * `git_config_from_file` does not.\n- *\n- */\n-struct config_include_data {\n-\tint depth;\n-\tconfig_fn_t fn;\n-\tvoid *data;\n-\tconst struct config_options *opts;\n-};\n-#define CONFIG_INCLUDE_INIT { 0 }\n-int git_config_include(const char *name, const char *value, void *data);\n-\n /*\n * Match and parse a config key of the form:\n *\n-- \n2.34.0.rc1.387.gb447b232ab-goog\n\n"},{"id":"441238","messageId":"1c1a07a0b65d4bbbb0f2628a3ddf1980e37d5065.1637020610.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1637020610.git.jonathantanmy@google.com","subject":"[PATCH v3 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-16T00:00:33Z","receivedAt":"2021-11-16T03:41:44Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This is a feature that supports config file inclusion conditional on\nwhether the repo has a remote with a URL that matches a glob.\n\nSimilar to my previous work on remote-suggested hooks [1], the main\nmotivation is to allow remote repo administrators to provide recommended\nconfigs in a way that can be consumed more easily (e.g. through a\npackage installable by a package manager - it could, for example,\ncontain a file to be included conditionally and a post-install script\nthat adds the include directive to the system-wide config file).\n\nIn order to do this, Git reruns the config parsing mechanism upon\nnoticing the first URL-conditional include in order to find all remote\nURLs, and these remote URLs are then used to determine if that first and\nall subsequent includes are executed. Remote URLs are not allowed to be\nconfigued in any URL-conditionally-included file.\n\n[1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n\nSigned-off-by: Jonathan Tan \n---\n Documentation/config.txt | 11 ++++\n config.c | 121 ++++++++++++++++++++++++++++++++++++---\n config.h | 7 +++\n t/t1300-config.sh | 100 ++++++++++++++++++++++++++++++++\n 4 files changed, 231 insertions(+), 8 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 0c0e6b859f..93d18b2fe9 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -159,6 +159,17 @@ all branches that begin with `foo/`. This is useful if your branches are\n organized hierarchically and you would like to apply a configuration to\n all the branches in that hierarchy.\n \n+`hasremoteurl`::\n+\tThe data that follows the keyword `hasremoteurl:` is taken to\n+\tbe a pattern with standard globbing wildcards and two\n+\tadditional ones, `**/` and `/**`, that can match multiple\n+\tcomponents. The rest of the config files will be scanned for\n+\tremote URLs, and then if there at least one remote URL that\n+\tmatches this pattern, the include condition is met.\n++\n+Files included by this option (directly or indirectly) are not allowed\n+to contain remote URLs.\n+\n A few more notes on matching via `gitdir` and `gitdir/i`:\n \n * Symlinks in `$GIT_DIR` are not resolved before matching.\ndiff --git a/config.c b/config.c\nindex 94ad5ce913..4ffc1e87e9 100644\n--- a/config.c\n+++ b/config.c\n@@ -125,6 +125,12 @@ struct config_include_data {\n \tconfig_fn_t fn;\n \tvoid *data;\n \tconst struct config_options *opts;\n+\tstruct git_config_source *config_source;\n+\n+\t/*\n+\t * All remote URLs discovered when reading all config files.\n+\t */\n+\tstruct string_list *remote_urls;\n };\n #define CONFIG_INCLUDE_INIT { 0 }\n \n@@ -316,12 +322,83 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n+static int add_remote_url(const char *var, const char *value, void *data)\n+{\n+\tstruct string_list *remote_urls = data;\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tstring_list_append(remote_urls, value);\n+\treturn 0;\n+}\n+\n+static void populate_remote_urls(struct config_include_data *inc)\n+{\n+\tstruct config_options opts;\n+\n+\tstruct config_source *store_cf = cf;\n+\tstruct key_value_info *store_kvi = current_config_kvi;\n+\tenum config_scope store_scope = current_parsing_scope;\n+\n+\topts = *inc->opts;\n+\topts.unconditional_remote_url = 1;\n+\n+\tcf = NULL;\n+\tcurrent_config_kvi = NULL;\n+\tcurrent_parsing_scope = 0;\n+\n+\tinc->remote_urls = xmalloc(sizeof(*inc->remote_urls));\n+\tstring_list_init_dup(inc->remote_urls);\n+\tconfig_with_options(add_remote_url, inc->remote_urls, inc->config_source, &opts);\n+\n+\tcf = store_cf;\n+\tcurrent_config_kvi = store_kvi;\n+\tcurrent_parsing_scope = store_scope;\n+}\n+\n+static int forbid_remote_url(const char *var, const char *value, void *data)\n+{\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasremoteurl\"));\n+\treturn 0;\n+}\n+\n+static int at_least_one_url_matches_glob(const char *glob, int glob_len,\n+\t\t\t\t\t struct string_list *remote_urls)\n+{\n+\tstruct strbuf pattern = STRBUF_INIT;\n+\tstruct string_list_item *url_item;\n+\tint found = 0;\n+\n+\tstrbuf_add(&pattern, glob, glob_len);\n+\tfor_each_string_list_item(url_item, remote_urls) {\n+\t\tif (!wildmatch(pattern.buf, url_item->string, WM_PATHNAME)) {\n+\t\t\tfound = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tstrbuf_release(&pattern);\n+\treturn found;\n+}\n+\n static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n \tconst char *cond, *key;\n \tsize_t cond_len;\n-\tint ret;\n+\tint ret = 0;\n \n \t/*\n \t * Pass along all values, including \"include\" directives; this makes it\n@@ -335,9 +412,29 @@ static int git_config_include(const char *var, const char *value, void *data)\n \t\tret = handle_path_include(value, inc);\n \n \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n-\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n-\t !strcmp(key, \"path\"))\n-\t\tret = handle_path_include(value, inc);\n+\t cond && !strcmp(key, \"path\")) {\n+\t\tconst char *url;\n+\t\tsize_t url_len;\n+\n+\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &url,\n+\t\t\t\t &url_len)) {\n+\t\t\tif (inc->opts->unconditional_remote_url) {\n+\t\t\t\tconfig_fn_t old_fn = inc->fn;\n+\n+\t\t\t\tinc->fn = forbid_remote_url;\n+\t\t\t\tret = handle_path_include(value, inc);\n+\t\t\t\tinc->fn = old_fn;\n+\t\t\t} else {\n+\t\t\t\tif (!inc->remote_urls)\n+\t\t\t\t\tpopulate_remote_urls(inc);\n+\t\t\t\tif (at_least_one_url_matches_glob(\n+\t\t\t\t\t\turl, url_len, inc->remote_urls))\n+\t\t\t\t\tret = handle_path_include(value, inc);\n+\t\t\t}\n+\t\t} else if (include_condition_is_true(inc->opts, cond, cond_len)) {\n+\t\t\tret = handle_path_include(value, inc);\n+\t\t}\n+\t}\n \n \treturn ret;\n }\n@@ -1933,11 +2030,13 @@ int config_with_options(config_fn_t fn, void *data,\n \t\t\tconst struct config_options *opts)\n {\n \tstruct config_include_data inc = CONFIG_INCLUDE_INIT;\n+\tint ret;\n \n \tif (opts->respect_includes) {\n \t\tinc.fn = fn;\n \t\tinc.data = data;\n \t\tinc.opts = opts;\n+\t\tinc.config_source = config_source;\n \t\tfn = git_config_include;\n \t\tdata = &inc;\n \t}\n@@ -1950,17 +2049,23 @@ int config_with_options(config_fn_t fn, void *data,\n \t * regular lookup sequence.\n \t */\n \tif (config_source && config_source->use_stdin) {\n-\t\treturn git_config_from_stdin(fn, data);\n+\t\tret = git_config_from_stdin(fn, data);\n \t} else if (config_source && config_source->file) {\n-\t\treturn git_config_from_file(fn, config_source->file, data);\n+\t\tret = git_config_from_file(fn, config_source->file, data);\n \t} else if (config_source && config_source->blob) {\n \t\tstruct repository *repo = config_source->repo ?\n \t\t\tconfig_source->repo : the_repository;\n-\t\treturn git_config_from_blob_ref(fn, repo, config_source->blob,\n+\t\tret = git_config_from_blob_ref(fn, repo, config_source->blob,\n \t\t\t\t\t\tdata);\n+\t} else {\n+\t\tret = do_git_config_sequence(opts, fn, data);\n \t}\n \n-\treturn do_git_config_sequence(opts, fn, data);\n+\tif (inc.remote_urls) {\n+\t\tstring_list_clear(inc.remote_urls, 0);\n+\t\tFREE_AND_NULL(inc.remote_urls);\n+\t}\n+\treturn ret;\n }\n \n static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)\ndiff --git a/config.h b/config.h\nindex 48a5e472ca..c24458b10a 100644\n--- a/config.h\n+++ b/config.h\n@@ -89,6 +89,13 @@ struct config_options {\n \tunsigned int ignore_worktree : 1;\n \tunsigned int ignore_cmdline : 1;\n \tunsigned int system_gently : 1;\n+\n+\t/*\n+\t * For internal use. Include all includeif.hasremoteurl paths without\n+\t * checking if the repo has that remote URL.\n+\t */\n+\tunsigned int unconditional_remote_url : 1;\n+\n \tconst char *commondir;\n \tconst char *git_dir;\n \tconfig_parser_event_fn_t event_fn;\ndiff --git a/t/t1300-config.sh b/t/t1300-config.sh\nindex 9ff46f3b04..9daab4c6da 100755\n--- a/t/t1300-config.sh\n+++ b/t/t1300-config.sh\n@@ -2387,4 +2387,104 @@ test_expect_success '--get and --get-all with --fixed-value' '\n \ttest_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n '\n \n+test_expect_success 'includeIf.hasremoteurl' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >\"$(pwd)\"/include-this <<-\\EOF &&\n+\t[user]\n+\t\tthis = this-is-included\n+\tEOF\n+\tcat >\"$(pwd)\"/dont-include-that <<-\\EOF &&\n+\t[user]\n+\t\tthat = that-is-not-included\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasremoteurl:foo\"]\n+\t\tpath = \"$(pwd)/include-this\"\n+\t[includeIf \"hasremoteurl:bar\"]\n+\t\tpath = \"$(pwd)/dont-include-that\"\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\tEOF\n+\n+\techo this-is-included >expect-this &&\n+\tgit -C hasremoteurlTest config --get user.this >actual-this &&\n+\ttest_cmp expect-this actual-this &&\n+\n+\ttest_must_fail git -C hasremoteurlTest config --get user.that\n+'\n+\n+test_expect_success 'includeIf.hasremoteurl respects last-config-wins' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >\"$(pwd)\"/include-two-three <<-\\EOF &&\n+\t[user]\n+\t\ttwo = included-config\n+\t\tthree = included-config\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\t[user]\n+\t\tone = main-config\n+\t\ttwo = main-config\n+\t[includeIf \"hasremoteurl:foo\"]\n+\t\tpath = \"$(pwd)/include-two-three\"\n+\t[user]\n+\t\tthree = main-config\n+\tEOF\n+\n+\techo main-config >expect-main-config &&\n+\techo included-config >expect-included-config &&\n+\n+\tgit -C hasremoteurlTest config --get user.one >actual &&\n+\ttest_cmp expect-main-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.two >actual &&\n+\ttest_cmp expect-included-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.three >actual &&\n+\ttest_cmp expect-main-config actual\n+'\n+\n+test_expect_success 'includeIf.hasremoteurl globs' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tprintf \"[user]\\ndss = yes\\n\" >\"$(pwd)/double-star-start\" &&\n+\tprintf \"[user]\\ndse = yes\\n\" >\"$(pwd)/double-star-end\" &&\n+\tprintf \"[user]\\ndsm = yes\\n\" >\"$(pwd)/double-star-middle\" &&\n+\tprintf \"[user]\\nssm = yes\\n\" >\"$(pwd)/single-star-middle\" &&\n+\tprintf \"[user]\\nno = no\\n\" >\"$(pwd)/no\" &&\n+\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = https://foo/bar/baz\n+\t[includeIf \"hasremoteurl:**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-start\"\n+\t[includeIf \"hasremoteurl:**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasremoteurl:https:/**\"]\n+\t\tpath = \"$(pwd)/double-star-end\"\n+\t[includeIf \"hasremoteurl:nomatch:/**\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasremoteurl:https:/**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-middle\"\n+\t[includeIf \"hasremoteurl:https:/**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasremoteurl:https://*/bar/baz\"]\n+\t\tpath = \"$(pwd)/single-star-middle\"\n+\t[includeIf \"hasremoteurl:https://*/baz\"]\n+\t\tpath = \"$(pwd)/no\"\n+\tEOF\n+\n+\tgit -C hasremoteurlTest config --get user.dss &&\n+\tgit -C hasremoteurlTest config --get user.dse &&\n+\tgit -C hasremoteurlTest config --get user.dsm &&\n+\tgit -C hasremoteurlTest config --get user.ssm &&\n+\ttest_must_fail git -C hasremoteurlTest config --get user.no\n+'\n+\n test_done\n-- \n2.34.0.rc1.387.gb447b232ab-goog\n\n"},{"id":"442059","messageId":"kl6lilwjre3m.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56696","inReplyTo":"1c1a07a0b65d4bbbb0f2628a3ddf1980e37d5065.1637020610.git.jonathantanmy@google.com","subject":"Re: [PATCH v3 2/2] config: include file if remote URL matches a glob","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2021-11-22T22:59:41Z","receivedAt":"2021-11-22T22:59:45Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Jonathan Tan writes:\n\n> +`hasremoteurl`::\n> +\tThe data that follows the keyword `hasremoteurl:` is taken to\n> +\tbe a pattern with standard globbing wildcards and two\n> +\tadditional ones, `**/` and `/**`, that can match multiple\n> +\tcomponents. The rest of the config files will be scanned for\n> +\tremote URLs, and then if there at least one remote URL that\n\n if there {is,exists}* at least one remote URL that\n\n> +\tmatches this pattern, the include condition is met.\n> ++\n> +Files included by this option (directly or indirectly) are not allowed\n> +to contain remote URLs.\n\nAs Jeff mentioned earlier in this thread, this \"last-config-wins\" is a\npretty big exception to the existing semantics, as\nDocumentation/config.txt reads:\n\n The contents of the included file are inserted immediately, as if they\n had been found at the location of the include directive.\n\nAt minimum, I think we should call out this exception in\nDocumentation/config.txt and the commit message, but calling out *just*\nhasremoteurl makes this exception seem like a strange anomaly at first\nglance, even though we actually have a good idea of when and why we are\ndoing this (which is that it simplifies includes that rely on config\nvalues).\n\nI was a big fan of your includeIfDeferred proposal, and I still think\nthat it's easier for users to understand if we explicitly require\n\"includeIfDeferred\" instead of counting on them to remember when\n\"includeIf\" behaves as it always did vs this new 'deferred' behavior.\nThat said, I doubt most users actually rely on the inclusion order, and \nI am ok with this approach as long as we document the different\ninclusion order.\n\n\n> +static void populate_remote_urls(struct config_include_data *inc)\n> +{\n> +\tstruct config_options opts;\n> +\n> +\tstruct config_source *store_cf = cf;\n> +\tstruct key_value_info *store_kvi = current_config_kvi;\n> +\tenum config_scope store_scope = current_parsing_scope;\n> +\n> +\topts = *inc->opts;\n> +\topts.unconditional_remote_url = 1;\n> +\n> +\tcf = NULL;\n> +\tcurrent_config_kvi = NULL;\n> +\tcurrent_parsing_scope = 0;\n> +\n> +\tinc->remote_urls = xmalloc(sizeof(*inc->remote_urls));\n> +\tstring_list_init_dup(inc->remote_urls);\n> +\tconfig_with_options(add_remote_url, inc->remote_urls, inc->config_source, &opts);\n> +\n> +\tcf = store_cf;\n> +\tcurrent_config_kvi = store_kvi;\n> +\tcurrent_parsing_scope = store_scope;\n> +}\n\nThe algorithm is easy to understand and reuses config_with_options(),\nwhich is great.\n\n> +static int forbid_remote_url(const char *var, const char *value, void *data)\n> +{\n> +\tconst char *remote_name;\n> +\tsize_t remote_name_len;\n> +\tconst char *key;\n> +\n> +\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n> +\t\t\t &key) &&\n> +\t remote_name &&\n> +\t !strcmp(key, \"url\"))\n> +\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasremoteurl\"));\n> +\treturn 0;\n> +}\n> +\n> +static int at_least_one_url_matches_glob(const char *glob, int glob_len,\n> +\t\t\t\t\t struct string_list *remote_urls)\n> +{\n> +\tstruct strbuf pattern = STRBUF_INIT;\n> +\tstruct string_list_item *url_item;\n> +\tint found = 0;\n> +\n> +\tstrbuf_add(&pattern, glob, glob_len);\n> +\tfor_each_string_list_item(url_item, remote_urls) {\n> +\t\tif (!wildmatch(pattern.buf, url_item->string, WM_PATHNAME)) {\n> +\t\t\tfound = 1;\n> +\t\t\tbreak;\n> +\t\t}\n> +\t}\n> +\tstrbuf_release(&pattern);\n> +\treturn found;\n> +}\n> +\n> static int git_config_include(const char *var, const char *value, void *data)\n> {\n> \tstruct config_include_data *inc = data;\n> \tconst char *cond, *key;\n> \tsize_t cond_len;\n> -\tint ret;\n> +\tint ret = 0;\n> \n> \t/*\n> \t * Pass along all values, including \"include\" directives; this makes it\n> @@ -335,9 +412,29 @@ static int git_config_include(const char *var, const char *value, void *data)\n> \t\tret = handle_path_include(value, inc);\n> \n> \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n> -\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n> -\t !strcmp(key, \"path\"))\n> -\t\tret = handle_path_include(value, inc);\n> +\t cond && !strcmp(key, \"path\")) {\n> +\t\tconst char *url;\n> +\t\tsize_t url_len;\n> +\n> +\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &url,\n> +\t\t\t\t &url_len)) {\n> +\t\t\tif (inc->opts->unconditional_remote_url) {\n> +\t\t\t\tconfig_fn_t old_fn = inc->fn;\n> +\n> +\t\t\t\tinc->fn = forbid_remote_url;\n\nWhen unconditional_remote_url is true, we forbid remote urls in the\nincluded files as expected, but...\n\n> +\t\t\t\tret = handle_path_include(value, inc);\n> +\t\t\t\tinc->fn = old_fn;\n> +\t\t\t} else {\n> +\t\t\t\tif (!inc->remote_urls)\n> +\t\t\t\t\tpopulate_remote_urls(inc);\n> +\t\t\t\tif (at_least_one_url_matches_glob(\n> +\t\t\t\t\t\turl, url_len, inc->remote_urls))\n> +\t\t\t\t\tret = handle_path_include(value, inc);\n> +\t\t\t}\n> +\t\t} else if (include_condition_is_true(inc->opts, cond, cond_len)) {\n> +\t\t\tret = handle_path_include(value, inc);\n> +\t\t}\n> +\t}\n> \n> \treturn ret;\n> }\n\nIt's not clear to me whether we are forbidding the remote urls correctly\nwhen uncondition_remote_url is false. I would be convinced if we had\ntests that convered this behavior, but I did not find any such test\ncases.\n\n> diff --git a/t/t1300-config.sh b/t/t1300-config.sh\n> index 9ff46f3b04..9daab4c6da 100755\n> --- a/t/t1300-config.sh\n> +++ b/t/t1300-config.sh\n> @@ -2387,4 +2387,104 @@ test_expect_success '--get and --get-all with --fixed-value' '\n> \ttest_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n> '\n> \n> +test_expect_success 'includeIf.hasremoteurl' '\n> +\tgit init hasremoteurlTest &&\n> +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n> +\n> +\tcat >\"$(pwd)\"/include-this <<-\\EOF &&\n> +\t[user]\n> +\t\tthis = this-is-included\n> +\tEOF\n> +\tcat >\"$(pwd)\"/dont-include-that <<-\\EOF &&\n> +\t[user]\n> +\t\tthat = that-is-not-included\n> +\tEOF\n> +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n> +\t[includeIf \"hasremoteurl:foo\"]\n> +\t\tpath = \"$(pwd)/include-this\"\n> +\t[includeIf \"hasremoteurl:bar\"]\n> +\t\tpath = \"$(pwd)/dont-include-that\"\n> +\t[remote \"foo\"]\n> +\t\turl = foo\n> +\tEOF\n> +\n> +\techo this-is-included >expect-this &&\n> +\tgit -C hasremoteurlTest config --get user.this >actual-this &&\n> +\ttest_cmp expect-this actual-this &&\n> +\n> +\ttest_must_fail git -C hasremoteurlTest config --get user.that\n> +'\n> +\n> +test_expect_success 'includeIf.hasremoteurl respects last-config-wins' '\n> +\tgit init hasremoteurlTest &&\n> +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n> +\n> +\tcat >\"$(pwd)\"/include-two-three <<-\\EOF &&\n> +\t[user]\n> +\t\ttwo = included-config\n> +\t\tthree = included-config\n> +\tEOF\n> +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n> +\t[remote \"foo\"]\n> +\t\turl = foo\n> +\t[user]\n> +\t\tone = main-config\n> +\t\ttwo = main-config\n> +\t[includeIf \"hasremoteurl:foo\"]\n> +\t\tpath = \"$(pwd)/include-two-three\"\n> +\t[user]\n> +\t\tthree = main-config\n> +\tEOF\n> +\n> +\techo main-config >expect-main-config &&\n> +\techo included-config >expect-included-config &&\n> +\n> +\tgit -C hasremoteurlTest config --get user.one >actual &&\n> +\ttest_cmp expect-main-config actual &&\n> +\n> +\tgit -C hasremoteurlTest config --get user.two >actual &&\n> +\ttest_cmp expect-included-config actual &&\n> +\n> +\tgit -C hasremoteurlTest config --get user.three >actual &&\n> +\ttest_cmp expect-main-config actual\n> +'\n> +\n> +test_expect_success 'includeIf.hasremoteurl globs' '\n> +\tgit init hasremoteurlTest &&\n> +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n> +\n> +\tprintf \"[user]\\ndss = yes\\n\" >\"$(pwd)/double-star-start\" &&\n> +\tprintf \"[user]\\ndse = yes\\n\" >\"$(pwd)/double-star-end\" &&\n> +\tprintf \"[user]\\ndsm = yes\\n\" >\"$(pwd)/double-star-middle\" &&\n> +\tprintf \"[user]\\nssm = yes\\n\" >\"$(pwd)/single-star-middle\" &&\n> +\tprintf \"[user]\\nno = no\\n\" >\"$(pwd)/no\" &&\n> +\n> +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n> +\t[remote \"foo\"]\n> +\t\turl = https://foo/bar/baz\n> +\t[includeIf \"hasremoteurl:**/baz\"]\n> +\t\tpath = \"$(pwd)/double-star-start\"\n> +\t[includeIf \"hasremoteurl:**/nomatch\"]\n> +\t\tpath = \"$(pwd)/no\"\n> +\t[includeIf \"hasremoteurl:https:/**\"]\n> +\t\tpath = \"$(pwd)/double-star-end\"\n> +\t[includeIf \"hasremoteurl:nomatch:/**\"]\n> +\t\tpath = \"$(pwd)/no\"\n\nAs mentioned above, I would have expected to find test cases that test\nwhether or not we forbid the remote urls correctly, but the tests are\npretty clear.\n"},{"id":"442078","messageId":"xmqqtug3aco7.fsf@gitster.g","threadId":"56696","inReplyTo":"1c1a07a0b65d4bbbb0f2628a3ddf1980e37d5065.1637020610.git.jonathantanmy@google.com","subject":"Re: [PATCH v3 2/2] config: include file if remote URL matches a glob","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-23T01:22:32Z","receivedAt":"2021-11-23T01:22:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> This is a feature that supports config file inclusion conditional on\n> whether the repo has a remote with a URL that matches a glob.\n>\n> Similar to my previous work on remote-suggested hooks [1], the main\n> motivation is to allow remote repo administrators to provide recommended\n> configs in a way that can be consumed more easily (e.g. through a\n> package installable by a package manager - it could, for example,\n> contain a file to be included conditionally and a post-install script\n> that adds the include directive to the system-wide config file).\n>\n> In order to do this, Git reruns the config parsing mechanism upon\n> noticing the first URL-conditional include in order to find all remote\n> URLs, and these remote URLs are then used to determine if that first and\n> all subsequent includes are executed. Remote URLs are not allowed to be\n> configued in any URL-conditionally-included file.\n>\n> [1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n>\n> Signed-off-by: Jonathan Tan \n> ---\n> Documentation/config.txt | 11 ++++\n> config.c | 121 ++++++++++++++++++++++++++++++++++++---\n> config.h | 7 +++\n> t/t1300-config.sh | 100 ++++++++++++++++++++++++++++++++\n> 4 files changed, 231 insertions(+), 8 deletions(-)\n\nHere is just a design level comment, without trying to outline the\nimplementation in my head like I usually do before making any\nsuggestion, but it strikes me somewhat sad that config.c needs to\nknow specifically about \"remote_url\".\n\nI wonder if this can be a more generalized framework that allows us\nto say \"we introduce a new [includeIf] variant to get another file\nincluded only if some condition is met for the configuration\nvariables we read without the includeIf directive\", with variations\nof \"condition\" including\n\n - a literal X is among the values of multi-valued variable Y.\n - a pattern X matches one of the values of multi-valued variable Y.\n - a literal Y is the name of an existing configuration variable.\n - a pattern Y matches the name of an existing configuration variable.\n\nIf that is done, I would imagine that the feature can become a thin\nspecialization e.g. \"there is an existing configuration variable\nwhose name is 'remotes.https://github.com/janathantanmy/git.url'\"\n\nPerhaps I am dreaming?\n\nThanks.\n\n\n\n"},{"id":"442082","messageId":"211123.86pmqrwtf2.gmgdl@evledraar.gmail.com","threadId":"56696","inReplyTo":"1c1a07a0b65d4bbbb0f2628a3ddf1980e37d5065.1637020610.git.jonathantanmy@google.com","subject":"Re: [PATCH v3 2/2] config: include file if remote URL matches a glob","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-11-23T01:27:33Z","receivedAt":"2021-11-23T01:29:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Nov 15 2021, Jonathan Tan wrote:\n\n> +`hasremoteurl`::\n> +\tThe data that follows the keyword `hasremoteurl:` is taken to\n\nBoth here..\n\n> +\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasremoteurl\"));\n\n..and here...\n\n> +\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &url,\n\n...but not here (C code)..\n\n> +\t * For internal use. Include all includeif.hasremoteurl paths without\n\n..but here..\n\n> +test_expect_success 'includeIf.hasremoteurl' '\n\n..and also here etc., let's consistently camelCase config keys whenever\nwe're not using them for lookups in the C\ncode.\n\nI.e. \"includeIf.hasRemoteUrl\" (possibly \"includeIf.hasRemoteURL\"?). It\nmakes them a lot easier to read, and makes the end-user documentation &\nmessaging more consistent.\n"},{"id":"442535","messageId":"20211129183348.384067-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"211123.86pmqrwtf2.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v3 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-29T18:33:48Z","receivedAt":"2021-11-29T22:30:22Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> On Mon, Nov 15 2021, Jonathan Tan wrote:\n> \n> > +`hasremoteurl`::\n> > +\tThe data that follows the keyword `hasremoteurl:` is taken to\n> \n> Both here..\n> \n> > +\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasremoteurl\"));\n> \n> ..and here...\n> \n> > +\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &url,\n> \n> ...but not here (C code)..\n> \n> > +\t * For internal use. Include all includeif.hasremoteurl paths without\n> \n> ..but here..\n> \n> > +test_expect_success 'includeIf.hasremoteurl' '\n> \n> ..and also here etc., let's consistently camelCase config keys whenever\n> we're not using them for lookups in the C\n> code.\n> \n> I.e. \"includeIf.hasRemoteUrl\" (possibly \"includeIf.hasRemoteURL\"?). It\n> makes them a lot easier to read, and makes the end-user documentation &\n> messaging more consistent.\n\nThe middle part is not case-insensitive, though - I tried changing it in\nthe test and the test now fails. (Unless you mean that we should also\nchange the code to make it case-insensitive - but I would think that\nit's better for the URL to be case-sensitive, and by extension, the\n\"hasremoteurl:\" part connected to it.)\n"},{"id":"442538","messageId":"b2dcae03ed3016f96a5368db925fa3736c3d7c58.1638217387.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1638217387.git.jonathantanmy@google.com","subject":"[PATCH v4 1/2] config: make git_config_include() static","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-29T20:23:41Z","receivedAt":"2021-11-29T22:33:38Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"It is not used from outside the file in which it is declared.\n\nSigned-off-by: Jonathan Tan \n---\n config.c | 12 +++++++++++-\n config.h | 37 ++++---------------------------------\n 2 files changed, 15 insertions(+), 34 deletions(-)\n\ndiff --git a/config.c b/config.c\nindex 2dcbe901b6..94ad5ce913 100644\n--- a/config.c\n+++ b/config.c\n@@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n \treturn conf->u.buf.pos;\n }\n \n+struct config_include_data {\n+\tint depth;\n+\tconfig_fn_t fn;\n+\tvoid *data;\n+\tconst struct config_options *opts;\n+};\n+#define CONFIG_INCLUDE_INIT { 0 }\n+\n+static int git_config_include(const char *var, const char *value, void *data);\n+\n #define MAX_INCLUDE_DEPTH 10\n static const char include_depth_advice[] = N_(\n \"exceeded maximum include depth (%d) while including\\n\"\n@@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n-int git_config_include(const char *var, const char *value, void *data)\n+static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n \tconst char *cond, *key;\ndiff --git a/config.h b/config.h\nindex f119de0130..48a5e472ca 100644\n--- a/config.h\n+++ b/config.h\n@@ -126,6 +126,8 @@ int git_default_config(const char *, const char *, void *);\n /**\n * Read a specific file in git-config format.\n * This function takes the same callback and data parameters as `git_config`.\n+ *\n+ * Unlike git_config(), this function does not respect includes.\n */\n int git_config_from_file(config_fn_t fn, const char *, void *);\n \n@@ -158,6 +160,8 @@ void read_very_early_config(config_fn_t cb, void *data);\n * will first feed the user-wide one to the callback, and then the\n * repo-specific one; by overwriting, the higher-priority repo-specific\n * value is left at the end).\n+ *\n+ * Unlike git_config_from_file(), this function respects includes.\n */\n void git_config(config_fn_t fn, void *);\n \n@@ -338,39 +342,6 @@ const char *current_config_origin_type(void);\n const char *current_config_name(void);\n int current_config_line(void);\n \n-/**\n- * Include Directives\n- * ------------------\n- *\n- * By default, the config parser does not respect include directives.\n- * However, a caller can use the special `git_config_include` wrapper\n- * callback to support them. To do so, you simply wrap your \"real\" callback\n- * function and data pointer in a `struct config_include_data`, and pass\n- * the wrapper to the regular config-reading functions. For example:\n- *\n- * -------------------------------------------\n- * int read_file_with_include(const char *file, config_fn_t fn, void *data)\n- * {\n- * struct config_include_data inc = CONFIG_INCLUDE_INIT;\n- * inc.fn = fn;\n- * inc.data = data;\n- * return git_config_from_file(git_config_include, file, &inc);\n- * }\n- * -------------------------------------------\n- *\n- * `git_config` respects includes automatically. The lower-level\n- * `git_config_from_file` does not.\n- *\n- */\n-struct config_include_data {\n-\tint depth;\n-\tconfig_fn_t fn;\n-\tvoid *data;\n-\tconst struct config_options *opts;\n-};\n-#define CONFIG_INCLUDE_INIT { 0 }\n-int git_config_include(const char *name, const char *value, void *data);\n-\n /*\n * Match and parse a config key of the form:\n *\n-- \n2.34.0.rc2.393.gf8c9666880-goog\n\n"},{"id":"442539","messageId":"211129.864k7ug02c.gmgdl@evledraar.gmail.com","threadId":"56696","inReplyTo":"cover.1638217387.git.jonathantanmy@google.com","subject":"Re: [PATCH v4 0/2] Conditional config includes based on remote URL","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-11-29T20:48:16Z","receivedAt":"2021-11-29T22:33:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Nov 29 2021, Jonathan Tan wrote:\n\n> Thanks everyone for your comments. Here's an update.\n\nJust from skimming this (minor) feedback on v3 still applies:\nhttps://lore.kernel.org/git/211123.86pmqrwtf2.gmgdl@evledraar.gmail.com/\n\nI.e. s/hasremoteurl/hasRemoteURL/ etc. in appropriate places.\n"},{"id":"442544","messageId":"20211129175300.371981-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"kl6lilwjre3m.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v3 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-29T17:53:00Z","receivedAt":"2021-11-29T22:44:57Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Glen Choo writes:\n> Jonathan Tan writes:\n> \n> > +`hasremoteurl`::\n> > +\tThe data that follows the keyword `hasremoteurl:` is taken to\n> > +\tbe a pattern with standard globbing wildcards and two\n> > +\tadditional ones, `**/` and `/**`, that can match multiple\n> > +\tcomponents. The rest of the config files will be scanned for\n> > +\tremote URLs, and then if there at least one remote URL that\n> \n> if there {is,exists}* at least one remote URL that\n\nAh, good catch.\n\n> > +\tmatches this pattern, the include condition is met.\n> > ++\n> > +Files included by this option (directly or indirectly) are not allowed\n> > +to contain remote URLs.\n> \n> As Jeff mentioned earlier in this thread, this \"last-config-wins\" is a\n> pretty big exception to the existing semantics, as\n> Documentation/config.txt reads:\n> \n> The contents of the included file are inserted immediately, as if they\n> had been found at the location of the include directive.\n> \n> At minimum, I think we should call out this exception in\n> Documentation/config.txt and the commit message, but calling out *just*\n> hasremoteurl makes this exception seem like a strange anomaly at first\n> glance, even though we actually have a good idea of when and why we are\n> doing this (which is that it simplifies includes that rely on config\n> values).\n\nI've switched it to expand-in-place semantics. The scanning for remote\nURLs does not mean that those configs are applied before the include.\nI'll add a note to the documentation about that, but if you can think of\na better way to explain that, that would be great.\n\nThe patch includes a test \"includeIf.hasremoteurl respects\nlast-config-wins\". Take a look and see if it matches your expected\nbehavior, and let me know if it could be clearer.\n\n> I was a big fan of your includeIfDeferred proposal, and I still think\n> that it's easier for users to understand if we explicitly require\n> \"includeIfDeferred\" instead of counting on them to remember when\n> \"includeIf\" behaves as it always did vs this new 'deferred' behavior.\n> That said, I doubt most users actually rely on the inclusion order, and \n> I am ok with this approach as long as we document the different\n> inclusion order.\n\nThe user still needs to know that config variables in the future can\naffect the behavior of the include, but perhaps that will be easier than\nremembering that certain configs are deferred.\n\n> It's not clear to me whether we are forbidding the remote urls correctly\n> when uncondition_remote_url is false. I would be convinced if we had\n> tests that convered this behavior, but I did not find any such test\n> cases.\n\n[snip]\n\n> As mentioned above, I would have expected to find test cases that test\n> whether or not we forbid the remote urls correctly, but the tests are\n> pretty clear.\n\nAh yes, I should include a test for this. I'll include it in the next\nreroll.\n"},{"id":"442545","messageId":"cover.1638217387.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"[PATCH v4 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-29T20:23:40Z","receivedAt":"2021-11-29T22:46:14Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Thanks everyone for your comments. Here's an update.\n\nJonathan Tan (2):\n config: make git_config_include() static\n config: include file if remote URL matches a glob\n\n Documentation/config.txt | 12 ++++\n config.c | 133 ++++++++++++++++++++++++++++++++++++---\n config.h | 44 ++++---------\n t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++\n 4 files changed, 265 insertions(+), 42 deletions(-)\n\nRange-diff against v3:\n1: b2dcae03ed = 1: b2dcae03ed config: make git_config_include() static\n2: 1c1a07a0b6 ! 2: 3b3af0da98 config: include file if remote URL matches a glob\n @@ Documentation/config.txt: all branches that begin with `foo/`. This is useful if\n +\tThe data that follows the keyword `hasremoteurl:` is taken to\n +\tbe a pattern with standard globbing wildcards and two\n +\tadditional ones, `**/` and `/**`, that can match multiple\n -+\tcomponents. The rest of the config files will be scanned for\n -+\tremote URLs, and then if there at least one remote URL that\n -+\tmatches this pattern, the include condition is met.\n ++\tcomponents. The first time this keyword is seen, the rest of\n ++\tthe config files will be scanned for remote URLs (without\n ++\tapplying any values). If there exists at least one remote URL\n ++\tthat matches this pattern, the include condition is met.\n ++\n +Files included by this option (directly or indirectly) are not allowed\n +to contain remote URLs.\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\tgit -C hasremoteurlTest config --get user.ssm &&\n +\ttest_must_fail git -C hasremoteurlTest config --get user.no\n +'\n ++\n ++test_expect_success 'includeIf.hasremoteurl forbids remote url in such included files' '\n ++\tgit init hasremoteurlTest &&\n ++\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n ++\n ++\tcat >\"$(pwd)\"/include-with-url <<-\\EOF &&\n ++\t[remote \"bar\"]\n ++\t\turl = bar\n ++\tEOF\n ++\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n ++\t[includeIf \"hasremoteurl:foo\"]\n ++\t\tpath = \"$(pwd)/include-with-url\"\n ++\tEOF\n ++\n ++\t# test with any Git command\n ++\ttest_must_fail git -C hasremoteurlTest status 2>err &&\n ++\tgrep \"fatal: remote URLs cannot be configured in file directly or indirectly included by includeIf.hasremoteurl\" err\n ++'\n +\n test_done\n-- \n2.34.0.rc2.393.gf8c9666880-goog\n\n"},{"id":"442549","messageId":"211129.86zgpmelcx.gmgdl@evledraar.gmail.com","threadId":"56696","inReplyTo":"20211129183348.384067-1-jonathantanmy@google.com","subject":"Re: [PATCH v3 2/2] config: include file if remote URL matches a glob","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-11-29T20:50:11Z","receivedAt":"2021-11-29T22:51:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Nov 29 2021, Jonathan Tan wrote:\n\n>> On Mon, Nov 15 2021, Jonathan Tan wrote:\n>> \n>> > +`hasremoteurl`::\n>> > +\tThe data that follows the keyword `hasremoteurl:` is taken to\n>> \n>> Both here..\n>> \n>> > +\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasremoteurl\"));\n>> \n>> ..and here...\n>> \n>> > +\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &url,\n>> \n>> ...but not here (C code)..\n>> \n>> > +\t * For internal use. Include all includeif.hasremoteurl paths without\n>> \n>> ..but here..\n>> \n>> > +test_expect_success 'includeIf.hasremoteurl' '\n>> \n>> ..and also here etc., let's consistently camelCase config keys whenever\n>> we're not using them for lookups in the C\n>> code.\n>> \n>> I.e. \"includeIf.hasRemoteUrl\" (possibly \"includeIf.hasRemoteURL\"?). It\n>> makes them a lot easier to read, and makes the end-user documentation &\n>> messaging more consistent.\n>\n> The middle part is not case-insensitive, though - I tried changing it in\n> the test and the test now fails. (Unless you mean that we should also\n> change the code to make it case-insensitive - but I would think that\n> it's better for the URL to be case-sensitive, and by extension, the\n> \"hasremoteurl:\" part connected to it.)\n\nAh, I forgot about that edge case. sorry. And sent [1] without having\nseen this as a reminder on v4. Makes sense.\n\n(I seem to be getting really slow delivery from kernel.org to GMail\nthese days, sometimes I can see things on lore.kernel.org hours or half\na day before it pops up in my mail...)\n\n1. https://lore.kernel.org/git/211129.864k7ug02c.gmgdl@evledraar.gmail.com/\n"},{"id":"442550","messageId":"3b3af0da983c65ac8282b14141495f6859cd575a.1638217387.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1638217387.git.jonathantanmy@google.com","subject":"[PATCH v4 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-29T20:23:42Z","receivedAt":"2021-11-29T22:53:25Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This is a feature that supports config file inclusion conditional on\nwhether the repo has a remote with a URL that matches a glob.\n\nSimilar to my previous work on remote-suggested hooks [1], the main\nmotivation is to allow remote repo administrators to provide recommended\nconfigs in a way that can be consumed more easily (e.g. through a\npackage installable by a package manager - it could, for example,\ncontain a file to be included conditionally and a post-install script\nthat adds the include directive to the system-wide config file).\n\nIn order to do this, Git reruns the config parsing mechanism upon\nnoticing the first URL-conditional include in order to find all remote\nURLs, and these remote URLs are then used to determine if that first and\nall subsequent includes are executed. Remote URLs are not allowed to be\nconfigued in any URL-conditionally-included file.\n\n[1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n\nSigned-off-by: Jonathan Tan \n---\n Documentation/config.txt | 12 ++++\n config.c | 121 ++++++++++++++++++++++++++++++++++++---\n config.h | 7 +++\n t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++++++\n 4 files changed, 250 insertions(+), 8 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 0c0e6b859f..bfc9e22d78 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -159,6 +159,18 @@ all branches that begin with `foo/`. This is useful if your branches are\n organized hierarchically and you would like to apply a configuration to\n all the branches in that hierarchy.\n \n+`hasremoteurl`::\n+\tThe data that follows the keyword `hasremoteurl:` is taken to\n+\tbe a pattern with standard globbing wildcards and two\n+\tadditional ones, `**/` and `/**`, that can match multiple\n+\tcomponents. The first time this keyword is seen, the rest of\n+\tthe config files will be scanned for remote URLs (without\n+\tapplying any values). If there exists at least one remote URL\n+\tthat matches this pattern, the include condition is met.\n++\n+Files included by this option (directly or indirectly) are not allowed\n+to contain remote URLs.\n+\n A few more notes on matching via `gitdir` and `gitdir/i`:\n \n * Symlinks in `$GIT_DIR` are not resolved before matching.\ndiff --git a/config.c b/config.c\nindex 94ad5ce913..4ffc1e87e9 100644\n--- a/config.c\n+++ b/config.c\n@@ -125,6 +125,12 @@ struct config_include_data {\n \tconfig_fn_t fn;\n \tvoid *data;\n \tconst struct config_options *opts;\n+\tstruct git_config_source *config_source;\n+\n+\t/*\n+\t * All remote URLs discovered when reading all config files.\n+\t */\n+\tstruct string_list *remote_urls;\n };\n #define CONFIG_INCLUDE_INIT { 0 }\n \n@@ -316,12 +322,83 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n+static int add_remote_url(const char *var, const char *value, void *data)\n+{\n+\tstruct string_list *remote_urls = data;\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tstring_list_append(remote_urls, value);\n+\treturn 0;\n+}\n+\n+static void populate_remote_urls(struct config_include_data *inc)\n+{\n+\tstruct config_options opts;\n+\n+\tstruct config_source *store_cf = cf;\n+\tstruct key_value_info *store_kvi = current_config_kvi;\n+\tenum config_scope store_scope = current_parsing_scope;\n+\n+\topts = *inc->opts;\n+\topts.unconditional_remote_url = 1;\n+\n+\tcf = NULL;\n+\tcurrent_config_kvi = NULL;\n+\tcurrent_parsing_scope = 0;\n+\n+\tinc->remote_urls = xmalloc(sizeof(*inc->remote_urls));\n+\tstring_list_init_dup(inc->remote_urls);\n+\tconfig_with_options(add_remote_url, inc->remote_urls, inc->config_source, &opts);\n+\n+\tcf = store_cf;\n+\tcurrent_config_kvi = store_kvi;\n+\tcurrent_parsing_scope = store_scope;\n+}\n+\n+static int forbid_remote_url(const char *var, const char *value, void *data)\n+{\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasremoteurl\"));\n+\treturn 0;\n+}\n+\n+static int at_least_one_url_matches_glob(const char *glob, int glob_len,\n+\t\t\t\t\t struct string_list *remote_urls)\n+{\n+\tstruct strbuf pattern = STRBUF_INIT;\n+\tstruct string_list_item *url_item;\n+\tint found = 0;\n+\n+\tstrbuf_add(&pattern, glob, glob_len);\n+\tfor_each_string_list_item(url_item, remote_urls) {\n+\t\tif (!wildmatch(pattern.buf, url_item->string, WM_PATHNAME)) {\n+\t\t\tfound = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tstrbuf_release(&pattern);\n+\treturn found;\n+}\n+\n static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n \tconst char *cond, *key;\n \tsize_t cond_len;\n-\tint ret;\n+\tint ret = 0;\n \n \t/*\n \t * Pass along all values, including \"include\" directives; this makes it\n@@ -335,9 +412,29 @@ static int git_config_include(const char *var, const char *value, void *data)\n \t\tret = handle_path_include(value, inc);\n \n \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n-\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n-\t !strcmp(key, \"path\"))\n-\t\tret = handle_path_include(value, inc);\n+\t cond && !strcmp(key, \"path\")) {\n+\t\tconst char *url;\n+\t\tsize_t url_len;\n+\n+\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &url,\n+\t\t\t\t &url_len)) {\n+\t\t\tif (inc->opts->unconditional_remote_url) {\n+\t\t\t\tconfig_fn_t old_fn = inc->fn;\n+\n+\t\t\t\tinc->fn = forbid_remote_url;\n+\t\t\t\tret = handle_path_include(value, inc);\n+\t\t\t\tinc->fn = old_fn;\n+\t\t\t} else {\n+\t\t\t\tif (!inc->remote_urls)\n+\t\t\t\t\tpopulate_remote_urls(inc);\n+\t\t\t\tif (at_least_one_url_matches_glob(\n+\t\t\t\t\t\turl, url_len, inc->remote_urls))\n+\t\t\t\t\tret = handle_path_include(value, inc);\n+\t\t\t}\n+\t\t} else if (include_condition_is_true(inc->opts, cond, cond_len)) {\n+\t\t\tret = handle_path_include(value, inc);\n+\t\t}\n+\t}\n \n \treturn ret;\n }\n@@ -1933,11 +2030,13 @@ int config_with_options(config_fn_t fn, void *data,\n \t\t\tconst struct config_options *opts)\n {\n \tstruct config_include_data inc = CONFIG_INCLUDE_INIT;\n+\tint ret;\n \n \tif (opts->respect_includes) {\n \t\tinc.fn = fn;\n \t\tinc.data = data;\n \t\tinc.opts = opts;\n+\t\tinc.config_source = config_source;\n \t\tfn = git_config_include;\n \t\tdata = &inc;\n \t}\n@@ -1950,17 +2049,23 @@ int config_with_options(config_fn_t fn, void *data,\n \t * regular lookup sequence.\n \t */\n \tif (config_source && config_source->use_stdin) {\n-\t\treturn git_config_from_stdin(fn, data);\n+\t\tret = git_config_from_stdin(fn, data);\n \t} else if (config_source && config_source->file) {\n-\t\treturn git_config_from_file(fn, config_source->file, data);\n+\t\tret = git_config_from_file(fn, config_source->file, data);\n \t} else if (config_source && config_source->blob) {\n \t\tstruct repository *repo = config_source->repo ?\n \t\t\tconfig_source->repo : the_repository;\n-\t\treturn git_config_from_blob_ref(fn, repo, config_source->blob,\n+\t\tret = git_config_from_blob_ref(fn, repo, config_source->blob,\n \t\t\t\t\t\tdata);\n+\t} else {\n+\t\tret = do_git_config_sequence(opts, fn, data);\n \t}\n \n-\treturn do_git_config_sequence(opts, fn, data);\n+\tif (inc.remote_urls) {\n+\t\tstring_list_clear(inc.remote_urls, 0);\n+\t\tFREE_AND_NULL(inc.remote_urls);\n+\t}\n+\treturn ret;\n }\n \n static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)\ndiff --git a/config.h b/config.h\nindex 48a5e472ca..c24458b10a 100644\n--- a/config.h\n+++ b/config.h\n@@ -89,6 +89,13 @@ struct config_options {\n \tunsigned int ignore_worktree : 1;\n \tunsigned int ignore_cmdline : 1;\n \tunsigned int system_gently : 1;\n+\n+\t/*\n+\t * For internal use. Include all includeif.hasremoteurl paths without\n+\t * checking if the repo has that remote URL.\n+\t */\n+\tunsigned int unconditional_remote_url : 1;\n+\n \tconst char *commondir;\n \tconst char *git_dir;\n \tconfig_parser_event_fn_t event_fn;\ndiff --git a/t/t1300-config.sh b/t/t1300-config.sh\nindex 9ff46f3b04..9bd299d3f8 100755\n--- a/t/t1300-config.sh\n+++ b/t/t1300-config.sh\n@@ -2387,4 +2387,122 @@ test_expect_success '--get and --get-all with --fixed-value' '\n \ttest_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n '\n \n+test_expect_success 'includeIf.hasremoteurl' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >\"$(pwd)\"/include-this <<-\\EOF &&\n+\t[user]\n+\t\tthis = this-is-included\n+\tEOF\n+\tcat >\"$(pwd)\"/dont-include-that <<-\\EOF &&\n+\t[user]\n+\t\tthat = that-is-not-included\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasremoteurl:foo\"]\n+\t\tpath = \"$(pwd)/include-this\"\n+\t[includeIf \"hasremoteurl:bar\"]\n+\t\tpath = \"$(pwd)/dont-include-that\"\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\tEOF\n+\n+\techo this-is-included >expect-this &&\n+\tgit -C hasremoteurlTest config --get user.this >actual-this &&\n+\ttest_cmp expect-this actual-this &&\n+\n+\ttest_must_fail git -C hasremoteurlTest config --get user.that\n+'\n+\n+test_expect_success 'includeIf.hasremoteurl respects last-config-wins' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >\"$(pwd)\"/include-two-three <<-\\EOF &&\n+\t[user]\n+\t\ttwo = included-config\n+\t\tthree = included-config\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\t[user]\n+\t\tone = main-config\n+\t\ttwo = main-config\n+\t[includeIf \"hasremoteurl:foo\"]\n+\t\tpath = \"$(pwd)/include-two-three\"\n+\t[user]\n+\t\tthree = main-config\n+\tEOF\n+\n+\techo main-config >expect-main-config &&\n+\techo included-config >expect-included-config &&\n+\n+\tgit -C hasremoteurlTest config --get user.one >actual &&\n+\ttest_cmp expect-main-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.two >actual &&\n+\ttest_cmp expect-included-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.three >actual &&\n+\ttest_cmp expect-main-config actual\n+'\n+\n+test_expect_success 'includeIf.hasremoteurl globs' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tprintf \"[user]\\ndss = yes\\n\" >\"$(pwd)/double-star-start\" &&\n+\tprintf \"[user]\\ndse = yes\\n\" >\"$(pwd)/double-star-end\" &&\n+\tprintf \"[user]\\ndsm = yes\\n\" >\"$(pwd)/double-star-middle\" &&\n+\tprintf \"[user]\\nssm = yes\\n\" >\"$(pwd)/single-star-middle\" &&\n+\tprintf \"[user]\\nno = no\\n\" >\"$(pwd)/no\" &&\n+\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = https://foo/bar/baz\n+\t[includeIf \"hasremoteurl:**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-start\"\n+\t[includeIf \"hasremoteurl:**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasremoteurl:https:/**\"]\n+\t\tpath = \"$(pwd)/double-star-end\"\n+\t[includeIf \"hasremoteurl:nomatch:/**\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasremoteurl:https:/**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-middle\"\n+\t[includeIf \"hasremoteurl:https:/**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasremoteurl:https://*/bar/baz\"]\n+\t\tpath = \"$(pwd)/single-star-middle\"\n+\t[includeIf \"hasremoteurl:https://*/baz\"]\n+\t\tpath = \"$(pwd)/no\"\n+\tEOF\n+\n+\tgit -C hasremoteurlTest config --get user.dss &&\n+\tgit -C hasremoteurlTest config --get user.dse &&\n+\tgit -C hasremoteurlTest config --get user.dsm &&\n+\tgit -C hasremoteurlTest config --get user.ssm &&\n+\ttest_must_fail git -C hasremoteurlTest config --get user.no\n+'\n+\n+test_expect_success 'includeIf.hasremoteurl forbids remote url in such included files' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >\"$(pwd)\"/include-with-url <<-\\EOF &&\n+\t[remote \"bar\"]\n+\t\turl = bar\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasremoteurl:foo\"]\n+\t\tpath = \"$(pwd)/include-with-url\"\n+\tEOF\n+\n+\t# test with any Git command\n+\ttest_must_fail git -C hasremoteurlTest status 2>err &&\n+\tgrep \"fatal: remote URLs cannot be configured in file directly or indirectly included by includeIf.hasremoteurl\" err\n+'\n+\n test_done\n-- \n2.34.0.rc2.393.gf8c9666880-goog\n\n"},{"id":"442557","messageId":"20211129181811.375318-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"xmqqtug3aco7.fsf@gitster.g","subject":"Re: [PATCH v3 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-29T18:18:11Z","receivedAt":"2021-11-29T23:04:43Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Junio C Hamano writes:\n> Here is just a design level comment, without trying to outline the\n> implementation in my head like I usually do before making any\n> suggestion, but it strikes me somewhat sad that config.c needs to\n> know specifically about \"remote_url\".\n> \n> I wonder if this can be a more generalized framework that allows us\n> to say \"we introduce a new [includeIf] variant to get another file\n> included only if some condition is met for the configuration\n> variables we read without the includeIf directive\", with variations\n> of \"condition\" including\n> \n> - a literal X is among the values of multi-valued variable Y.\n> - a pattern X matches one of the values of multi-valued variable Y.\n> - a literal Y is the name of an existing configuration variable.\n> - a pattern Y matches the name of an existing configuration variable.\n> \n> If that is done, I would imagine that the feature can become a thin\n> specialization e.g. \"there is an existing configuration variable\n> whose name is 'remotes.https://github.com/janathantanmy/git.url'\"\n> \n> Perhaps I am dreaming?\n> \n> Thanks.\n\nI think that the hard part of this is how to present this to the end\nuser - right now, we just have one pattern of variable (remote.*.url,\nwhere \"*\" is a wildcard) and one pattern of value with specific\nproperties (e.g. this is a glob, not a regular expression, and the\nspecial value \"**\" is supported).\n\nOnce we figure that out, I would think that we could implement it in a\nway similar to this patch. As for whether we should wait until the full\nfeature before merging any code that does includeIf based on a variable\n(in order to avoid having code that would quickly be replaced by other\ncode), in this case, unless there is another use case for this, I think\nwe should proceed with the use case that we know about first\n(conditional include of a file supplied by a remote repo administrator).\n"},{"id":"442616","messageId":"xmqq4k7u2i9f.fsf@gitster.g","threadId":"56696","inReplyTo":"211129.864k7ug02c.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v4 0/2] Conditional config includes based on remote URL","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-30T07:51:40Z","receivedAt":"2021-11-30T07:51:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Mon, Nov 29 2021, Jonathan Tan wrote:\n>\n>> Thanks everyone for your comments. Here's an update.\n>\n> Just from skimming this (minor) feedback on v3 still applies:\n> https://lore.kernel.org/git/211123.86pmqrwtf2.gmgdl@evledraar.gmail.com/\n>\n> I.e. s/hasremoteurl/hasRemoteURL/ etc. in appropriate places.\n\nIs there any appropriate place, though?\n\n\"hasremoteurl\" is a new directive to be used as the leading part of\n in the name of `includeIf..path` variable.\nThe part is case sensitive, and we do not want people to\nspell it, and the existing \"gitdir\", \"gitdir/i\", and \"onbranch\", in\nmixed cases.\n\nSee config.c::include_condition_is_true() function and its use of\nskip_prefix_mem() to locate these existing conditions.\n\nIt is troubling that this patch is *NOT* extend the implementation\nof include_condition_is_true() function (which gives a very clean\nabstraction and makes the caller very readable); it instead mucks\nwith the caller of include_condition_is_true() and adds a parallel\nlogic that include_condition_is_true() does not know about. It may\nhave been an expedite way to implement this, and the result may not\nseem to hurt when include_condition_is_true() is called by only one\ncaller, but I find the resulting code structure unnecessarily ugly.\n\nCan't the body of if (skip_prefix_mem(...\"hasremoteurl:\", ...)) block\nbecome include_by_remoteurl() function, similar to include_by_foo()\nfunctions include_condition_is_true() already calls?\n"},{"id":"442786","messageId":"xmqqlf1417mo.fsf@gitster.g","threadId":"56696","inReplyTo":"20211129181811.375318-1-jonathantanmy@google.com","subject":"Re: [PATCH v3 2/2] config: include file if remote URL matches a glob","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-01T18:51:11Z","receivedAt":"2021-12-01T18:51:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n>> variables we read without the includeIf directive\", with variations\n>> of \"condition\" including\n>> \n>> - a literal X is among the values of multi-valued variable Y.\n>> - a pattern X matches one of the values of multi-valued variable Y.\n>> - a literal Y is the name of an existing configuration variable.\n>> - a pattern Y matches the name of an existing configuration variable.\n> ...\n> code), in this case, unless there is another use case for this, I think\n> we should proceed with the use case that we know about first\n> (conditional include of a file supplied by a remote repo administrator).\n\nDoing it that way without thinking flexibility through will paint us\ninto a corner, from which we cannot get out of, doesn't it?\n\nPeople will start asking \"Why should we even have\n'hasremoteurl:$URL' variant in 'includeIf' conditions, when one of\nthe 'variableExists:Y' and friends can express the same thing\",\nsomebody new who is not yet in this community today will propose\ndeprecating hasremoteurl in favor of more generalized approach and\nwe have to give a sad answer \"no, we earlier made a mistake of\nstarting with a special case variant for expediency's sake, without\nthinking the general cases through. Because existing users depend\non it, we have to support it til the end of time.\"\n\nWe have the same regret with \"why do we need grep.extendedRegexp\nwhen grep.patternType suffices?\" I am reluctant to see us knowingly\ncommit the same mistake here, unless there is a very good reason.\n"},{"id":"442854","messageId":"xmqqpmqfwl32.fsf@gitster.g","threadId":"56696","inReplyTo":"3b3af0da983c65ac8282b14141495f6859cd575a.1638217387.git.jonathantanmy@google.com","subject":"Re: [PATCH v4 2/2] config: include file if remote URL matches a glob","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-02T06:57:05Z","receivedAt":"2021-12-02T06:57:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> +static int add_remote_url(const char *var, const char *value, void *data)\n> +{\n> +...\n> +}\n> +\n> +static void populate_remote_urls(struct config_include_data *inc)\n> +{\n> +...\n> +}\n> +\n> +static int forbid_remote_url(const char *var, const char *value, void *data)\n> +{\n> +...\n> +}\n> +\n> +static int at_least_one_url_matches_glob(const char *glob, int glob_len,\n> +\t\t\t\t\t struct string_list *remote_urls)\n> +{\n> +...\n> +}\n\nAll of the above makes sense; you prepare the remote URLs defined in\na string list, and have these helper functions that can determine if\nthe value given to hasremoteurl:* condition satisfies the condition.\n\n> static int git_config_include(const char *var, const char *value, void *data)\n> {\n> \tstruct config_include_data *inc = data;\n> \tconst char *cond, *key;\n> \tsize_t cond_len;\n> -\tint ret;\n> +\tint ret = 0;\n> \n> \t/*\n> \t * Pass along all values, including \"include\" directives; this makes it\n> @@ -335,9 +412,29 @@ static int git_config_include(const char *var, const char *value, void *data)\n> \t\tret = handle_path_include(value, inc);\n> \n> \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n> -\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n> -\t !strcmp(key, \"path\"))\n> -\t\tret = handle_path_include(value, inc);\n> +\t cond && !strcmp(key, \"path\")) {\n> +\t\tconst char *url;\n> +\t\tsize_t url_len;\n> +\n> +\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &url,\n> +\t\t\t\t &url_len)) {\n> +\t\t\tif (inc->opts->unconditional_remote_url) {\n> +\t\t\t\tconfig_fn_t old_fn = inc->fn;\n> +\n> +\t\t\t\tinc->fn = forbid_remote_url;\n> +\t\t\t\tret = handle_path_include(value, inc);\n> +\t\t\t\tinc->fn = old_fn;\n> +\t\t\t} else {\n> +\t\t\t\tif (!inc->remote_urls)\n> +\t\t\t\t\tpopulate_remote_urls(inc);\n> +\t\t\t\tif (at_least_one_url_matches_glob(\n> +\t\t\t\t\t\turl, url_len, inc->remote_urls))\n> +\t\t\t\t\tret = handle_path_include(value, inc);\n> +\t\t\t}\n> +\t\t} else if (include_condition_is_true(inc->opts, cond, cond_len)) {\n> +\t\t\tret = handle_path_include(value, inc);\n> +\t\t}\n\nThis looks iffy, especialy in a patch that is not marked as [RFC].\n\nI can see that include_condition_is_true() only passes inc->opts and\nyou need some other parts of inc for your purpose, and it may be the\nprimary reason why you munge this caller instead of adding function\ninclude_by_remoteurl() and making include_condition_is_true() call\nit. But wouldn't it be sufficient to pass inc (not inc->opts) to\ninclude_condition_is_true(), and have it dereference inc->opts when\ncalling include_by_gitdir() and friends that want opts, while\npassing inc to include_by_remoteurl()?\n"},{"id":"442895","messageId":"20211202174119.1376799-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"xmqqpmqfwl32.fsf@gitster.g","subject":"Re: [PATCH v4 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-02T17:41:18Z","receivedAt":"2021-12-02T17:41:37Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Junio C Hamano writes:\n> > @@ -335,9 +412,29 @@ static int git_config_include(const char *var, const char *value, void *data)\n> > \t\tret = handle_path_include(value, inc);\n> > \n> > \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n> > -\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n> > -\t !strcmp(key, \"path\"))\n> > -\t\tret = handle_path_include(value, inc);\n> > +\t cond && !strcmp(key, \"path\")) {\n> > +\t\tconst char *url;\n> > +\t\tsize_t url_len;\n> > +\n> > +\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &url,\n> > +\t\t\t\t &url_len)) {\n> > +\t\t\tif (inc->opts->unconditional_remote_url) {\n> > +\t\t\t\tconfig_fn_t old_fn = inc->fn;\n> > +\n> > +\t\t\t\tinc->fn = forbid_remote_url;\n> > +\t\t\t\tret = handle_path_include(value, inc);\n> > +\t\t\t\tinc->fn = old_fn;\n> > +\t\t\t} else {\n> > +\t\t\t\tif (!inc->remote_urls)\n> > +\t\t\t\t\tpopulate_remote_urls(inc);\n> > +\t\t\t\tif (at_least_one_url_matches_glob(\n> > +\t\t\t\t\t\turl, url_len, inc->remote_urls))\n> > +\t\t\t\t\tret = handle_path_include(value, inc);\n> > +\t\t\t}\n> > +\t\t} else if (include_condition_is_true(inc->opts, cond, cond_len)) {\n> > +\t\t\tret = handle_path_include(value, inc);\n> > +\t\t}\n> \n> This looks iffy, especialy in a patch that is not marked as [RFC].\n> \n> I can see that include_condition_is_true() only passes inc->opts and\n> you need some other parts of inc for your purpose, and it may be the\n> primary reason why you munge this caller instead of adding function\n> include_by_remoteurl() and making include_condition_is_true() call\n> it. But wouldn't it be sufficient to pass inc (not inc->opts) to\n> include_condition_is_true(), and have it dereference inc->opts when\n> calling include_by_gitdir() and friends that want opts, while\n> passing inc to include_by_remoteurl()?\n\nThanks for taking a look. I think the primary reason why I wrote it like\nthis was because originally (in v1) I included at the end, not inline,\nbut of course that is no longer the case so I should revisit it. I'll\nhave to pay attention to how inc->fn is swapped for forbid_remote_url\n(to prevent remote URLs from being configured in included files), but\nperhaps handle_path_include() can do that. I'll take a look.\n"},{"id":"442918","messageId":"20211202231403.3451471-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"xmqqlf1417mo.fsf@gitster.g","subject":"Re: [PATCH v3 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-02T23:14:03Z","receivedAt":"2021-12-02T23:14:10Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Junio C Hamano writes:\n> Jonathan Tan writes:\n> \n> >> variables we read without the includeIf directive\", with variations\n> >> of \"condition\" including\n> >> \n> >> - a literal X is among the values of multi-valued variable Y.\n> >> - a pattern X matches one of the values of multi-valued variable Y.\n> >> - a literal Y is the name of an existing configuration variable.\n> >> - a pattern Y matches the name of an existing configuration variable.\n> > ...\n> > code), in this case, unless there is another use case for this, I think\n> > we should proceed with the use case that we know about first\n> > (conditional include of a file supplied by a remote repo administrator).\n> \n> Doing it that way without thinking flexibility through will paint us\n> into a corner, from which we cannot get out of, doesn't it?\n> \n> People will start asking \"Why should we even have\n> 'hasremoteurl:$URL' variant in 'includeIf' conditions, when one of\n> the 'variableExists:Y' and friends can express the same thing\",\n> somebody new who is not yet in this community today will propose\n> deprecating hasremoteurl in favor of more generalized approach and\n> we have to give a sad answer \"no, we earlier made a mistake of\n> starting with a special case variant for expediency's sake, without\n> thinking the general cases through. Because existing users depend\n> on it, we have to support it til the end of time.\"\n> \n> We have the same regret with \"why do we need grep.extendedRegexp\n> when grep.patternType suffices?\" I am reluctant to see us knowingly\n> commit the same mistake here, unless there is a very good reason.\n\nHmm...that's true. I was thinking that there wouldn't be a way to\npredict exactly what we'll need, but perhaps making the config variable\nname be of the form 'includeIf.\"hasconfig:remote.*.url\".path' might give\nus enough flexibility for the future.. I'll take a look.\n"},{"id":"442920","messageId":"cover.1638487815.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"[PATCH v5 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-02T23:31:36Z","receivedAt":"2021-12-02T23:31:43Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Thanks, Junio, for your comments. I think the code is more clearly laid \nout now. \n \nThe main changes from v4 are that I've maintained the existing code\nstructure more, and changed the keyword used to something that hopefully\nwill be more forwards compatible. I've also updated the documentation to\nexplain the forwards compatibility idea.\n\nJonathan Tan (2):\n config: make git_config_include() static\n config: include file if remote URL matches a glob\n\n Documentation/config.txt | 16 +++++\n config.c | 134 +++++++++++++++++++++++++++++++++++----\n config.h | 46 ++++----------\n t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++\n 4 files changed, 270 insertions(+), 44 deletions(-)\n\nRange-diff against v4:\n1: b2dcae03ed = 1: b2dcae03ed config: make git_config_include() static\n2: 3b3af0da98 ! 2: d3b8e00717 config: include file if remote URL matches a glob\n @@ Documentation/config.txt: all branches that begin with `foo/`. This is useful if\n organized hierarchically and you would like to apply a configuration to\n all the branches in that hierarchy.\n \n -+`hasremoteurl`::\n -+\tThe data that follows the keyword `hasremoteurl:` is taken to\n ++`hasconfig:remote.*.url:`::\n ++\tThe data that follows this keyword is taken to\n +\tbe a pattern with standard globbing wildcards and two\n +\tadditional ones, `**/` and `/**`, that can match multiple\n +\tcomponents. The first time this keyword is seen, the rest of\n @@ Documentation/config.txt: all branches that begin with `foo/`. This is useful if\n ++\n +Files included by this option (directly or indirectly) are not allowed\n +to contain remote URLs.\n +++\n ++This keyword is designed to be forwards compatible with a naming\n ++scheme that supports more variable-based include conditions, but\n ++currently Git only supports the exact keyword described above.\n +\n A few more notes on matching via `gitdir` and `gitdir/i`:\n \n @@ config.c: struct config_include_data {\n };\n #define CONFIG_INCLUDE_INIT { 0 }\n \n -@@ config.c: static int include_condition_is_true(const struct config_options *opts,\n - \treturn 0;\n +@@ config.c: static int include_by_branch(const char *cond, size_t cond_len)\n + \treturn ret;\n }\n \n +-static int include_condition_is_true(const struct config_options *opts,\n +static int add_remote_url(const char *var, const char *value, void *data)\n +{\n +\tstruct string_list *remote_urls = data;\n @@ config.c: static int include_condition_is_true(const struct config_options *opts\n +\t\t\t &key) &&\n +\t remote_name &&\n +\t !strcmp(key, \"url\"))\n -+\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasremoteurl\"));\n ++\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\"));\n +\treturn 0;\n +}\n +\n @@ config.c: static int include_condition_is_true(const struct config_options *opts\n +\treturn found;\n +}\n +\n - static int git_config_include(const char *var, const char *value, void *data)\n ++static int include_condition_is_true(struct config_include_data *inc,\n + \t\t\t\t const char *cond, size_t cond_len)\n {\n - \tstruct config_include_data *inc = data;\n - \tconst char *cond, *key;\n - \tsize_t cond_len;\n --\tint ret;\n -+\tint ret = 0;\n ++\tconst struct config_options *opts = inc->opts;\n + \n +-\tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len))\n ++\tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len)) {\n + \t\treturn include_by_gitdir(opts, cond, cond_len, 0);\n +-\telse if (skip_prefix_mem(cond, cond_len, \"gitdir/i:\", &cond, &cond_len))\n ++\t} else if (skip_prefix_mem(cond, cond_len, \"gitdir/i:\", &cond, &cond_len)) {\n + \t\treturn include_by_gitdir(opts, cond, cond_len, 1);\n +-\telse if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len))\n ++\t} else if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len)) {\n + \t\treturn include_by_branch(cond, cond_len);\n ++\t} else if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n ++\t\t\t\t &cond_len)) {\n ++\t\tif (inc->opts->unconditional_remote_url)\n ++\t\t\treturn 1;\n ++\t\tif (!inc->remote_urls)\n ++\t\t\tpopulate_remote_urls(inc);\n ++\t\treturn at_least_one_url_matches_glob(cond, cond_len,\n ++\t\t\t\t\t\t inc->remote_urls);\n ++\t}\n \n - \t/*\n - \t * Pass along all values, including \"include\" directives; this makes it\n + \t/* unknown conditionals are always false */\n + \treturn 0;\n @@ config.c: static int git_config_include(const char *var, const char *value, void *data)\n \t\tret = handle_path_include(value, inc);\n \n \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n -\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n -\t !strcmp(key, \"path\"))\n --\t\tret = handle_path_include(value, inc);\n -+\t cond && !strcmp(key, \"path\")) {\n -+\t\tconst char *url;\n -+\t\tsize_t url_len;\n -+\n -+\t\tif (skip_prefix_mem(cond, cond_len, \"hasremoteurl:\", &url,\n -+\t\t\t\t &url_len)) {\n -+\t\t\tif (inc->opts->unconditional_remote_url) {\n -+\t\t\t\tconfig_fn_t old_fn = inc->fn;\n -+\n -+\t\t\t\tinc->fn = forbid_remote_url;\n -+\t\t\t\tret = handle_path_include(value, inc);\n -+\t\t\t\tinc->fn = old_fn;\n -+\t\t\t} else {\n -+\t\t\t\tif (!inc->remote_urls)\n -+\t\t\t\t\tpopulate_remote_urls(inc);\n -+\t\t\t\tif (at_least_one_url_matches_glob(\n -+\t\t\t\t\t\turl, url_len, inc->remote_urls))\n -+\t\t\t\t\tret = handle_path_include(value, inc);\n -+\t\t\t}\n -+\t\t} else if (include_condition_is_true(inc->opts, cond, cond_len)) {\n -+\t\t\tret = handle_path_include(value, inc);\n -+\t\t}\n ++\t cond && include_condition_is_true(inc, cond, cond_len) &&\n ++\t !strcmp(key, \"path\")) {\n ++\t\tconfig_fn_t old_fn = inc->fn;\n ++\n ++\t\tif (inc->opts->unconditional_remote_url)\n ++\t\t\tinc->fn = forbid_remote_url;\n + \t\tret = handle_path_include(value, inc);\n ++\t\tif (inc->opts->unconditional_remote_url)\n ++\t\t\tinc->fn = old_fn;\n +\t}\n \n \treturn ret;\n @@ config.h: struct config_options {\n +\n +\t/*\n +\t * For internal use. Include all includeif.hasremoteurl paths without\n -+\t * checking if the repo has that remote URL.\n ++\t * checking if the repo has that remote URL, and when doing so, verify\n ++\t * that files included in this way do not configure any remote URLs\n ++\t * themselves.\n +\t */\n +\tunsigned int unconditional_remote_url : 1;\n +\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n \ttest_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n '\n \n -+test_expect_success 'includeIf.hasremoteurl' '\n ++test_expect_success 'includeIf.hasconfig:remote.*.url' '\n +\tgit init hasremoteurlTest &&\n +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n +\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\t\tthat = that-is-not-included\n +\tEOF\n +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n -+\t[includeIf \"hasremoteurl:foo\"]\n ++\t[includeIf \"hasconfig:remote.*.url:foo\"]\n +\t\tpath = \"$(pwd)/include-this\"\n -+\t[includeIf \"hasremoteurl:bar\"]\n ++\t[includeIf \"hasconfig:remote.*.url:bar\"]\n +\t\tpath = \"$(pwd)/dont-include-that\"\n +\t[remote \"foo\"]\n +\t\turl = foo\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\ttest_must_fail git -C hasremoteurlTest config --get user.that\n +'\n +\n -+test_expect_success 'includeIf.hasremoteurl respects last-config-wins' '\n ++test_expect_success 'includeIf.hasconfig:remote.*.url respects last-config-wins' '\n +\tgit init hasremoteurlTest &&\n +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n +\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\t[user]\n +\t\tone = main-config\n +\t\ttwo = main-config\n -+\t[includeIf \"hasremoteurl:foo\"]\n ++\t[includeIf \"hasconfig:remote.*.url:foo\"]\n +\t\tpath = \"$(pwd)/include-two-three\"\n +\t[user]\n +\t\tthree = main-config\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\ttest_cmp expect-main-config actual\n +'\n +\n -+test_expect_success 'includeIf.hasremoteurl globs' '\n ++test_expect_success 'includeIf.hasconfig:remote.*.url globs' '\n +\tgit init hasremoteurlTest &&\n +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n +\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n +\t[remote \"foo\"]\n +\t\turl = https://foo/bar/baz\n -+\t[includeIf \"hasremoteurl:**/baz\"]\n ++\t[includeIf \"hasconfig:remote.*.url:**/baz\"]\n +\t\tpath = \"$(pwd)/double-star-start\"\n -+\t[includeIf \"hasremoteurl:**/nomatch\"]\n ++\t[includeIf \"hasconfig:remote.*.url:**/nomatch\"]\n +\t\tpath = \"$(pwd)/no\"\n -+\t[includeIf \"hasremoteurl:https:/**\"]\n ++\t[includeIf \"hasconfig:remote.*.url:https:/**\"]\n +\t\tpath = \"$(pwd)/double-star-end\"\n -+\t[includeIf \"hasremoteurl:nomatch:/**\"]\n ++\t[includeIf \"hasconfig:remote.*.url:nomatch:/**\"]\n +\t\tpath = \"$(pwd)/no\"\n -+\t[includeIf \"hasremoteurl:https:/**/baz\"]\n ++\t[includeIf \"hasconfig:remote.*.url:https:/**/baz\"]\n +\t\tpath = \"$(pwd)/double-star-middle\"\n -+\t[includeIf \"hasremoteurl:https:/**/nomatch\"]\n ++\t[includeIf \"hasconfig:remote.*.url:https:/**/nomatch\"]\n +\t\tpath = \"$(pwd)/no\"\n -+\t[includeIf \"hasremoteurl:https://*/bar/baz\"]\n ++\t[includeIf \"hasconfig:remote.*.url:https://*/bar/baz\"]\n +\t\tpath = \"$(pwd)/single-star-middle\"\n -+\t[includeIf \"hasremoteurl:https://*/baz\"]\n ++\t[includeIf \"hasconfig:remote.*.url:https://*/baz\"]\n +\t\tpath = \"$(pwd)/no\"\n +\tEOF\n +\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\ttest_must_fail git -C hasremoteurlTest config --get user.no\n +'\n +\n -+test_expect_success 'includeIf.hasremoteurl forbids remote url in such included files' '\n ++test_expect_success 'includeIf.hasconfig:remote.*.url forbids remote url in such included files' '\n +\tgit init hasremoteurlTest &&\n +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n +\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\t\turl = bar\n +\tEOF\n +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n -+\t[includeIf \"hasremoteurl:foo\"]\n ++\t[includeIf \"hasconfig:remote.*.url:foo\"]\n +\t\tpath = \"$(pwd)/include-with-url\"\n +\tEOF\n +\n +\t# test with any Git command\n +\ttest_must_fail git -C hasremoteurlTest status 2>err &&\n -+\tgrep \"fatal: remote URLs cannot be configured in file directly or indirectly included by includeIf.hasremoteurl\" err\n ++\tgrep \"fatal: remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\" err\n +'\n +\n test_done\n-- \n2.34.1.400.ga245620fadb-goog\n\n"},{"id":"442921","messageId":"b2dcae03ed3016f96a5368db925fa3736c3d7c58.1638487815.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1638487815.git.jonathantanmy@google.com","subject":"[PATCH v5 1/2] config: make git_config_include() static","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-02T23:31:37Z","receivedAt":"2021-12-02T23:31:47Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"It is not used from outside the file in which it is declared.\n\nSigned-off-by: Jonathan Tan \n---\n config.c | 12 +++++++++++-\n config.h | 37 ++++---------------------------------\n 2 files changed, 15 insertions(+), 34 deletions(-)\n\ndiff --git a/config.c b/config.c\nindex 2dcbe901b6..94ad5ce913 100644\n--- a/config.c\n+++ b/config.c\n@@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n \treturn conf->u.buf.pos;\n }\n \n+struct config_include_data {\n+\tint depth;\n+\tconfig_fn_t fn;\n+\tvoid *data;\n+\tconst struct config_options *opts;\n+};\n+#define CONFIG_INCLUDE_INIT { 0 }\n+\n+static int git_config_include(const char *var, const char *value, void *data);\n+\n #define MAX_INCLUDE_DEPTH 10\n static const char include_depth_advice[] = N_(\n \"exceeded maximum include depth (%d) while including\\n\"\n@@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n-int git_config_include(const char *var, const char *value, void *data)\n+static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n \tconst char *cond, *key;\ndiff --git a/config.h b/config.h\nindex f119de0130..48a5e472ca 100644\n--- a/config.h\n+++ b/config.h\n@@ -126,6 +126,8 @@ int git_default_config(const char *, const char *, void *);\n /**\n * Read a specific file in git-config format.\n * This function takes the same callback and data parameters as `git_config`.\n+ *\n+ * Unlike git_config(), this function does not respect includes.\n */\n int git_config_from_file(config_fn_t fn, const char *, void *);\n \n@@ -158,6 +160,8 @@ void read_very_early_config(config_fn_t cb, void *data);\n * will first feed the user-wide one to the callback, and then the\n * repo-specific one; by overwriting, the higher-priority repo-specific\n * value is left at the end).\n+ *\n+ * Unlike git_config_from_file(), this function respects includes.\n */\n void git_config(config_fn_t fn, void *);\n \n@@ -338,39 +342,6 @@ const char *current_config_origin_type(void);\n const char *current_config_name(void);\n int current_config_line(void);\n \n-/**\n- * Include Directives\n- * ------------------\n- *\n- * By default, the config parser does not respect include directives.\n- * However, a caller can use the special `git_config_include` wrapper\n- * callback to support them. To do so, you simply wrap your \"real\" callback\n- * function and data pointer in a `struct config_include_data`, and pass\n- * the wrapper to the regular config-reading functions. For example:\n- *\n- * -------------------------------------------\n- * int read_file_with_include(const char *file, config_fn_t fn, void *data)\n- * {\n- * struct config_include_data inc = CONFIG_INCLUDE_INIT;\n- * inc.fn = fn;\n- * inc.data = data;\n- * return git_config_from_file(git_config_include, file, &inc);\n- * }\n- * -------------------------------------------\n- *\n- * `git_config` respects includes automatically. The lower-level\n- * `git_config_from_file` does not.\n- *\n- */\n-struct config_include_data {\n-\tint depth;\n-\tconfig_fn_t fn;\n-\tvoid *data;\n-\tconst struct config_options *opts;\n-};\n-#define CONFIG_INCLUDE_INIT { 0 }\n-int git_config_include(const char *name, const char *value, void *data);\n-\n /*\n * Match and parse a config key of the form:\n *\n-- \n2.34.1.400.ga245620fadb-goog\n\n"},{"id":"442922","messageId":"d3b8e007177f93b53dcb43d88890228b468b4975.1638487815.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1638487815.git.jonathantanmy@google.com","subject":"[PATCH v5 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-02T23:31:38Z","receivedAt":"2021-12-02T23:31:48Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This is a feature that supports config file inclusion conditional on\nwhether the repo has a remote with a URL that matches a glob.\n\nSimilar to my previous work on remote-suggested hooks [1], the main\nmotivation is to allow remote repo administrators to provide recommended\nconfigs in a way that can be consumed more easily (e.g. through a\npackage installable by a package manager - it could, for example,\ncontain a file to be included conditionally and a post-install script\nthat adds the include directive to the system-wide config file).\n\nIn order to do this, Git reruns the config parsing mechanism upon\nnoticing the first URL-conditional include in order to find all remote\nURLs, and these remote URLs are then used to determine if that first and\nall subsequent includes are executed. Remote URLs are not allowed to be\nconfigued in any URL-conditionally-included file.\n\n[1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n\nSigned-off-by: Jonathan Tan \n---\n Documentation/config.txt | 16 +++++\n config.c | 122 +++++++++++++++++++++++++++++++++++----\n config.h | 9 +++\n t/t1300-config.sh | 118 +++++++++++++++++++++++++++++++++++++\n 4 files changed, 255 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 0c0e6b859f..e0e5ca558e 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -159,6 +159,22 @@ all branches that begin with `foo/`. This is useful if your branches are\n organized hierarchically and you would like to apply a configuration to\n all the branches in that hierarchy.\n \n+`hasconfig:remote.*.url:`::\n+\tThe data that follows this keyword is taken to\n+\tbe a pattern with standard globbing wildcards and two\n+\tadditional ones, `**/` and `/**`, that can match multiple\n+\tcomponents. The first time this keyword is seen, the rest of\n+\tthe config files will be scanned for remote URLs (without\n+\tapplying any values). If there exists at least one remote URL\n+\tthat matches this pattern, the include condition is met.\n++\n+Files included by this option (directly or indirectly) are not allowed\n+to contain remote URLs.\n++\n+This keyword is designed to be forwards compatible with a naming\n+scheme that supports more variable-based include conditions, but\n+currently Git only supports the exact keyword described above.\n+\n A few more notes on matching via `gitdir` and `gitdir/i`:\n \n * Symlinks in `$GIT_DIR` are not resolved before matching.\ndiff --git a/config.c b/config.c\nindex 94ad5ce913..d2cf95add2 100644\n--- a/config.c\n+++ b/config.c\n@@ -125,6 +125,12 @@ struct config_include_data {\n \tconfig_fn_t fn;\n \tvoid *data;\n \tconst struct config_options *opts;\n+\tstruct git_config_source *config_source;\n+\n+\t/*\n+\t * All remote URLs discovered when reading all config files.\n+\t */\n+\tstruct string_list *remote_urls;\n };\n #define CONFIG_INCLUDE_INIT { 0 }\n \n@@ -301,16 +307,97 @@ static int include_by_branch(const char *cond, size_t cond_len)\n \treturn ret;\n }\n \n-static int include_condition_is_true(const struct config_options *opts,\n+static int add_remote_url(const char *var, const char *value, void *data)\n+{\n+\tstruct string_list *remote_urls = data;\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tstring_list_append(remote_urls, value);\n+\treturn 0;\n+}\n+\n+static void populate_remote_urls(struct config_include_data *inc)\n+{\n+\tstruct config_options opts;\n+\n+\tstruct config_source *store_cf = cf;\n+\tstruct key_value_info *store_kvi = current_config_kvi;\n+\tenum config_scope store_scope = current_parsing_scope;\n+\n+\topts = *inc->opts;\n+\topts.unconditional_remote_url = 1;\n+\n+\tcf = NULL;\n+\tcurrent_config_kvi = NULL;\n+\tcurrent_parsing_scope = 0;\n+\n+\tinc->remote_urls = xmalloc(sizeof(*inc->remote_urls));\n+\tstring_list_init_dup(inc->remote_urls);\n+\tconfig_with_options(add_remote_url, inc->remote_urls, inc->config_source, &opts);\n+\n+\tcf = store_cf;\n+\tcurrent_config_kvi = store_kvi;\n+\tcurrent_parsing_scope = store_scope;\n+}\n+\n+static int forbid_remote_url(const char *var, const char *value, void *data)\n+{\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\"));\n+\treturn 0;\n+}\n+\n+static int at_least_one_url_matches_glob(const char *glob, int glob_len,\n+\t\t\t\t\t struct string_list *remote_urls)\n+{\n+\tstruct strbuf pattern = STRBUF_INIT;\n+\tstruct string_list_item *url_item;\n+\tint found = 0;\n+\n+\tstrbuf_add(&pattern, glob, glob_len);\n+\tfor_each_string_list_item(url_item, remote_urls) {\n+\t\tif (!wildmatch(pattern.buf, url_item->string, WM_PATHNAME)) {\n+\t\t\tfound = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tstrbuf_release(&pattern);\n+\treturn found;\n+}\n+\n+static int include_condition_is_true(struct config_include_data *inc,\n \t\t\t\t const char *cond, size_t cond_len)\n {\n+\tconst struct config_options *opts = inc->opts;\n \n-\tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len))\n+\tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len)) {\n \t\treturn include_by_gitdir(opts, cond, cond_len, 0);\n-\telse if (skip_prefix_mem(cond, cond_len, \"gitdir/i:\", &cond, &cond_len))\n+\t} else if (skip_prefix_mem(cond, cond_len, \"gitdir/i:\", &cond, &cond_len)) {\n \t\treturn include_by_gitdir(opts, cond, cond_len, 1);\n-\telse if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len))\n+\t} else if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len)) {\n \t\treturn include_by_branch(cond, cond_len);\n+\t} else if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n+\t\t\t\t &cond_len)) {\n+\t\tif (inc->opts->unconditional_remote_url)\n+\t\t\treturn 1;\n+\t\tif (!inc->remote_urls)\n+\t\t\tpopulate_remote_urls(inc);\n+\t\treturn at_least_one_url_matches_glob(cond, cond_len,\n+\t\t\t\t\t\t inc->remote_urls);\n+\t}\n \n \t/* unknown conditionals are always false */\n \treturn 0;\n@@ -335,9 +422,16 @@ static int git_config_include(const char *var, const char *value, void *data)\n \t\tret = handle_path_include(value, inc);\n \n \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n-\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n-\t !strcmp(key, \"path\"))\n+\t cond && include_condition_is_true(inc, cond, cond_len) &&\n+\t !strcmp(key, \"path\")) {\n+\t\tconfig_fn_t old_fn = inc->fn;\n+\n+\t\tif (inc->opts->unconditional_remote_url)\n+\t\t\tinc->fn = forbid_remote_url;\n \t\tret = handle_path_include(value, inc);\n+\t\tif (inc->opts->unconditional_remote_url)\n+\t\t\tinc->fn = old_fn;\n+\t}\n \n \treturn ret;\n }\n@@ -1933,11 +2027,13 @@ int config_with_options(config_fn_t fn, void *data,\n \t\t\tconst struct config_options *opts)\n {\n \tstruct config_include_data inc = CONFIG_INCLUDE_INIT;\n+\tint ret;\n \n \tif (opts->respect_includes) {\n \t\tinc.fn = fn;\n \t\tinc.data = data;\n \t\tinc.opts = opts;\n+\t\tinc.config_source = config_source;\n \t\tfn = git_config_include;\n \t\tdata = &inc;\n \t}\n@@ -1950,17 +2046,23 @@ int config_with_options(config_fn_t fn, void *data,\n \t * regular lookup sequence.\n \t */\n \tif (config_source && config_source->use_stdin) {\n-\t\treturn git_config_from_stdin(fn, data);\n+\t\tret = git_config_from_stdin(fn, data);\n \t} else if (config_source && config_source->file) {\n-\t\treturn git_config_from_file(fn, config_source->file, data);\n+\t\tret = git_config_from_file(fn, config_source->file, data);\n \t} else if (config_source && config_source->blob) {\n \t\tstruct repository *repo = config_source->repo ?\n \t\t\tconfig_source->repo : the_repository;\n-\t\treturn git_config_from_blob_ref(fn, repo, config_source->blob,\n+\t\tret = git_config_from_blob_ref(fn, repo, config_source->blob,\n \t\t\t\t\t\tdata);\n+\t} else {\n+\t\tret = do_git_config_sequence(opts, fn, data);\n \t}\n \n-\treturn do_git_config_sequence(opts, fn, data);\n+\tif (inc.remote_urls) {\n+\t\tstring_list_clear(inc.remote_urls, 0);\n+\t\tFREE_AND_NULL(inc.remote_urls);\n+\t}\n+\treturn ret;\n }\n \n static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)\ndiff --git a/config.h b/config.h\nindex 48a5e472ca..ab0106d287 100644\n--- a/config.h\n+++ b/config.h\n@@ -89,6 +89,15 @@ struct config_options {\n \tunsigned int ignore_worktree : 1;\n \tunsigned int ignore_cmdline : 1;\n \tunsigned int system_gently : 1;\n+\n+\t/*\n+\t * For internal use. Include all includeif.hasremoteurl paths without\n+\t * checking if the repo has that remote URL, and when doing so, verify\n+\t * that files included in this way do not configure any remote URLs\n+\t * themselves.\n+\t */\n+\tunsigned int unconditional_remote_url : 1;\n+\n \tconst char *commondir;\n \tconst char *git_dir;\n \tconfig_parser_event_fn_t event_fn;\ndiff --git a/t/t1300-config.sh b/t/t1300-config.sh\nindex 9ff46f3b04..0f7bae31b4 100755\n--- a/t/t1300-config.sh\n+++ b/t/t1300-config.sh\n@@ -2387,4 +2387,122 @@ test_expect_success '--get and --get-all with --fixed-value' '\n \ttest_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n '\n \n+test_expect_success 'includeIf.hasconfig:remote.*.url' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >\"$(pwd)\"/include-this <<-\\EOF &&\n+\t[user]\n+\t\tthis = this-is-included\n+\tEOF\n+\tcat >\"$(pwd)\"/dont-include-that <<-\\EOF &&\n+\t[user]\n+\t\tthat = that-is-not-included\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n+\t\tpath = \"$(pwd)/include-this\"\n+\t[includeIf \"hasconfig:remote.*.url:bar\"]\n+\t\tpath = \"$(pwd)/dont-include-that\"\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\tEOF\n+\n+\techo this-is-included >expect-this &&\n+\tgit -C hasremoteurlTest config --get user.this >actual-this &&\n+\ttest_cmp expect-this actual-this &&\n+\n+\ttest_must_fail git -C hasremoteurlTest config --get user.that\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url respects last-config-wins' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >\"$(pwd)\"/include-two-three <<-\\EOF &&\n+\t[user]\n+\t\ttwo = included-config\n+\t\tthree = included-config\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\t[user]\n+\t\tone = main-config\n+\t\ttwo = main-config\n+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n+\t\tpath = \"$(pwd)/include-two-three\"\n+\t[user]\n+\t\tthree = main-config\n+\tEOF\n+\n+\techo main-config >expect-main-config &&\n+\techo included-config >expect-included-config &&\n+\n+\tgit -C hasremoteurlTest config --get user.one >actual &&\n+\ttest_cmp expect-main-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.two >actual &&\n+\ttest_cmp expect-included-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.three >actual &&\n+\ttest_cmp expect-main-config actual\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url globs' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tprintf \"[user]\\ndss = yes\\n\" >\"$(pwd)/double-star-start\" &&\n+\tprintf \"[user]\\ndse = yes\\n\" >\"$(pwd)/double-star-end\" &&\n+\tprintf \"[user]\\ndsm = yes\\n\" >\"$(pwd)/double-star-middle\" &&\n+\tprintf \"[user]\\nssm = yes\\n\" >\"$(pwd)/single-star-middle\" &&\n+\tprintf \"[user]\\nno = no\\n\" >\"$(pwd)/no\" &&\n+\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = https://foo/bar/baz\n+\t[includeIf \"hasconfig:remote.*.url:**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-start\"\n+\t[includeIf \"hasconfig:remote.*.url:**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**\"]\n+\t\tpath = \"$(pwd)/double-star-end\"\n+\t[includeIf \"hasconfig:remote.*.url:nomatch:/**\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-middle\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https://*/bar/baz\"]\n+\t\tpath = \"$(pwd)/single-star-middle\"\n+\t[includeIf \"hasconfig:remote.*.url:https://*/baz\"]\n+\t\tpath = \"$(pwd)/no\"\n+\tEOF\n+\n+\tgit -C hasremoteurlTest config --get user.dss &&\n+\tgit -C hasremoteurlTest config --get user.dse &&\n+\tgit -C hasremoteurlTest config --get user.dsm &&\n+\tgit -C hasremoteurlTest config --get user.ssm &&\n+\ttest_must_fail git -C hasremoteurlTest config --get user.no\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url forbids remote url in such included files' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >\"$(pwd)\"/include-with-url <<-\\EOF &&\n+\t[remote \"bar\"]\n+\t\turl = bar\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n+\t\tpath = \"$(pwd)/include-with-url\"\n+\tEOF\n+\n+\t# test with any Git command\n+\ttest_must_fail git -C hasremoteurlTest status 2>err &&\n+\tgrep \"fatal: remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\" err\n+'\n+\n test_done\n-- \n2.34.1.400.ga245620fadb-goog\n\n"},{"id":"443189","messageId":"211206.86zgpdpmyy.gmgdl@evledraar.gmail.com","threadId":"56696","inReplyTo":"cover.1638487815.git.jonathantanmy@google.com","subject":"Re: [PATCH v5 0/2] Conditional config includes based on remote URL","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-12-06T18:57:42Z","receivedAt":"2021-12-06T19:11:54Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Dec 02 2021, Jonathan Tan wrote:\n\n> Thanks, Junio, for your comments. I think the code is more clearly laid \n> out now. \n> \n> The main changes from v4 are that I've maintained the existing code\n> structure more, and changed the keyword used to something that hopefully\n> will be more forwards compatible. I've also updated the documentation to\n> explain the forwards compatibility idea.\n\nI read through this and came up with the below as a proposed squash-in\njust while reading through it. These may or may not help. Changes:\n\n * There was some needless \"$(pwd)\" in the tests\n * Inlining the \"remote_urls\" in the struct makes its management easier;\n and the free/NULL checks just check .nr now, and string_list_clear() can be\n unconditional.\n * Created a include_by_remote_url() function. Makes the overall diff smaller\n since you don't need to add braces to everything in include_condition_is_true()\n\nOther comments (not related to the below):\n\n * It would be nice if e.g. the \"includeIf.hasconfig:remote.*.url globs\" test\n were split up by condition, but maybe that's a hassle (would need a small helper).\n\n Just something that would have helped while hacking on this, i.e. now most of it\n was an all-or-nothing failure & peek at the trace output\n\n * Your last test appears to entirely forbid recursion. I.e. we die if you include config\n which in turn tries to use this include mechanism, right?\n\n That's probably wise, and it is explicitly documented.\n\n But as far as the documentation about this being a forward-compatible facility, do we\n think that this limitation would apply to any future config key? I.e. if I include based\n on \"user.email\" nothing in that to-be-included can set user.email?\n\n That's probably OK, just wondering. In any case it can always be expanded later on.\n\ndiff --git a/config.c b/config.c\nindex 39ac38e0e78..91b0a328e59 100644\n--- a/config.c\n+++ b/config.c\n@@ -130,9 +130,11 @@ struct config_include_data {\n \t/*\n \t * All remote URLs discovered when reading all config files.\n \t */\n-\tstruct string_list *remote_urls;\n+\tstruct string_list remote_urls;\n };\n-#define CONFIG_INCLUDE_INIT { 0 }\n+#define CONFIG_INCLUDE_INIT { \\\n+\t.remote_urls = STRING_LIST_INIT_DUP, \\\n+}\n \n static int git_config_include(const char *var, const char *value, void *data);\n \n@@ -340,9 +342,7 @@ static void populate_remote_urls(struct config_include_data *inc)\n \tcurrent_config_kvi = NULL;\n \tcurrent_parsing_scope = 0;\n \n-\tinc->remote_urls = xmalloc(sizeof(*inc->remote_urls));\n-\tstring_list_init_dup(inc->remote_urls);\n-\tconfig_with_options(add_remote_url, inc->remote_urls, inc->config_source, &opts);\n+\tconfig_with_options(add_remote_url, &inc->remote_urls, inc->config_source, &opts);\n \n \tcf = store_cf;\n \tcurrent_config_kvi = store_kvi;\n@@ -381,26 +381,31 @@ static int at_least_one_url_matches_glob(const char *glob, int glob_len,\n \treturn found;\n }\n \n+static int include_by_remote_url(struct config_include_data *inc,\n+\t\t\t\t const char *cond, size_t cond_len)\n+{\n+\tif (inc->opts->unconditional_remote_url)\n+\t\treturn 1;\n+\tif (!inc->remote_urls.nr)\n+\t\tpopulate_remote_urls(inc);\n+\treturn at_least_one_url_matches_glob(cond, cond_len,\n+\t\t\t\t\t &inc->remote_urls);\n+}\n+\n static int include_condition_is_true(struct config_include_data *inc,\n \t\t\t\t const char *cond, size_t cond_len)\n {\n \tconst struct config_options *opts = inc->opts;\n \n-\tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len)) {\n+\tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len))\n \t\treturn include_by_gitdir(opts, cond, cond_len, 0);\n-\t} else if (skip_prefix_mem(cond, cond_len, \"gitdir/i:\", &cond, &cond_len)) {\n+\telse if (skip_prefix_mem(cond, cond_len, \"gitdir/i:\", &cond, &cond_len))\n \t\treturn include_by_gitdir(opts, cond, cond_len, 1);\n-\t} else if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len)) {\n+\telse if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len))\n \t\treturn include_by_branch(cond, cond_len);\n-\t} else if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n-\t\t\t\t &cond_len)) {\n-\t\tif (inc->opts->unconditional_remote_url)\n-\t\t\treturn 1;\n-\t\tif (!inc->remote_urls)\n-\t\t\tpopulate_remote_urls(inc);\n-\t\treturn at_least_one_url_matches_glob(cond, cond_len,\n-\t\t\t\t\t\t inc->remote_urls);\n-\t}\n+\telse if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n+\t\t\t\t &cond_len))\n+\t\treturn include_by_remote_url(inc, cond, cond_len);\n \n \t/* unknown conditionals are always false */\n \treturn 0;\n@@ -2061,10 +2066,7 @@ int config_with_options(config_fn_t fn, void *data,\n \t\tret = do_git_config_sequence(opts, fn, data);\n \t}\n \n-\tif (inc.remote_urls) {\n-\t\tstring_list_clear(inc.remote_urls, 0);\n-\t\tFREE_AND_NULL(inc.remote_urls);\n-\t}\n+\tstring_list_clear(&inc.remote_urls, 0);\n \treturn ret;\n }\n \ndiff --git a/t/t1300-config.sh b/t/t1300-config.sh\nindex 0f7bae31b4b..8310562b842 100755\n--- a/t/t1300-config.sh\n+++ b/t/t1300-config.sh\n@@ -2391,11 +2391,11 @@ test_expect_success 'includeIf.hasconfig:remote.*.url' '\n \tgit init hasremoteurlTest &&\n \ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n \n-\tcat >\"$(pwd)\"/include-this <<-\\EOF &&\n+\tcat >include-this <<-\\EOF &&\n \t[user]\n \t\tthis = this-is-included\n \tEOF\n-\tcat >\"$(pwd)\"/dont-include-that <<-\\EOF &&\n+\tcat >dont-include-that <<-\\EOF &&\n \t[user]\n \t\tthat = that-is-not-included\n \tEOF\n@@ -2419,7 +2419,7 @@ test_expect_success 'includeIf.hasconfig:remote.*.url respects last-config-wins'\n \tgit init hasremoteurlTest &&\n \ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n \n-\tcat >\"$(pwd)\"/include-two-three <<-\\EOF &&\n+\tcat >include-two-three <<-\\EOF &&\n \t[user]\n \t\ttwo = included-config\n \t\tthree = included-config\n@@ -2453,11 +2453,11 @@ test_expect_success 'includeIf.hasconfig:remote.*.url globs' '\n \tgit init hasremoteurlTest &&\n \ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n \n-\tprintf \"[user]\\ndss = yes\\n\" >\"$(pwd)/double-star-start\" &&\n-\tprintf \"[user]\\ndse = yes\\n\" >\"$(pwd)/double-star-end\" &&\n-\tprintf \"[user]\\ndsm = yes\\n\" >\"$(pwd)/double-star-middle\" &&\n-\tprintf \"[user]\\nssm = yes\\n\" >\"$(pwd)/single-star-middle\" &&\n-\tprintf \"[user]\\nno = no\\n\" >\"$(pwd)/no\" &&\n+\tprintf \"[user]\\ndss = yes\\n\" >double-star-start &&\n+\tprintf \"[user]\\ndse = yes\\n\" >double-star-end &&\n+\tprintf \"[user]\\ndsm = yes\\n\" >double-star-middle &&\n+\tprintf \"[user]\\nssm = yes\\n\" >single-star-middle &&\n+\tprintf \"[user]\\nno = no\\n\" >no &&\n \n \tcat >>hasremoteurlTest/.git/config <<-EOF &&\n \t[remote \"foo\"]\n@@ -2491,7 +2491,7 @@ test_expect_success 'includeIf.hasconfig:remote.*.url forbids remote url in such\n \tgit init hasremoteurlTest &&\n \ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n \n-\tcat >\"$(pwd)\"/include-with-url <<-\\EOF &&\n+\tcat >include-with-url <<-\\EOF &&\n \t[remote \"bar\"]\n \t\turl = bar\n \tEOF\n\n"},{"id":"443210","messageId":"kl6lmtldcqjq.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56696","inReplyTo":"d3b8e007177f93b53dcb43d88890228b468b4975.1638487815.git.jonathantanmy@google.com","subject":"Re: [PATCH v5 2/2] config: include file if remote URL matches a glob","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2021-12-06T22:32:57Z","receivedAt":"2021-12-06T22:33:02Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Jonathan Tan writes:\n\n> Documentation/config.txt | 16 +++++\n> config.c | 122 +++++++++++++++++++++++++++++++++++----\n> config.h | 9 +++\n> t/t1300-config.sh | 118 +++++++++++++++++++++++++++++++++++++\n> 4 files changed, 255 insertions(+), 10 deletions(-)\n>\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 0c0e6b859f..e0e5ca558e 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -159,6 +159,22 @@ all branches that begin with `foo/`. This is useful if your branches are\n> organized hierarchically and you would like to apply a configuration to\n> all the branches in that hierarchy.\n> \n> +`hasconfig:remote.*.url:`::\n> +\tThe data that follows this keyword is taken to\n> +\tbe a pattern with standard globbing wildcards and two\n> +\tadditional ones, `**/` and `/**`, that can match multiple\n> +\tcomponents. The first time this keyword is seen, the rest of\n> +\tthe config files will be scanned for remote URLs (without\n> +\tapplying any values). If there exists at least one remote URL\n> +\tthat matches this pattern, the include condition is met.\n> ++\n> +Files included by this option (directly or indirectly) are not allowed\n> +to contain remote URLs.\n> ++\n> +This keyword is designed to be forwards compatible with a naming\n> +scheme that supports more variable-based include conditions, but\n> +currently Git only supports the exact keyword described above.\n> +\n\nA reader of this description doesn't have any reason to think that\n`hasconfig:remote.*.url` wouldn't respect in-place semantics, so my\nconcern in [1] is addressed.\n\n`hasconfig:foo.*.bar` seems reasonable from a forwards-compatibility\nperspective. Ideally, it would be nice to see a generic implementation\nthat actually handles config values beyond `remote.*.url`, but unless we\ntake a closer look at all config values and the conditions we would like\nto support, a generic implementation seems like a premature\noptimization that won't age well.\n\nSo OK to having a forward-compatible name without a forward compatible\nimplementation.\n\n> +static int include_condition_is_true(struct config_include_data *inc,\n> \t\t\t\t const char *cond, size_t cond_len)\n> {\n> +\tconst struct config_options *opts = inc->opts;\n> \n> -\tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len))\n> +\tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len)) {\n> \t\treturn include_by_gitdir(opts, cond, cond_len, 0);\n> -\telse if (skip_prefix_mem(cond, cond_len, \"gitdir/i:\", &cond, &cond_len))\n> +\t} else if (skip_prefix_mem(cond, cond_len, \"gitdir/i:\", &cond, &cond_len)) {\n> \t\treturn include_by_gitdir(opts, cond, cond_len, 1);\n> -\telse if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len))\n> +\t} else if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len)) {\n> \t\treturn include_by_branch(cond, cond_len);\n> +\t} else if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n> +\t\t\t\t &cond_len)) {\n> +\t\tif (inc->opts->unconditional_remote_url)\n> +\t\t\treturn 1;\n> +\t\tif (!inc->remote_urls)\n> +\t\t\tpopulate_remote_urls(inc);\n> +\t\treturn at_least_one_url_matches_glob(cond, cond_len,\n> +\t\t\t\t\t\t inc->remote_urls);\n> +\t}\n> \n> \t/* unknown conditionals are always false */\n> \treturn 0;\n\nNit: I have a preference for Ævar's version [2], which looks more\nconsistent with the rest of the function i.e. handling the match using a\nhelper function.\n\n> +test_expect_success 'includeIf.hasconfig:remote.*.url forbids remote url in such included files' '\n> +\tgit init hasremoteurlTest &&\n> +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n> +\n> +\tcat >\"$(pwd)\"/include-with-url <<-\\EOF &&\n> +\t[remote \"bar\"]\n> +\t\turl = bar\n> +\tEOF\n> +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n> +\t[includeIf \"hasconfig:remote.*.url:foo\"]\n> +\t\tpath = \"$(pwd)/include-with-url\"\n> +\tEOF\n> +\n> +\t# test with any Git command\n> +\ttest_must_fail git -C hasremoteurlTest status 2>err &&\n> +\tgrep \"fatal: remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\" err\n> +'\n> +\n> test_done\n> -- \n> 2.34.1.400.ga245620fadb-goog\n\nThis addresses the test coverage comment in [1]. Great!\n\n[1] https://lore.kernel.org/git/kl6lilwjre3m.fsf@chooglen-macbookpro.roam.corp.google.com\n[2] https://lore.kernel.org/git/211206.86zgpdpmyy.gmgdl@evledraar.gmail.com\n"},{"id":"443312","messageId":"20211207174617.1118046-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"211206.86zgpdpmyy.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v5 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-07T17:46:17Z","receivedAt":"2021-12-07T17:46:23Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> I read through this and came up with the below as a proposed squash-in\n> just while reading through it. These may or may not help. Changes:\n> \n> * There was some needless \"$(pwd)\" in the tests\n\nAh, thanks for catching that.\n\n> * Inlining the \"remote_urls\" in the struct makes its management easier;\n> and the free/NULL checks just check .nr now, and string_list_clear() can be\n> unconditional.\n\nI don't think we can do this - nr might still be 0 after a scan if we\ndon't have remote URLs for some reason, so we still need to distinguish\nbetween not-scanned and scanned-with-zero-URLs.\n\n> * Created a include_by_remote_url() function. Makes the overall diff smaller\n> since you don't need to add braces to everything in include_condition_is_true()\n\nAh, good idea. I'll do this.\n\n> Other comments (not related to the below):\n> \n> * It would be nice if e.g. the \"includeIf.hasconfig:remote.*.url globs\" test\n> were split up by condition, but maybe that's a hassle (would need a small helper).\n> \n> Just something that would have helped while hacking on this, i.e. now most of it\n> was an all-or-nothing failure & peek at the trace output\n\nWhat do you mean by condition? There seems to only be one condition\n(whether the URL is there or not), unless you were thinking of smaller\nsubdivisions.\n\n> * Your last test appears to entirely forbid recursion. I.e. we die if you include config\n> which in turn tries to use this include mechanism, right?\n> \n> That's probably wise, and it is explicitly documented.\n> \n> But as far as the documentation about this being a forward-compatible facility, do we\n> think that this limitation would apply to any future config key? I.e. if I include based\n> on \"user.email\" nothing in that to-be-included can set user.email?\n> \n> That's probably OK, just wondering. In any case it can always be expanded later on.\n\nWe can decide later what the future facility will be, but I envision\nthat we will not allow included files to set config that can affect any\ninclude directives in use. So, for example, if I have a user.email-based\ninclude, none of my config-conditionally included files can set user.email.\n"},{"id":"443313","messageId":"20211207175302.1119160-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"kl6lmtldcqjq.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v5 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-07T17:53:02Z","receivedAt":"2021-12-07T17:53:06Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Glen Choo writes:\n> A reader of this description doesn't have any reason to think that\n> `hasconfig:remote.*.url` wouldn't respect in-place semantics, so my\n> concern in [1] is addressed.\n> \n> `hasconfig:foo.*.bar` seems reasonable from a forwards-compatibility\n> perspective. Ideally, it would be nice to see a generic implementation\n> that actually handles config values beyond `remote.*.url`, but unless we\n> take a closer look at all config values and the conditions we would like\n> to support, a generic implementation seems like a premature\n> optimization that won't age well.\n> \n> So OK to having a forward-compatible name without a forward compatible\n> implementation.\n\nThanks for taking a look at this.\n\n> Nit: I have a preference for Ævar's version [2], which looks more\n> consistent with the rest of the function i.e. handling the match using a\n> helper function.\n\nI agree - I'll use it.\n"},{"id":"443315","messageId":"211207.86k0ggnvfo.gmgdl@evledraar.gmail.com","threadId":"56696","inReplyTo":"20211207174617.1118046-1-jonathantanmy@google.com","subject":"Re: [PATCH v5 0/2] Conditional config includes based on remote URL","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-12-07T17:56:40Z","receivedAt":"2021-12-07T18:04:17Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Dec 07 2021, Jonathan Tan wrote:\n\n>> I read through this and came up with the below as a proposed squash-in\n>> just while reading through it. These may or may not help. Changes:\n>> \n>> * There was some needless \"$(pwd)\" in the tests\n>\n> Ah, thanks for catching that.\n>\n>> * Inlining the \"remote_urls\" in the struct makes its management easier;\n>> and the free/NULL checks just check .nr now, and string_list_clear() can be\n>> unconditional.\n>\n> I don't think we can do this - nr might still be 0 after a scan if we\n> don't have remote URLs for some reason, so we still need to distinguish\n> between not-scanned and scanned-with-zero-URLs.\n\nYou mean so that we don't double-free? The way string_list_clear()\nprotects against that, but maybe there's something else.\n\nWhatever it is (if there's anything) it could use test coverage then :)\n\n>> * Created a include_by_remote_url() function. Makes the overall diff smaller\n>> since you don't need to add braces to everything in include_condition_is_true()\n>\n> Ah, good idea. I'll do this.\n>\n>> Other comments (not related to the below):\n>> \n>> * It would be nice if e.g. the \"includeIf.hasconfig:remote.*.url globs\" test\n>> were split up by condition, but maybe that's a hassle (would need a small helper).\n>> \n>> Just something that would have helped while hacking on this, i.e. now most of it\n>> was an all-or-nothing failure & peek at the trace output\n>\n> What do you mean by condition? There seems to only be one condition\n> (whether the URL is there or not), unless you were thinking of smaller\n> subdivisions.\n\nMaybe I'm just misunderstanding the intent here, but aren't you trying\nto guard against the case of having a ~/.gitconfig that includes\n~/.gitconfig.d/for-this-url, and *that* file in turns changes the\nremote's \"url\" in its config, followed by another \"include if url\nmatches\" condition therein?\n\nI.e. I read (more like skimmed) the documentation & test at the end as\nforbidding that, but maybe that's OK?\n\n>> * Your last test appears to entirely forbid recursion. I.e. we die if you include config\n>> which in turn tries to use this include mechanism, right?\n>> \n>> That's probably wise, and it is explicitly documented.\n>> \n>> But as far as the documentation about this being a forward-compatible facility, do we\n>> think that this limitation would apply to any future config key? I.e. if I include based\n>> on \"user.email\" nothing in that to-be-included can set user.email?\n>> \n>> That's probably OK, just wondering. In any case it can always be expanded later on.\n>\n> We can decide later what the future facility will be, but I envision\n> that we will not allow included files to set config that can affect any\n> include directives in use. So, for example, if I have a user.email-based\n> include, none of my config-conditionally included files can set user.email.\n\nI didn't look deeply at the implementation at all, but why would this be\na problem?\n\nYou parse ~/.gitconfig, it has user.name=foo, then right after in that\nfile we do:\n\n [includeIf \"hasconfig:user.name:*foo*\"]\n path = ~/.gitconfig.d/foo\n\nNow the top of ~/.gitconfig.d/foo we have:\n\n [user]\n name = bar\n [includeIf \"hasconfig:user.name:*bar*\"]\n path = ~/.gitconfig.d/bar\n\nWhy would it matter that we included on user.name=foo before?\n\nDoesn't that only matter *while* we process that first \"path\" line? Once\nwe move past it we update our configset to user.name=bar once we hit the\n\"name\" line of the included file.\n\nThen when we get another \"hasconfig:user.name\" we just match it to our\ncurrent user.name=*bar*.\n\nNo?\n\nAnyway, I think it's fine to punt on it for now or whatever, just\ncurious...\n"},{"id":"443338","messageId":"20211207185256.2305093-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"211207.86k0ggnvfo.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v5 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-07T18:52:56Z","receivedAt":"2021-12-07T18:53:00Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> >> * Inlining the \"remote_urls\" in the struct makes its management easier;\n> >> and the free/NULL checks just check .nr now, and string_list_clear() can be\n> >> unconditional.\n> >\n> > I don't think we can do this - nr might still be 0 after a scan if we\n> > don't have remote URLs for some reason, so we still need to distinguish\n> > between not-scanned and scanned-with-zero-URLs.\n> \n> You mean so that we don't double-free? The way string_list_clear()\n> protects against that, but maybe there's something else.\n> \n> Whatever it is (if there's anything) it could use test coverage then :)\n\nNo - we only want to do one scan per config read. If we scan and there\nare no remote URLs, with your scheme, next time we encounter another\nincludeIf.hasconfig, we would need to scan again (because nr is still\n0). With my scheme, we can see that the pointer is non-NULL, so we know\nthat we have already scanned.\n\n> >> * It would be nice if e.g. the \"includeIf.hasconfig:remote.*.url globs\" test\n> >> were split up by condition, but maybe that's a hassle (would need a small helper).\n> >> \n> >> Just something that would have helped while hacking on this, i.e. now most of it\n> >> was an all-or-nothing failure & peek at the trace output\n> >\n> > What do you mean by condition? There seems to only be one condition\n> > (whether the URL is there or not), unless you were thinking of smaller\n> > subdivisions.\n> \n> Maybe I'm just misunderstanding the intent here, but aren't you trying\n> to guard against the case of having a ~/.gitconfig that includes\n> ~/.gitconfig.d/for-this-url, and *that* file in turns changes the\n> remote's \"url\" in its config, followed by another \"include if url\n> matches\" condition therein?\n> \n> I.e. I read (more like skimmed) the documentation & test at the end as\n> forbidding that, but maybe that's OK?\n\nIf we're including \"~/.gitconfig.d/for-this-url\" by includeIf.hasconfig,\nthen yes, I'm guarding against that and other similar conditions.\n\n> > We can decide later what the future facility will be, but I envision\n> > that we will not allow included files to set config that can affect any\n> > include directives in use. So, for example, if I have a user.email-based\n> > include, none of my config-conditionally included files can set user.email.\n> \n> I didn't look deeply at the implementation at all, but why would this be\n> a problem?\n> \n> You parse ~/.gitconfig, it has user.name=foo, then right after in that\n> file we do:\n> \n> [includeIf \"hasconfig:user.name:*foo*\"]\n> path = ~/.gitconfig.d/foo\n> \n> Now the top of ~/.gitconfig.d/foo we have:\n> \n> [user]\n> name = bar\n> [includeIf \"hasconfig:user.name:*bar*\"]\n> path = ~/.gitconfig.d/bar\n> \n> Why would it matter that we included on user.name=foo before?\n> \n> Doesn't that only matter *while* we process that first \"path\" line? Once\n> we move past it we update our configset to user.name=bar once we hit the\n> \"name\" line of the included file.\n> \n> Then when we get another \"hasconfig:user.name\" we just match it to our\n> current user.name=*bar*.\n> \n> No?\n> \n> Anyway, I think it's fine to punt on it for now or whatever, just\n> curious...\n\nWell, we can't punt on it because what you describe also applies to\nremote URL :-)\n\nSo what you're saying is that once we have decided to include a file, we\nalways include it in its entirety regardless of whether the condition\nchanges during the file's include. That's reasonable, but other people\ncould have differing opinions. In this case, I think it's fine just to\nprohibit it entirely. In the future, we may look into relaxing this\ncondition.\n"},{"id":"443384","messageId":"cover.1638919346.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"[PATCH v6 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-07T23:23:50Z","receivedAt":"2021-12-07T23:23:59Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Here's a reroll addressing Ævar's comments about needless $(pwd) and\nseparating out include_by_remote_url() to make things more consistent.\n\nJonathan Tan (2):\n config: make git_config_include() static\n config: include file if remote URL matches a glob\n\n Documentation/config.txt | 16 +++++\n config.c | 133 ++++++++++++++++++++++++++++++++++++---\n config.h | 46 ++++----------\n t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++\n 4 files changed, 272 insertions(+), 41 deletions(-)\n\nRange-diff against v5:\n1: b2dcae03ed = 1: b2dcae03ed config: make git_config_include() static\n2: d3b8e00717 ! 2: de2be06818 config: include file if remote URL matches a glob\n @@ config.c: static int include_by_branch(const char *cond, size_t cond_len)\n +\treturn found;\n +}\n +\n ++static int include_by_remote_url(struct config_include_data *inc,\n ++\t\tconst char *cond, size_t cond_len)\n ++{\n ++\tif (inc->opts->unconditional_remote_url)\n ++\t\treturn 1;\n ++\tif (!inc->remote_urls)\n ++\t\tpopulate_remote_urls(inc);\n ++\treturn at_least_one_url_matches_glob(cond, cond_len,\n ++\t\t\t\t\t inc->remote_urls);\n ++}\n ++\n +static int include_condition_is_true(struct config_include_data *inc,\n \t\t\t\t const char *cond, size_t cond_len)\n {\n +\tconst struct config_options *opts = inc->opts;\n \n --\tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len))\n -+\tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len)) {\n + \tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len))\n \t\treturn include_by_gitdir(opts, cond, cond_len, 0);\n --\telse if (skip_prefix_mem(cond, cond_len, \"gitdir/i:\", &cond, &cond_len))\n -+\t} else if (skip_prefix_mem(cond, cond_len, \"gitdir/i:\", &cond, &cond_len)) {\n +@@ config.c: static int include_condition_is_true(const struct config_options *opts,\n \t\treturn include_by_gitdir(opts, cond, cond_len, 1);\n --\telse if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len))\n -+\t} else if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len)) {\n + \telse if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len))\n \t\treturn include_by_branch(cond, cond_len);\n -+\t} else if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n -+\t\t\t\t &cond_len)) {\n -+\t\tif (inc->opts->unconditional_remote_url)\n -+\t\t\treturn 1;\n -+\t\tif (!inc->remote_urls)\n -+\t\t\tpopulate_remote_urls(inc);\n -+\t\treturn at_least_one_url_matches_glob(cond, cond_len,\n -+\t\t\t\t\t\t inc->remote_urls);\n -+\t}\n ++\telse if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n ++\t\t\t\t &cond_len))\n ++\t\treturn include_by_remote_url(inc, cond, cond_len);\n \n \t/* unknown conditionals are always false */\n \treturn 0;\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\tgit init hasremoteurlTest &&\n +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n +\n -+\tcat >\"$(pwd)\"/include-this <<-\\EOF &&\n ++\tcat >include-this <<-\\EOF &&\n +\t[user]\n +\t\tthis = this-is-included\n +\tEOF\n -+\tcat >\"$(pwd)\"/dont-include-that <<-\\EOF &&\n ++\tcat >dont-include-that <<-\\EOF &&\n +\t[user]\n +\t\tthat = that-is-not-included\n +\tEOF\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\tgit init hasremoteurlTest &&\n +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n +\n -+\tcat >\"$(pwd)\"/include-two-three <<-\\EOF &&\n ++\tcat >include-two-three <<-\\EOF &&\n +\t[user]\n +\t\ttwo = included-config\n +\t\tthree = included-config\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\tgit init hasremoteurlTest &&\n +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n +\n -+\tprintf \"[user]\\ndss = yes\\n\" >\"$(pwd)/double-star-start\" &&\n -+\tprintf \"[user]\\ndse = yes\\n\" >\"$(pwd)/double-star-end\" &&\n -+\tprintf \"[user]\\ndsm = yes\\n\" >\"$(pwd)/double-star-middle\" &&\n -+\tprintf \"[user]\\nssm = yes\\n\" >\"$(pwd)/single-star-middle\" &&\n -+\tprintf \"[user]\\nno = no\\n\" >\"$(pwd)/no\" &&\n ++\tprintf \"[user]\\ndss = yes\\n\" >double-star-start &&\n ++\tprintf \"[user]\\ndse = yes\\n\" >double-star-end &&\n ++\tprintf \"[user]\\ndsm = yes\\n\" >double-star-middle &&\n ++\tprintf \"[user]\\nssm = yes\\n\" >single-star-middle &&\n ++\tprintf \"[user]\\nno = no\\n\" >no &&\n +\n +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n +\t[remote \"foo\"]\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\tgit init hasremoteurlTest &&\n +\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n +\n -+\tcat >\"$(pwd)\"/include-with-url <<-\\EOF &&\n ++\tcat >include-with-url <<-\\EOF &&\n +\t[remote \"bar\"]\n +\t\turl = bar\n +\tEOF\n-- \n2.34.1.400.ga245620fadb-goog\n\n"},{"id":"443385","messageId":"b2dcae03ed3016f96a5368db925fa3736c3d7c58.1638919346.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1638919346.git.jonathantanmy@google.com","subject":"[PATCH v6 1/2] config: make git_config_include() static","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-07T23:23:51Z","receivedAt":"2021-12-07T23:24:00Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"It is not used from outside the file in which it is declared.\n\nSigned-off-by: Jonathan Tan \n---\n config.c | 12 +++++++++++-\n config.h | 37 ++++---------------------------------\n 2 files changed, 15 insertions(+), 34 deletions(-)\n\ndiff --git a/config.c b/config.c\nindex 2dcbe901b6..94ad5ce913 100644\n--- a/config.c\n+++ b/config.c\n@@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n \treturn conf->u.buf.pos;\n }\n \n+struct config_include_data {\n+\tint depth;\n+\tconfig_fn_t fn;\n+\tvoid *data;\n+\tconst struct config_options *opts;\n+};\n+#define CONFIG_INCLUDE_INIT { 0 }\n+\n+static int git_config_include(const char *var, const char *value, void *data);\n+\n #define MAX_INCLUDE_DEPTH 10\n static const char include_depth_advice[] = N_(\n \"exceeded maximum include depth (%d) while including\\n\"\n@@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n-int git_config_include(const char *var, const char *value, void *data)\n+static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n \tconst char *cond, *key;\ndiff --git a/config.h b/config.h\nindex f119de0130..48a5e472ca 100644\n--- a/config.h\n+++ b/config.h\n@@ -126,6 +126,8 @@ int git_default_config(const char *, const char *, void *);\n /**\n * Read a specific file in git-config format.\n * This function takes the same callback and data parameters as `git_config`.\n+ *\n+ * Unlike git_config(), this function does not respect includes.\n */\n int git_config_from_file(config_fn_t fn, const char *, void *);\n \n@@ -158,6 +160,8 @@ void read_very_early_config(config_fn_t cb, void *data);\n * will first feed the user-wide one to the callback, and then the\n * repo-specific one; by overwriting, the higher-priority repo-specific\n * value is left at the end).\n+ *\n+ * Unlike git_config_from_file(), this function respects includes.\n */\n void git_config(config_fn_t fn, void *);\n \n@@ -338,39 +342,6 @@ const char *current_config_origin_type(void);\n const char *current_config_name(void);\n int current_config_line(void);\n \n-/**\n- * Include Directives\n- * ------------------\n- *\n- * By default, the config parser does not respect include directives.\n- * However, a caller can use the special `git_config_include` wrapper\n- * callback to support them. To do so, you simply wrap your \"real\" callback\n- * function and data pointer in a `struct config_include_data`, and pass\n- * the wrapper to the regular config-reading functions. For example:\n- *\n- * -------------------------------------------\n- * int read_file_with_include(const char *file, config_fn_t fn, void *data)\n- * {\n- * struct config_include_data inc = CONFIG_INCLUDE_INIT;\n- * inc.fn = fn;\n- * inc.data = data;\n- * return git_config_from_file(git_config_include, file, &inc);\n- * }\n- * -------------------------------------------\n- *\n- * `git_config` respects includes automatically. The lower-level\n- * `git_config_from_file` does not.\n- *\n- */\n-struct config_include_data {\n-\tint depth;\n-\tconfig_fn_t fn;\n-\tvoid *data;\n-\tconst struct config_options *opts;\n-};\n-#define CONFIG_INCLUDE_INIT { 0 }\n-int git_config_include(const char *name, const char *value, void *data);\n-\n /*\n * Match and parse a config key of the form:\n *\n-- \n2.34.1.400.ga245620fadb-goog\n\n"},{"id":"443386","messageId":"de2be06818781bbef29926c0e8e8981746242c60.1638919346.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1638919346.git.jonathantanmy@google.com","subject":"[PATCH v6 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-07T23:23:52Z","receivedAt":"2021-12-07T23:24:04Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This is a feature that supports config file inclusion conditional on\nwhether the repo has a remote with a URL that matches a glob.\n\nSimilar to my previous work on remote-suggested hooks [1], the main\nmotivation is to allow remote repo administrators to provide recommended\nconfigs in a way that can be consumed more easily (e.g. through a\npackage installable by a package manager - it could, for example,\ncontain a file to be included conditionally and a post-install script\nthat adds the include directive to the system-wide config file).\n\nIn order to do this, Git reruns the config parsing mechanism upon\nnoticing the first URL-conditional include in order to find all remote\nURLs, and these remote URLs are then used to determine if that first and\nall subsequent includes are executed. Remote URLs are not allowed to be\nconfigued in any URL-conditionally-included file.\n\n[1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n\nSigned-off-by: Jonathan Tan \n---\n Documentation/config.txt | 16 ++++++\n config.c | 121 ++++++++++++++++++++++++++++++++++++---\n config.h | 9 +++\n t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++++++\n 4 files changed, 257 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 0c0e6b859f..e0e5ca558e 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -159,6 +159,22 @@ all branches that begin with `foo/`. This is useful if your branches are\n organized hierarchically and you would like to apply a configuration to\n all the branches in that hierarchy.\n \n+`hasconfig:remote.*.url:`::\n+\tThe data that follows this keyword is taken to\n+\tbe a pattern with standard globbing wildcards and two\n+\tadditional ones, `**/` and `/**`, that can match multiple\n+\tcomponents. The first time this keyword is seen, the rest of\n+\tthe config files will be scanned for remote URLs (without\n+\tapplying any values). If there exists at least one remote URL\n+\tthat matches this pattern, the include condition is met.\n++\n+Files included by this option (directly or indirectly) are not allowed\n+to contain remote URLs.\n++\n+This keyword is designed to be forwards compatible with a naming\n+scheme that supports more variable-based include conditions, but\n+currently Git only supports the exact keyword described above.\n+\n A few more notes on matching via `gitdir` and `gitdir/i`:\n \n * Symlinks in `$GIT_DIR` are not resolved before matching.\ndiff --git a/config.c b/config.c\nindex 94ad5ce913..f17053d91b 100644\n--- a/config.c\n+++ b/config.c\n@@ -125,6 +125,12 @@ struct config_include_data {\n \tconfig_fn_t fn;\n \tvoid *data;\n \tconst struct config_options *opts;\n+\tstruct git_config_source *config_source;\n+\n+\t/*\n+\t * All remote URLs discovered when reading all config files.\n+\t */\n+\tstruct string_list *remote_urls;\n };\n #define CONFIG_INCLUDE_INIT { 0 }\n \n@@ -301,9 +307,92 @@ static int include_by_branch(const char *cond, size_t cond_len)\n \treturn ret;\n }\n \n-static int include_condition_is_true(const struct config_options *opts,\n+static int add_remote_url(const char *var, const char *value, void *data)\n+{\n+\tstruct string_list *remote_urls = data;\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tstring_list_append(remote_urls, value);\n+\treturn 0;\n+}\n+\n+static void populate_remote_urls(struct config_include_data *inc)\n+{\n+\tstruct config_options opts;\n+\n+\tstruct config_source *store_cf = cf;\n+\tstruct key_value_info *store_kvi = current_config_kvi;\n+\tenum config_scope store_scope = current_parsing_scope;\n+\n+\topts = *inc->opts;\n+\topts.unconditional_remote_url = 1;\n+\n+\tcf = NULL;\n+\tcurrent_config_kvi = NULL;\n+\tcurrent_parsing_scope = 0;\n+\n+\tinc->remote_urls = xmalloc(sizeof(*inc->remote_urls));\n+\tstring_list_init_dup(inc->remote_urls);\n+\tconfig_with_options(add_remote_url, inc->remote_urls, inc->config_source, &opts);\n+\n+\tcf = store_cf;\n+\tcurrent_config_kvi = store_kvi;\n+\tcurrent_parsing_scope = store_scope;\n+}\n+\n+static int forbid_remote_url(const char *var, const char *value, void *data)\n+{\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\"));\n+\treturn 0;\n+}\n+\n+static int at_least_one_url_matches_glob(const char *glob, int glob_len,\n+\t\t\t\t\t struct string_list *remote_urls)\n+{\n+\tstruct strbuf pattern = STRBUF_INIT;\n+\tstruct string_list_item *url_item;\n+\tint found = 0;\n+\n+\tstrbuf_add(&pattern, glob, glob_len);\n+\tfor_each_string_list_item(url_item, remote_urls) {\n+\t\tif (!wildmatch(pattern.buf, url_item->string, WM_PATHNAME)) {\n+\t\t\tfound = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tstrbuf_release(&pattern);\n+\treturn found;\n+}\n+\n+static int include_by_remote_url(struct config_include_data *inc,\n+\t\tconst char *cond, size_t cond_len)\n+{\n+\tif (inc->opts->unconditional_remote_url)\n+\t\treturn 1;\n+\tif (!inc->remote_urls)\n+\t\tpopulate_remote_urls(inc);\n+\treturn at_least_one_url_matches_glob(cond, cond_len,\n+\t\t\t\t\t inc->remote_urls);\n+}\n+\n+static int include_condition_is_true(struct config_include_data *inc,\n \t\t\t\t const char *cond, size_t cond_len)\n {\n+\tconst struct config_options *opts = inc->opts;\n \n \tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len))\n \t\treturn include_by_gitdir(opts, cond, cond_len, 0);\n@@ -311,6 +400,9 @@ static int include_condition_is_true(const struct config_options *opts,\n \t\treturn include_by_gitdir(opts, cond, cond_len, 1);\n \telse if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len))\n \t\treturn include_by_branch(cond, cond_len);\n+\telse if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n+\t\t\t\t &cond_len))\n+\t\treturn include_by_remote_url(inc, cond, cond_len);\n \n \t/* unknown conditionals are always false */\n \treturn 0;\n@@ -335,9 +427,16 @@ static int git_config_include(const char *var, const char *value, void *data)\n \t\tret = handle_path_include(value, inc);\n \n \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n-\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n-\t !strcmp(key, \"path\"))\n+\t cond && include_condition_is_true(inc, cond, cond_len) &&\n+\t !strcmp(key, \"path\")) {\n+\t\tconfig_fn_t old_fn = inc->fn;\n+\n+\t\tif (inc->opts->unconditional_remote_url)\n+\t\t\tinc->fn = forbid_remote_url;\n \t\tret = handle_path_include(value, inc);\n+\t\tif (inc->opts->unconditional_remote_url)\n+\t\t\tinc->fn = old_fn;\n+\t}\n \n \treturn ret;\n }\n@@ -1933,11 +2032,13 @@ int config_with_options(config_fn_t fn, void *data,\n \t\t\tconst struct config_options *opts)\n {\n \tstruct config_include_data inc = CONFIG_INCLUDE_INIT;\n+\tint ret;\n \n \tif (opts->respect_includes) {\n \t\tinc.fn = fn;\n \t\tinc.data = data;\n \t\tinc.opts = opts;\n+\t\tinc.config_source = config_source;\n \t\tfn = git_config_include;\n \t\tdata = &inc;\n \t}\n@@ -1950,17 +2051,23 @@ int config_with_options(config_fn_t fn, void *data,\n \t * regular lookup sequence.\n \t */\n \tif (config_source && config_source->use_stdin) {\n-\t\treturn git_config_from_stdin(fn, data);\n+\t\tret = git_config_from_stdin(fn, data);\n \t} else if (config_source && config_source->file) {\n-\t\treturn git_config_from_file(fn, config_source->file, data);\n+\t\tret = git_config_from_file(fn, config_source->file, data);\n \t} else if (config_source && config_source->blob) {\n \t\tstruct repository *repo = config_source->repo ?\n \t\t\tconfig_source->repo : the_repository;\n-\t\treturn git_config_from_blob_ref(fn, repo, config_source->blob,\n+\t\tret = git_config_from_blob_ref(fn, repo, config_source->blob,\n \t\t\t\t\t\tdata);\n+\t} else {\n+\t\tret = do_git_config_sequence(opts, fn, data);\n \t}\n \n-\treturn do_git_config_sequence(opts, fn, data);\n+\tif (inc.remote_urls) {\n+\t\tstring_list_clear(inc.remote_urls, 0);\n+\t\tFREE_AND_NULL(inc.remote_urls);\n+\t}\n+\treturn ret;\n }\n \n static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)\ndiff --git a/config.h b/config.h\nindex 48a5e472ca..ab0106d287 100644\n--- a/config.h\n+++ b/config.h\n@@ -89,6 +89,15 @@ struct config_options {\n \tunsigned int ignore_worktree : 1;\n \tunsigned int ignore_cmdline : 1;\n \tunsigned int system_gently : 1;\n+\n+\t/*\n+\t * For internal use. Include all includeif.hasremoteurl paths without\n+\t * checking if the repo has that remote URL, and when doing so, verify\n+\t * that files included in this way do not configure any remote URLs\n+\t * themselves.\n+\t */\n+\tunsigned int unconditional_remote_url : 1;\n+\n \tconst char *commondir;\n \tconst char *git_dir;\n \tconfig_parser_event_fn_t event_fn;\ndiff --git a/t/t1300-config.sh b/t/t1300-config.sh\nindex 9ff46f3b04..8310562b84 100755\n--- a/t/t1300-config.sh\n+++ b/t/t1300-config.sh\n@@ -2387,4 +2387,122 @@ test_expect_success '--get and --get-all with --fixed-value' '\n \ttest_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n '\n \n+test_expect_success 'includeIf.hasconfig:remote.*.url' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >include-this <<-\\EOF &&\n+\t[user]\n+\t\tthis = this-is-included\n+\tEOF\n+\tcat >dont-include-that <<-\\EOF &&\n+\t[user]\n+\t\tthat = that-is-not-included\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n+\t\tpath = \"$(pwd)/include-this\"\n+\t[includeIf \"hasconfig:remote.*.url:bar\"]\n+\t\tpath = \"$(pwd)/dont-include-that\"\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\tEOF\n+\n+\techo this-is-included >expect-this &&\n+\tgit -C hasremoteurlTest config --get user.this >actual-this &&\n+\ttest_cmp expect-this actual-this &&\n+\n+\ttest_must_fail git -C hasremoteurlTest config --get user.that\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url respects last-config-wins' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >include-two-three <<-\\EOF &&\n+\t[user]\n+\t\ttwo = included-config\n+\t\tthree = included-config\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\t[user]\n+\t\tone = main-config\n+\t\ttwo = main-config\n+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n+\t\tpath = \"$(pwd)/include-two-three\"\n+\t[user]\n+\t\tthree = main-config\n+\tEOF\n+\n+\techo main-config >expect-main-config &&\n+\techo included-config >expect-included-config &&\n+\n+\tgit -C hasremoteurlTest config --get user.one >actual &&\n+\ttest_cmp expect-main-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.two >actual &&\n+\ttest_cmp expect-included-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.three >actual &&\n+\ttest_cmp expect-main-config actual\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url globs' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tprintf \"[user]\\ndss = yes\\n\" >double-star-start &&\n+\tprintf \"[user]\\ndse = yes\\n\" >double-star-end &&\n+\tprintf \"[user]\\ndsm = yes\\n\" >double-star-middle &&\n+\tprintf \"[user]\\nssm = yes\\n\" >single-star-middle &&\n+\tprintf \"[user]\\nno = no\\n\" >no &&\n+\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = https://foo/bar/baz\n+\t[includeIf \"hasconfig:remote.*.url:**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-start\"\n+\t[includeIf \"hasconfig:remote.*.url:**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**\"]\n+\t\tpath = \"$(pwd)/double-star-end\"\n+\t[includeIf \"hasconfig:remote.*.url:nomatch:/**\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-middle\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https://*/bar/baz\"]\n+\t\tpath = \"$(pwd)/single-star-middle\"\n+\t[includeIf \"hasconfig:remote.*.url:https://*/baz\"]\n+\t\tpath = \"$(pwd)/no\"\n+\tEOF\n+\n+\tgit -C hasremoteurlTest config --get user.dss &&\n+\tgit -C hasremoteurlTest config --get user.dse &&\n+\tgit -C hasremoteurlTest config --get user.dsm &&\n+\tgit -C hasremoteurlTest config --get user.ssm &&\n+\ttest_must_fail git -C hasremoteurlTest config --get user.no\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url forbids remote url in such included files' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >include-with-url <<-\\EOF &&\n+\t[remote \"bar\"]\n+\t\turl = bar\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n+\t\tpath = \"$(pwd)/include-with-url\"\n+\tEOF\n+\n+\t# test with any Git command\n+\ttest_must_fail git -C hasremoteurlTest status 2>err &&\n+\tgrep \"fatal: remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\" err\n+'\n+\n test_done\n-- \n2.34.1.400.ga245620fadb-goog\n\n"},{"id":"443499","messageId":"kl6l35n2c3ay.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56696","inReplyTo":"de2be06818781bbef29926c0e8e8981746242c60.1638919346.git.jonathantanmy@google.com","subject":"Re: [PATCH v6 2/2] config: include file if remote URL matches a glob","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2021-12-08T19:19:33Z","receivedAt":"2021-12-08T19:19:36Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Jonathan Tan writes:\n\n> @@ -335,9 +427,16 @@ static int git_config_include(const char *var, const char *value, void *data)\n> \t\tret = handle_path_include(value, inc);\n> \n> \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n> -\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n> -\t !strcmp(key, \"path\"))\n> +\t cond && include_condition_is_true(inc, cond, cond_len) &&\n> +\t !strcmp(key, \"path\")) {\n> +\t\tconfig_fn_t old_fn = inc->fn;\n> +\n> +\t\tif (inc->opts->unconditional_remote_url)\n> +\t\t\tinc->fn = forbid_remote_url;\n> \t\tret = handle_path_include(value, inc);\n> +\t\tif (inc->opts->unconditional_remote_url)\n> +\t\t\tinc->fn = old_fn;\n> +\t}\n> \n> \treturn ret;\n> }\n\nMinor nit: it looks like we don't need to restore inc->fn conditionally,\nso instead of:\n\n\tif (inc->opts->unconditional_remote_url)\n\t\t\tinc->fn = old_fn;\n\nwe could just have:\n\n inc->fn = old_fn;\n\nwhich (purely as a matter of personal taste) looks a bit more consistent\nwith the unconditional assignment of:\n\n config_fn_t old_fn = inc->fn;\n\n\n\nNo comments on the rest of the patch; it looks clean and\neasy-to-understand :)\n"},{"id":"443512","messageId":"kl6lwnkean33.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56696","inReplyTo":"de2be06818781bbef29926c0e8e8981746242c60.1638919346.git.jonathantanmy@google.com","subject":"Re: [PATCH v6 2/2] config: include file if remote URL matches a glob","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2021-12-08T19:55:12Z","receivedAt":"2021-12-08T19:55:16Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Jonathan Tan writes:\n\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 0c0e6b859f..e0e5ca558e 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -159,6 +159,22 @@ all branches that begin with `foo/`. This is useful if your branches are\n> organized hierarchically and you would like to apply a configuration to\n> all the branches in that hierarchy.\n> \n> +`hasconfig:remote.*.url:`::\n> +\tThe data that follows this keyword is taken to\n> +\tbe a pattern with standard globbing wildcards and two\n> +\tadditional ones, `**/` and `/**`, that can match multiple\n> +\tcomponents. The first time this keyword is seen, the rest of\n> +\tthe config files will be scanned for remote URLs (without\n> +\tapplying any values). If there exists at least one remote URL\n> +\tthat matches this pattern, the include condition is met.\n> ++\n> +Files included by this option (directly or indirectly) are not allowed\n> +to contain remote URLs.\n\nWondering out loud.. Reading this and Ævar's comment [1], I wonder if we\nshould make it clear to users *why* we choose to forbid remote URLs.\n\nSince this series is setting a precedent for future \"hasconfig:\"\nconditions (files included by \"hasconfig:foo.*.bar\" cannot contain any\n\"foo.*.bar\" values), it would be useful to git developers to explain\n*why* we chose to do this. And if we're documenting it for ourselves,\nwe might as well write it in the public docs. That way, users would know\nthat this is more of a guardrail (because it's simpler to understand\nthis way) than a hard limitation.\n\n[1] https://lore.kernel.org/git/211207.86k0ggnvfo.gmgdl@evledraar.gmail.com\n"},{"id":"443704","messageId":"20211209221622.510369-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"kl6l35n2c3ay.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v6 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-09T22:16:22Z","receivedAt":"2021-12-09T22:16:29Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Glen Choo writes:\n> Minor nit: it looks like we don't need to restore inc->fn conditionally,\n> so instead of:\n> \n> \tif (inc->opts->unconditional_remote_url)\n> \t\t\tinc->fn = old_fn;\n> \n> we could just have:\n> \n> inc->fn = old_fn;\n> \n> which (purely as a matter of personal taste) looks a bit more consistent\n> with the unconditional assignment of:\n> \n> config_fn_t old_fn = inc->fn;\n> \n> \n> \n> No comments on the rest of the patch; it looks clean and\n> easy-to-understand :)\n\nThanks for taking a look. This is a good suggestion - I'll use it.\n"},{"id":"443709","messageId":"20211209223919.513113-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"kl6lwnkean33.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v6 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-09T22:39:19Z","receivedAt":"2021-12-09T22:39:23Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Glen Choo writes:\n> > +`hasconfig:remote.*.url:`::\n> > +\tThe data that follows this keyword is taken to\n> > +\tbe a pattern with standard globbing wildcards and two\n> > +\tadditional ones, `**/` and `/**`, that can match multiple\n> > +\tcomponents. The first time this keyword is seen, the rest of\n> > +\tthe config files will be scanned for remote URLs (without\n> > +\tapplying any values). If there exists at least one remote URL\n> > +\tthat matches this pattern, the include condition is met.\n> > ++\n> > +Files included by this option (directly or indirectly) are not allowed\n> > +to contain remote URLs.\n> \n> Wondering out loud.. Reading this and Ævar's comment [1], I wonder if we\n> should make it clear to users *why* we choose to forbid remote URLs.\n> \n> Since this series is setting a precedent for future \"hasconfig:\"\n> conditions (files included by \"hasconfig:foo.*.bar\" cannot contain any\n> \"foo.*.bar\" values), it would be useful to git developers to explain\n> *why* we chose to do this. And if we're documenting it for ourselves,\n> we might as well write it in the public docs. That way, users would know\n> that this is more of a guardrail (because it's simpler to understand\n> this way) than a hard limitation.\n> \n> [1] https://lore.kernel.org/git/211207.86k0ggnvfo.gmgdl@evledraar.gmail.com\n\nThe explanation is rather long, though. It goes something like this:\n\n If the main config is:\n\n [remote a]\n url = bar\n [includeif hasconfig:remote.*.url:foo]\n path = foo\n [includeif hasconfig:remote.*.url:bar]\n path = bar\n\n and \"bar\" contains:\n\n [remote b]\n url = foo\n\n Should \"foo\" be included? For now, we avoid these situations\n completely by prohibiting URLs from being configured in \"includeif\n hasconfig\".\n\nIf you can think of a concise explanation, maybe we can include it.\n"},{"id":"443717","messageId":"kl6l8rwt9ww2.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56696","inReplyTo":"20211209223919.513113-1-jonathantanmy@google.com","subject":"Re: [PATCH v6 2/2] config: include file if remote URL matches a glob","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2021-12-09T23:33:17Z","receivedAt":"2021-12-09T23:33:21Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Jonathan Tan writes:\n\n>> > +`hasconfig:remote.*.url:`::\n>> > +\tThe data that follows this keyword is taken to\n>> > +\tbe a pattern with standard globbing wildcards and two\n>> > +\tadditional ones, `**/` and `/**`, that can match multiple\n>> > +\tcomponents. The first time this keyword is seen, the rest of\n>> > +\tthe config files will be scanned for remote URLs (without\n>> > +\tapplying any values). If there exists at least one remote URL\n>> > +\tthat matches this pattern, the include condition is met.\n>> > ++\n>> > +Files included by this option (directly or indirectly) are not allowed\n>> > +to contain remote URLs.\n>> \n>> Wondering out loud.. Reading this and Ævar's comment [1], I wonder if we\n>> should make it clear to users *why* we choose to forbid remote URLs.\n>> \n>> Since this series is setting a precedent for future \"hasconfig:\"\n>> conditions (files included by \"hasconfig:foo.*.bar\" cannot contain any\n>> \"foo.*.bar\" values), it would be useful to git developers to explain\n>> *why* we chose to do this. And if we're documenting it for ourselves,\n>> we might as well write it in the public docs. That way, users would know\n>> that this is more of a guardrail (because it's simpler to understand\n>> this way) than a hard limitation.\n>> \n>> [1] https://lore.kernel.org/git/211207.86k0ggnvfo.gmgdl@evledraar.gmail.com\n>\n> The explanation is rather long, though. It goes something like this:\n>\n> If the main config is:\n>\n> [remote a]\n> url = bar\n> [includeif hasconfig:remote.*.url:foo]\n> path = foo\n> [includeif hasconfig:remote.*.url:bar]\n> path = bar\n>\n> and \"bar\" contains:\n>\n> [remote b]\n> url = foo\n>\n> Should \"foo\" be included? For now, we avoid these situations\n> completely by prohibiting URLs from being configured in \"includeif\n> hasconfig\".\n>\n> If you can think of a concise explanation, maybe we can include it.\n\nYeah, I can't think of a concise-yet-clear way to convey this to users\n(if I had thought of one, I wouldn't have prefaced my original comment\nwith \"Wondering out loud\").\n\nSpitballing here...\n\n `hasconfig:remote.*.url:`::\n The data that follows this keyword is taken to\n be a pattern with standard globbing wildcards and two\n additional ones, `**/` and `/**`, that can match multiple\n components. The first time this keyword is seen, the rest of\n the config files will be scanned for remote URLs (without\n applying any values). If there exists at least one remote URL\n that matches this pattern, the include condition is met.\n\n - Files included by this option (directly or indirectly) are not allowed\n - to contain remote URLs.\n + Because new remote URLs might affect the correctness of the include\n + condition, files included by this option (directly or indirectly) are\n + not allowed to contain remote URLs.\n\nAlthough, upon further reflection, I wonder if this approach of banning\nconfig variables really gives us the safety we want after all. Reworking\nyour example, say we expand \"hasconfig\" to include\n\"hasconfig:branch.*.merge\" then we can have this in the main config:\n\n [remote a]\n url = baz\n [branch c]\n merge = bar\n\n [includeif hasconfig:remote.*.url:foo]\n path = foo\n [includeif hasconfig:branch.*.merge:bar]\n path = bar\n\nand \"bar\" contains:\n\n [remote b]\n url = foo\n\nwe end up with the exact same question of \"Should \"foo\" be included?\".\nThis shows that the rule isn't actually \"files included by\nhasconfig:remote.*.url cannot include remote.*.url\", but the much more\nrestrictive \"files included by hasconfig: cannot include any\nconfig values that can appear in hasconfig\". This sounds pretty unusable\nto me..\n\nBut I think that with the semantics you've defined, we don't really need\nto forbid config variables. This section describes:\n\n The first time this keyword is seen, the rest of the config files will\n be scanned for remote URLs (without applying any values). If there\n exists at least one remote URL that matches this pattern, the include\n condition is met.\n\nwhich, to me, gives us a pass to say \"the first time we see a hasconfig,\nwe will do an additional scan without applying values\". That doesn't\nsound _too_ confusing to me, but I don't know how it looks to someone\nwith fresh eyes.\n\nForgive me if this exact suggestion came up before on-list (I know we've\ndiscussed this exact approach off-list).\n"},{"id":"443836","messageId":"xmqqmtl8m8wj.fsf@gitster.g","threadId":"56696","inReplyTo":"20211209223919.513113-1-jonathantanmy@google.com","subject":"Re: [PATCH v6 2/2] config: include file if remote URL matches a glob","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-10T21:45:16Z","receivedAt":"2021-12-10T21:45:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> The explanation is rather long, though. It goes something like this:\n>\n> If the main config is:\n>\n> [remote a]\n> url = bar\n> [includeif hasconfig:remote.*.url:foo]\n> path = foo\n> [includeif hasconfig:remote.*.url:bar]\n> path = bar\n>\n> and \"bar\" contains:\n>\n> [remote b]\n> url = foo\n>\n> Should \"foo\" be included? For now, we avoid these situations\n> completely by prohibiting URLs from being configured in \"includeif\n> hasconfig\".\n>\n> If you can think of a concise explanation, maybe we can include it.\n\nPerhaps it is easier to approach it from the viewpoint of a new\nuser who is unfamiliar with what you designed.\n\nI would imagine that most users would find it natural if a single\npass precedure read and processed lines as it sees them.\n\nThat is, when the first includeif is evaluated, we have seen only\n'remote.a.url' whose value is 'bar', so the condition does not hold.\nand then when the second includeif is evaluated, it gets included,\nand we read 'bar'. But that is wher configuration reading ends;\nremote.b.url is not asked for after we process the second includeif\ntil the end.\n\nIf you explain\n\n (1) why such a simplest design would not work well; and\n\n (2) how the actual design is different from that simplest design to\n overcome it.\n\nit would be easier to grok?\n\nThanks.\n"},{"id":"444053","messageId":"20211213233546.773716-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"kl6l8rwt9ww2.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v6 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-13T23:35:46Z","receivedAt":"2021-12-13T23:35:50Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Glen Choo writes:\n> Yeah, I can't think of a concise-yet-clear way to convey this to users\n> (if I had thought of one, I wouldn't have prefaced my original comment\n> with \"Wondering out loud\").\n> \n> Spitballing here...\n> \n> `hasconfig:remote.*.url:`::\n> The data that follows this keyword is taken to\n> be a pattern with standard globbing wildcards and two\n> additional ones, `**/` and `/**`, that can match multiple\n> components. The first time this keyword is seen, the rest of\n> the config files will be scanned for remote URLs (without\n> applying any values). If there exists at least one remote URL\n> that matches this pattern, the include condition is met.\n> \n> - Files included by this option (directly or indirectly) are not allowed\n> - to contain remote URLs.\n> + Because new remote URLs might affect the correctness of the include\n> + condition, files included by this option (directly or indirectly) are\n> + not allowed to contain remote URLs.\n\nJunio suggested another approach [1] - I'll try that and see what I come\nup with.\n\n[1] https://lore.kernel.org/git/xmqqmtl8m8wj.fsf@gitster.g/\n\n> Although, upon further reflection, I wonder if this approach of banning\n> config variables really gives us the safety we want after all. Reworking\n> your example, say we expand \"hasconfig\" to include\n> \"hasconfig:branch.*.merge\" then we can have this in the main config:\n> \n> [remote a]\n> url = baz\n> [branch c]\n> merge = bar\n> \n> [includeif hasconfig:remote.*.url:foo]\n> path = foo\n> [includeif hasconfig:branch.*.merge:bar]\n> path = bar\n> \n> and \"bar\" contains:\n> \n> [remote b]\n> url = foo\n> \n> we end up with the exact same question of \"Should \"foo\" be included?\".\n> This shows that the rule isn't actually \"files included by\n> hasconfig:remote.*.url cannot include remote.*.url\", but the much more\n> restrictive \"files included by hasconfig: cannot include any\n> config values that can appear in hasconfig\". This sounds pretty unusable\n> to me..\n\nThis was my original idea actually (using any config variable anywhere\nbans you from that config variable in all \"includeif hasconfig\"). I\nthink it would still be usable - you just have to be careful in which\nconfig variables you use. But we don't have plans to include other\nvariables now anyway.\n\n> But I think that with the semantics you've defined, we don't really need\n> to forbid config variables. This section describes:\n> \n> The first time this keyword is seen, the rest of the config files will\n> be scanned for remote URLs (without applying any values). If there\n> exists at least one remote URL that matches this pattern, the include\n> condition is met.\n> \n> which, to me, gives us a pass to say \"the first time we see a hasconfig,\n> we will do an additional scan without applying values\". That doesn't\n> sound _too_ confusing to me, but I don't know how it looks to someone\n> with fresh eyes.\n> \n> Forgive me if this exact suggestion came up before on-list (I know we've\n> discussed this exact approach off-list).\n\nThis \"additional scan without applying values\" is not very well-defined,\nthough. In the scenario I described in [2], should \"foo\" be included?\n\"Yes\" because it is referenced (even though at that time, nobody has\never head of the URL \"foo\") or \"no\" because at that point in time in the\nscan, nobody has ever heard of the URL \"foo\"?\n\n[2] https://lore.kernel.org/git/20211209223919.513113-1-jonathantanmy@google.com/\n"},{"id":"444054","messageId":"20211213233712.774546-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"xmqqmtl8m8wj.fsf@gitster.g","subject":"Re: [PATCH v6 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-13T23:37:12Z","receivedAt":"2021-12-13T23:37:17Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Junio C Hamano writes:\n> Perhaps it is easier to approach it from the viewpoint of a new\n> user who is unfamiliar with what you designed.\n> \n> I would imagine that most users would find it natural if a single\n> pass precedure read and processed lines as it sees them.\n> \n> That is, when the first includeif is evaluated, we have seen only\n> 'remote.a.url' whose value is 'bar', so the condition does not hold.\n> and then when the second includeif is evaluated, it gets included,\n> and we read 'bar'. But that is wher configuration reading ends;\n> remote.b.url is not asked for after we process the second includeif\n> til the end.\n> \n> If you explain\n> \n> (1) why such a simplest design would not work well; and\n> \n> (2) how the actual design is different from that simplest design to\n> overcome it.\n> \n> it would be easier to grok?\n> \n> Thanks.\n\nThanks - this sounds like a good approach. I'll try this.\n"},{"id":"444124","messageId":"cover.1639509048.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"[PATCH v7 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-14T21:31:48Z","receivedAt":"2021-12-14T21:31:57Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Thanks, everyone, for your comments. I've followed Glen's code\nsuggestion and Junio's documentation suggestion, as you can see in the\nrange-diff.\n\nJonathan Tan (2):\n config: make git_config_include() static\n config: include file if remote URL matches a glob\n\n Documentation/config.txt | 27 ++++++++\n config.c | 132 ++++++++++++++++++++++++++++++++++++---\n config.h | 46 ++++----------\n t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++\n 4 files changed, 282 insertions(+), 41 deletions(-)\n\nRange-diff against v6:\n1: b2dcae03ed = 1: b2dcae03ed config: make git_config_include() static\n2: de2be06818 ! 2: 7c70089074 config: include file if remote URL matches a glob\n @@ Documentation/config.txt: all branches that begin with `foo/`. This is useful if\n +Files included by this option (directly or indirectly) are not allowed\n +to contain remote URLs.\n ++\n -+This keyword is designed to be forwards compatible with a naming\n -+scheme that supports more variable-based include conditions, but\n -+currently Git only supports the exact keyword described above.\n ++Note that unlike other includeIf conditions, resolving this condition\n ++relies on information that is not yet known at the point of reading the\n ++condition. A typical use case is this option being present as a\n ++system-level or global-level config, and the remote URL being in a\n ++local-level config; hence the need to scan ahead when resolving this\n ++condition. In order to avoid the chicken-and-egg problem in which\n ++potentially-included files can affect whether such files are potentially\n ++included, Git breaks the cycle by prohibiting these files from affecting\n ++the resolution of these conditions (thus, prohibiting them from\n ++declaring remote URLs).\n +++\n ++As for the naming of this keyword, it is for forwards compatibiliy with\n ++a naming scheme that supports more variable-based include conditions,\n ++but currently Git only supports the exact keyword described above.\n +\n A few more notes on matching via `gitdir` and `gitdir/i`:\n \n @@ config.c: static int git_config_include(const char *var, const char *value, void\n +\t\tif (inc->opts->unconditional_remote_url)\n +\t\t\tinc->fn = forbid_remote_url;\n \t\tret = handle_path_include(value, inc);\n -+\t\tif (inc->opts->unconditional_remote_url)\n -+\t\t\tinc->fn = old_fn;\n ++\t\tinc->fn = old_fn;\n +\t}\n \n \treturn ret;\n-- \n2.34.1.173.g76aa8bc2d0-goog\n\n"},{"id":"444125","messageId":"b2dcae03ed3016f96a5368db925fa3736c3d7c58.1639509048.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1639509048.git.jonathantanmy@google.com","subject":"[PATCH v7 1/2] config: make git_config_include() static","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-14T21:31:49Z","receivedAt":"2021-12-14T21:31:59Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"It is not used from outside the file in which it is declared.\n\nSigned-off-by: Jonathan Tan \n---\n config.c | 12 +++++++++++-\n config.h | 37 ++++---------------------------------\n 2 files changed, 15 insertions(+), 34 deletions(-)\n\ndiff --git a/config.c b/config.c\nindex 2dcbe901b6..94ad5ce913 100644\n--- a/config.c\n+++ b/config.c\n@@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n \treturn conf->u.buf.pos;\n }\n \n+struct config_include_data {\n+\tint depth;\n+\tconfig_fn_t fn;\n+\tvoid *data;\n+\tconst struct config_options *opts;\n+};\n+#define CONFIG_INCLUDE_INIT { 0 }\n+\n+static int git_config_include(const char *var, const char *value, void *data);\n+\n #define MAX_INCLUDE_DEPTH 10\n static const char include_depth_advice[] = N_(\n \"exceeded maximum include depth (%d) while including\\n\"\n@@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n-int git_config_include(const char *var, const char *value, void *data)\n+static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n \tconst char *cond, *key;\ndiff --git a/config.h b/config.h\nindex f119de0130..48a5e472ca 100644\n--- a/config.h\n+++ b/config.h\n@@ -126,6 +126,8 @@ int git_default_config(const char *, const char *, void *);\n /**\n * Read a specific file in git-config format.\n * This function takes the same callback and data parameters as `git_config`.\n+ *\n+ * Unlike git_config(), this function does not respect includes.\n */\n int git_config_from_file(config_fn_t fn, const char *, void *);\n \n@@ -158,6 +160,8 @@ void read_very_early_config(config_fn_t cb, void *data);\n * will first feed the user-wide one to the callback, and then the\n * repo-specific one; by overwriting, the higher-priority repo-specific\n * value is left at the end).\n+ *\n+ * Unlike git_config_from_file(), this function respects includes.\n */\n void git_config(config_fn_t fn, void *);\n \n@@ -338,39 +342,6 @@ const char *current_config_origin_type(void);\n const char *current_config_name(void);\n int current_config_line(void);\n \n-/**\n- * Include Directives\n- * ------------------\n- *\n- * By default, the config parser does not respect include directives.\n- * However, a caller can use the special `git_config_include` wrapper\n- * callback to support them. To do so, you simply wrap your \"real\" callback\n- * function and data pointer in a `struct config_include_data`, and pass\n- * the wrapper to the regular config-reading functions. For example:\n- *\n- * -------------------------------------------\n- * int read_file_with_include(const char *file, config_fn_t fn, void *data)\n- * {\n- * struct config_include_data inc = CONFIG_INCLUDE_INIT;\n- * inc.fn = fn;\n- * inc.data = data;\n- * return git_config_from_file(git_config_include, file, &inc);\n- * }\n- * -------------------------------------------\n- *\n- * `git_config` respects includes automatically. The lower-level\n- * `git_config_from_file` does not.\n- *\n- */\n-struct config_include_data {\n-\tint depth;\n-\tconfig_fn_t fn;\n-\tvoid *data;\n-\tconst struct config_options *opts;\n-};\n-#define CONFIG_INCLUDE_INIT { 0 }\n-int git_config_include(const char *name, const char *value, void *data);\n-\n /*\n * Match and parse a config key of the form:\n *\n-- \n2.34.1.173.g76aa8bc2d0-goog\n\n"},{"id":"444126","messageId":"7c700890741116b1b20936784d22d5a9f3c081e8.1639509048.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1639509048.git.jonathantanmy@google.com","subject":"[PATCH v7 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-12-14T21:31:50Z","receivedAt":"2021-12-14T21:32:02Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This is a feature that supports config file inclusion conditional on\nwhether the repo has a remote with a URL that matches a glob.\n\nSimilar to my previous work on remote-suggested hooks [1], the main\nmotivation is to allow remote repo administrators to provide recommended\nconfigs in a way that can be consumed more easily (e.g. through a\npackage installable by a package manager - it could, for example,\ncontain a file to be included conditionally and a post-install script\nthat adds the include directive to the system-wide config file).\n\nIn order to do this, Git reruns the config parsing mechanism upon\nnoticing the first URL-conditional include in order to find all remote\nURLs, and these remote URLs are then used to determine if that first and\nall subsequent includes are executed. Remote URLs are not allowed to be\nconfigued in any URL-conditionally-included file.\n\n[1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n\nSigned-off-by: Jonathan Tan \n---\n Documentation/config.txt | 27 +++++++++\n config.c | 120 ++++++++++++++++++++++++++++++++++++---\n config.h | 9 +++\n t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++++++\n 4 files changed, 267 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 0c0e6b859f..9b3480779e 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -159,6 +159,33 @@ all branches that begin with `foo/`. This is useful if your branches are\n organized hierarchically and you would like to apply a configuration to\n all the branches in that hierarchy.\n \n+`hasconfig:remote.*.url:`::\n+\tThe data that follows this keyword is taken to\n+\tbe a pattern with standard globbing wildcards and two\n+\tadditional ones, `**/` and `/**`, that can match multiple\n+\tcomponents. The first time this keyword is seen, the rest of\n+\tthe config files will be scanned for remote URLs (without\n+\tapplying any values). If there exists at least one remote URL\n+\tthat matches this pattern, the include condition is met.\n++\n+Files included by this option (directly or indirectly) are not allowed\n+to contain remote URLs.\n++\n+Note that unlike other includeIf conditions, resolving this condition\n+relies on information that is not yet known at the point of reading the\n+condition. A typical use case is this option being present as a\n+system-level or global-level config, and the remote URL being in a\n+local-level config; hence the need to scan ahead when resolving this\n+condition. In order to avoid the chicken-and-egg problem in which\n+potentially-included files can affect whether such files are potentially\n+included, Git breaks the cycle by prohibiting these files from affecting\n+the resolution of these conditions (thus, prohibiting them from\n+declaring remote URLs).\n++\n+As for the naming of this keyword, it is for forwards compatibiliy with\n+a naming scheme that supports more variable-based include conditions,\n+but currently Git only supports the exact keyword described above.\n+\n A few more notes on matching via `gitdir` and `gitdir/i`:\n \n * Symlinks in `$GIT_DIR` are not resolved before matching.\ndiff --git a/config.c b/config.c\nindex 94ad5ce913..ac4534ecf2 100644\n--- a/config.c\n+++ b/config.c\n@@ -125,6 +125,12 @@ struct config_include_data {\n \tconfig_fn_t fn;\n \tvoid *data;\n \tconst struct config_options *opts;\n+\tstruct git_config_source *config_source;\n+\n+\t/*\n+\t * All remote URLs discovered when reading all config files.\n+\t */\n+\tstruct string_list *remote_urls;\n };\n #define CONFIG_INCLUDE_INIT { 0 }\n \n@@ -301,9 +307,92 @@ static int include_by_branch(const char *cond, size_t cond_len)\n \treturn ret;\n }\n \n-static int include_condition_is_true(const struct config_options *opts,\n+static int add_remote_url(const char *var, const char *value, void *data)\n+{\n+\tstruct string_list *remote_urls = data;\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tstring_list_append(remote_urls, value);\n+\treturn 0;\n+}\n+\n+static void populate_remote_urls(struct config_include_data *inc)\n+{\n+\tstruct config_options opts;\n+\n+\tstruct config_source *store_cf = cf;\n+\tstruct key_value_info *store_kvi = current_config_kvi;\n+\tenum config_scope store_scope = current_parsing_scope;\n+\n+\topts = *inc->opts;\n+\topts.unconditional_remote_url = 1;\n+\n+\tcf = NULL;\n+\tcurrent_config_kvi = NULL;\n+\tcurrent_parsing_scope = 0;\n+\n+\tinc->remote_urls = xmalloc(sizeof(*inc->remote_urls));\n+\tstring_list_init_dup(inc->remote_urls);\n+\tconfig_with_options(add_remote_url, inc->remote_urls, inc->config_source, &opts);\n+\n+\tcf = store_cf;\n+\tcurrent_config_kvi = store_kvi;\n+\tcurrent_parsing_scope = store_scope;\n+}\n+\n+static int forbid_remote_url(const char *var, const char *value, void *data)\n+{\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\"));\n+\treturn 0;\n+}\n+\n+static int at_least_one_url_matches_glob(const char *glob, int glob_len,\n+\t\t\t\t\t struct string_list *remote_urls)\n+{\n+\tstruct strbuf pattern = STRBUF_INIT;\n+\tstruct string_list_item *url_item;\n+\tint found = 0;\n+\n+\tstrbuf_add(&pattern, glob, glob_len);\n+\tfor_each_string_list_item(url_item, remote_urls) {\n+\t\tif (!wildmatch(pattern.buf, url_item->string, WM_PATHNAME)) {\n+\t\t\tfound = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tstrbuf_release(&pattern);\n+\treturn found;\n+}\n+\n+static int include_by_remote_url(struct config_include_data *inc,\n+\t\tconst char *cond, size_t cond_len)\n+{\n+\tif (inc->opts->unconditional_remote_url)\n+\t\treturn 1;\n+\tif (!inc->remote_urls)\n+\t\tpopulate_remote_urls(inc);\n+\treturn at_least_one_url_matches_glob(cond, cond_len,\n+\t\t\t\t\t inc->remote_urls);\n+}\n+\n+static int include_condition_is_true(struct config_include_data *inc,\n \t\t\t\t const char *cond, size_t cond_len)\n {\n+\tconst struct config_options *opts = inc->opts;\n \n \tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len))\n \t\treturn include_by_gitdir(opts, cond, cond_len, 0);\n@@ -311,6 +400,9 @@ static int include_condition_is_true(const struct config_options *opts,\n \t\treturn include_by_gitdir(opts, cond, cond_len, 1);\n \telse if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len))\n \t\treturn include_by_branch(cond, cond_len);\n+\telse if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n+\t\t\t\t &cond_len))\n+\t\treturn include_by_remote_url(inc, cond, cond_len);\n \n \t/* unknown conditionals are always false */\n \treturn 0;\n@@ -335,9 +427,15 @@ static int git_config_include(const char *var, const char *value, void *data)\n \t\tret = handle_path_include(value, inc);\n \n \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n-\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n-\t !strcmp(key, \"path\"))\n+\t cond && include_condition_is_true(inc, cond, cond_len) &&\n+\t !strcmp(key, \"path\")) {\n+\t\tconfig_fn_t old_fn = inc->fn;\n+\n+\t\tif (inc->opts->unconditional_remote_url)\n+\t\t\tinc->fn = forbid_remote_url;\n \t\tret = handle_path_include(value, inc);\n+\t\tinc->fn = old_fn;\n+\t}\n \n \treturn ret;\n }\n@@ -1933,11 +2031,13 @@ int config_with_options(config_fn_t fn, void *data,\n \t\t\tconst struct config_options *opts)\n {\n \tstruct config_include_data inc = CONFIG_INCLUDE_INIT;\n+\tint ret;\n \n \tif (opts->respect_includes) {\n \t\tinc.fn = fn;\n \t\tinc.data = data;\n \t\tinc.opts = opts;\n+\t\tinc.config_source = config_source;\n \t\tfn = git_config_include;\n \t\tdata = &inc;\n \t}\n@@ -1950,17 +2050,23 @@ int config_with_options(config_fn_t fn, void *data,\n \t * regular lookup sequence.\n \t */\n \tif (config_source && config_source->use_stdin) {\n-\t\treturn git_config_from_stdin(fn, data);\n+\t\tret = git_config_from_stdin(fn, data);\n \t} else if (config_source && config_source->file) {\n-\t\treturn git_config_from_file(fn, config_source->file, data);\n+\t\tret = git_config_from_file(fn, config_source->file, data);\n \t} else if (config_source && config_source->blob) {\n \t\tstruct repository *repo = config_source->repo ?\n \t\t\tconfig_source->repo : the_repository;\n-\t\treturn git_config_from_blob_ref(fn, repo, config_source->blob,\n+\t\tret = git_config_from_blob_ref(fn, repo, config_source->blob,\n \t\t\t\t\t\tdata);\n+\t} else {\n+\t\tret = do_git_config_sequence(opts, fn, data);\n \t}\n \n-\treturn do_git_config_sequence(opts, fn, data);\n+\tif (inc.remote_urls) {\n+\t\tstring_list_clear(inc.remote_urls, 0);\n+\t\tFREE_AND_NULL(inc.remote_urls);\n+\t}\n+\treturn ret;\n }\n \n static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)\ndiff --git a/config.h b/config.h\nindex 48a5e472ca..ab0106d287 100644\n--- a/config.h\n+++ b/config.h\n@@ -89,6 +89,15 @@ struct config_options {\n \tunsigned int ignore_worktree : 1;\n \tunsigned int ignore_cmdline : 1;\n \tunsigned int system_gently : 1;\n+\n+\t/*\n+\t * For internal use. Include all includeif.hasremoteurl paths without\n+\t * checking if the repo has that remote URL, and when doing so, verify\n+\t * that files included in this way do not configure any remote URLs\n+\t * themselves.\n+\t */\n+\tunsigned int unconditional_remote_url : 1;\n+\n \tconst char *commondir;\n \tconst char *git_dir;\n \tconfig_parser_event_fn_t event_fn;\ndiff --git a/t/t1300-config.sh b/t/t1300-config.sh\nindex 9ff46f3b04..8310562b84 100755\n--- a/t/t1300-config.sh\n+++ b/t/t1300-config.sh\n@@ -2387,4 +2387,122 @@ test_expect_success '--get and --get-all with --fixed-value' '\n \ttest_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n '\n \n+test_expect_success 'includeIf.hasconfig:remote.*.url' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >include-this <<-\\EOF &&\n+\t[user]\n+\t\tthis = this-is-included\n+\tEOF\n+\tcat >dont-include-that <<-\\EOF &&\n+\t[user]\n+\t\tthat = that-is-not-included\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n+\t\tpath = \"$(pwd)/include-this\"\n+\t[includeIf \"hasconfig:remote.*.url:bar\"]\n+\t\tpath = \"$(pwd)/dont-include-that\"\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\tEOF\n+\n+\techo this-is-included >expect-this &&\n+\tgit -C hasremoteurlTest config --get user.this >actual-this &&\n+\ttest_cmp expect-this actual-this &&\n+\n+\ttest_must_fail git -C hasremoteurlTest config --get user.that\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url respects last-config-wins' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >include-two-three <<-\\EOF &&\n+\t[user]\n+\t\ttwo = included-config\n+\t\tthree = included-config\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = foo\n+\t[user]\n+\t\tone = main-config\n+\t\ttwo = main-config\n+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n+\t\tpath = \"$(pwd)/include-two-three\"\n+\t[user]\n+\t\tthree = main-config\n+\tEOF\n+\n+\techo main-config >expect-main-config &&\n+\techo included-config >expect-included-config &&\n+\n+\tgit -C hasremoteurlTest config --get user.one >actual &&\n+\ttest_cmp expect-main-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.two >actual &&\n+\ttest_cmp expect-included-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.three >actual &&\n+\ttest_cmp expect-main-config actual\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url globs' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tprintf \"[user]\\ndss = yes\\n\" >double-star-start &&\n+\tprintf \"[user]\\ndse = yes\\n\" >double-star-end &&\n+\tprintf \"[user]\\ndsm = yes\\n\" >double-star-middle &&\n+\tprintf \"[user]\\nssm = yes\\n\" >single-star-middle &&\n+\tprintf \"[user]\\nno = no\\n\" >no &&\n+\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = https://foo/bar/baz\n+\t[includeIf \"hasconfig:remote.*.url:**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-start\"\n+\t[includeIf \"hasconfig:remote.*.url:**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**\"]\n+\t\tpath = \"$(pwd)/double-star-end\"\n+\t[includeIf \"hasconfig:remote.*.url:nomatch:/**\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-middle\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https://*/bar/baz\"]\n+\t\tpath = \"$(pwd)/single-star-middle\"\n+\t[includeIf \"hasconfig:remote.*.url:https://*/baz\"]\n+\t\tpath = \"$(pwd)/no\"\n+\tEOF\n+\n+\tgit -C hasremoteurlTest config --get user.dss &&\n+\tgit -C hasremoteurlTest config --get user.dse &&\n+\tgit -C hasremoteurlTest config --get user.dsm &&\n+\tgit -C hasremoteurlTest config --get user.ssm &&\n+\ttest_must_fail git -C hasremoteurlTest config --get user.no\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url forbids remote url in such included files' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >include-with-url <<-\\EOF &&\n+\t[remote \"bar\"]\n+\t\turl = bar\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n+\t\tpath = \"$(pwd)/include-with-url\"\n+\tEOF\n+\n+\t# test with any Git command\n+\ttest_must_fail git -C hasremoteurlTest status 2>err &&\n+\tgrep \"fatal: remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\" err\n+'\n+\n test_done\n-- \n2.34.1.173.g76aa8bc2d0-goog\n\n"},{"id":"444266","messageId":"kl6lilvo8bco.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56696","inReplyTo":"7c700890741116b1b20936784d22d5a9f3c081e8.1639509048.git.jonathantanmy@google.com","subject":"Re: [PATCH v7 2/2] config: include file if remote URL matches a glob","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2021-12-16T21:54:15Z","receivedAt":"2021-12-16T21:54:18Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Jonathan Tan writes:\n\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 0c0e6b859f..9b3480779e 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -159,6 +159,33 @@ all branches that begin with `foo/`. This is useful if your branches are\n> organized hierarchically and you would like to apply a configuration to\n> all the branches in that hierarchy.\n> \n> +`hasconfig:remote.*.url:`::\n> +\tThe data that follows this keyword is taken to\n> +\tbe a pattern with standard globbing wildcards and two\n> +\tadditional ones, `**/` and `/**`, that can match multiple\n> +\tcomponents. The first time this keyword is seen, the rest of\n> +\tthe config files will be scanned for remote URLs (without\n> +\tapplying any values). If there exists at least one remote URL\n> +\tthat matches this pattern, the include condition is met.\n> ++\n> +Files included by this option (directly or indirectly) are not allowed\n> +to contain remote URLs.\n> ++\n> +Note that unlike other includeIf conditions, resolving this condition\n> +relies on information that is not yet known at the point of reading the\n> +condition. A typical use case is this option being present as a\n> +system-level or global-level config, and the remote URL being in a\n> +local-level config; hence the need to scan ahead when resolving this\n> +condition. In order to avoid the chicken-and-egg problem in which\n> +potentially-included files can affect whether such files are potentially\n> +included, Git breaks the cycle by prohibiting these files from affecting\n> +the resolution of these conditions (thus, prohibiting them from\n> +declaring remote URLs).\n\nPutting myself in the shoes of someone who is unfamiliar with the\nimplementation, I think that this becomes clear if you read it enough\ntimes (but also, I'm not a good reader), so this is ok.\n\nIt would be nice for this to be reviewed by someone who is _actually_\nunfamiliar, though.\n"},{"id":"444267","messageId":"kl6lfsqs8b6o.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56696","inReplyTo":"cover.1639509048.git.jonathantanmy@google.com","subject":"Re: [PATCH v7 0/2] Conditional config includes based on remote URL","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2021-12-16T21:57:51Z","receivedAt":"2021-12-16T21:57:55Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Jonathan Tan writes:\n\n> Thanks, everyone, for your comments. I've followed Glen's code\n> suggestion and Junio's documentation suggestion, as you can see in the\n> range-diff.\n>\n> Jonathan Tan (2):\n> config: make git_config_include() static\n> config: include file if remote URL matches a glob\n>\n> Documentation/config.txt | 27 ++++++++\n> config.c | 132 ++++++++++++++++++++++++++++++++++++---\n> config.h | 46 ++++----------\n> t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++\n> 4 files changed, 282 insertions(+), 41 deletions(-)\n>\n> Range-diff against v6:\n> 1: b2dcae03ed = 1: b2dcae03ed config: make git_config_include() static\n> 2: de2be06818 ! 2: 7c70089074 config: include file if remote URL matches a glob\n> @@ Documentation/config.txt: all branches that begin with `foo/`. This is useful if\n> +Files included by this option (directly or indirectly) are not allowed\n> +to contain remote URLs.\n> ++\n> -+This keyword is designed to be forwards compatible with a naming\n> -+scheme that supports more variable-based include conditions, but\n> -+currently Git only supports the exact keyword described above.\n> ++Note that unlike other includeIf conditions, resolving this condition\n> ++relies on information that is not yet known at the point of reading the\n> ++condition. A typical use case is this option being present as a\n> ++system-level or global-level config, and the remote URL being in a\n> ++local-level config; hence the need to scan ahead when resolving this\n> ++condition. In order to avoid the chicken-and-egg problem in which\n> ++potentially-included files can affect whether such files are potentially\n> ++included, Git breaks the cycle by prohibiting these files from affecting\n> ++the resolution of these conditions (thus, prohibiting them from\n> ++declaring remote URLs).\n> +++\n> ++As for the naming of this keyword, it is for forwards compatibiliy with\n> ++a naming scheme that supports more variable-based include conditions,\n> ++but currently Git only supports the exact keyword described above.\n> +\n> A few more notes on matching via `gitdir` and `gitdir/i`:\n> \n> @@ config.c: static int git_config_include(const char *var, const char *value, void\n> +\t\tif (inc->opts->unconditional_remote_url)\n> +\t\t\tinc->fn = forbid_remote_url;\n> \t\tret = handle_path_include(value, inc);\n> -+\t\tif (inc->opts->unconditional_remote_url)\n> -+\t\t\tinc->fn = old_fn;\n> ++\t\tinc->fn = old_fn;\n> +\t}\n> \n> \treturn ret;\n> -- \n> 2.34.1.173.g76aa8bc2d0-goog\n\nThe implementation looks good, and I think that the precedent we are\nsetting with \"hasconfig:\" is pretty well captured on this thread. \n\nThis looks good to me, though I'm not an expert in this area, so it\nwould be good for others to chime in.\n\nReviewed-by: Glen Choo \n"},{"id":"445038","messageId":"CABPp-BF+_33zR-vwh-tMTG=vxuah5jyuJVXssa-vAH51qdsrDQ@mail.gmail.com","threadId":"56696","inReplyTo":"7c700890741116b1b20936784d22d5a9f3c081e8.1639509048.git.jonathantanmy@google.com","subject":"Re: [PATCH v7 2/2] config: include file if remote URL matches a glob","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-12-28T00:55:53Z","receivedAt":"2021-12-28T00:56:09Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Dec 15, 2021 at 7:01 AM Jonathan Tan wrote:\n>\n> This is a feature that supports config file inclusion conditional on\n> whether the repo has a remote with a URL that matches a glob.\n>\n> Similar to my previous work on remote-suggested hooks [1], the main\n> motivation is to allow remote repo administrators to provide recommended\n> configs in a way that can be consumed more easily (e.g. through a\n> package installable by a package manager - it could, for example,\n> contain a file to be included conditionally and a post-install script\n> that adds the include directive to the system-wide config file).\n>\n> In order to do this, Git reruns the config parsing mechanism upon\n> noticing the first URL-conditional include in order to find all remote\n> URLs, and these remote URLs are then used to determine if that first and\n> all subsequent includes are executed. Remote URLs are not allowed to be\n> configued in any URL-conditionally-included file.\n>\n> [1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n>\n> Signed-off-by: Jonathan Tan \n> ---\n> Documentation/config.txt | 27 +++++++++\n> config.c | 120 ++++++++++++++++++++++++++++++++++++---\n> config.h | 9 +++\n> t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++++++\n> 4 files changed, 267 insertions(+), 7 deletions(-)\n>\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 0c0e6b859f..9b3480779e 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -159,6 +159,33 @@ all branches that begin with `foo/`. This is useful if your branches are\n> organized hierarchically and you would like to apply a configuration to\n> all the branches in that hierarchy.\n>\n> +`hasconfig:remote.*.url:`::\n> + The data that follows this keyword is taken to\n> + be a pattern with standard globbing wildcards and two\n> + additional ones, `**/` and `/**`, that can match multiple\n> + components. The first time this keyword is seen, the rest of\n> + the config files will be scanned for remote URLs (without\n> + applying any values). If there exists at least one remote URL\n> + that matches this pattern, the include condition is met.\n> ++\n> +Files included by this option (directly or indirectly) are not allowed\n> +to contain remote URLs.\n> ++\n> +Note that unlike other includeIf conditions, resolving this condition\n> +relies on information that is not yet known at the point of reading the\n> +condition. A typical use case is this option being present as a\n> +system-level or global-level config, and the remote URL being in a\n> +local-level config; hence the need to scan ahead when resolving this\n> +condition. In order to avoid the chicken-and-egg problem in which\n> +potentially-included files can affect whether such files are potentially\n> +included, Git breaks the cycle by prohibiting these files from affecting\n> +the resolution of these conditions (thus, prohibiting them from\n> +declaring remote URLs).\n> ++\n> +As for the naming of this keyword, it is for forwards compatibiliy with\n> +a naming scheme that supports more variable-based include conditions,\n> +but currently Git only supports the exact keyword described above.\n> +\n> A few more notes on matching via `gitdir` and `gitdir/i`:\n>\n> * Symlinks in `$GIT_DIR` are not resolved before matching.\n> diff --git a/config.c b/config.c\n> index 94ad5ce913..ac4534ecf2 100644\n> --- a/config.c\n> +++ b/config.c\n> @@ -125,6 +125,12 @@ struct config_include_data {\n> config_fn_t fn;\n> void *data;\n> const struct config_options *opts;\n> + struct git_config_source *config_source;\n> +\n> + /*\n> + * All remote URLs discovered when reading all config files.\n> + */\n> + struct string_list *remote_urls;\n> };\n> #define CONFIG_INCLUDE_INIT { 0 }\n>\n> @@ -301,9 +307,92 @@ static int include_by_branch(const char *cond, size_t cond_len)\n> return ret;\n> }\n>\n> -static int include_condition_is_true(const struct config_options *opts,\n> +static int add_remote_url(const char *var, const char *value, void *data)\n> +{\n> + struct string_list *remote_urls = data;\n> + const char *remote_name;\n> + size_t remote_name_len;\n> + const char *key;\n> +\n> + if (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n> + &key) &&\n> + remote_name &&\n> + !strcmp(key, \"url\"))\n> + string_list_append(remote_urls, value);\n> + return 0;\n> +}\n> +\n> +static void populate_remote_urls(struct config_include_data *inc)\n> +{\n> + struct config_options opts;\n> +\n> + struct config_source *store_cf = cf;\n> + struct key_value_info *store_kvi = current_config_kvi;\n> + enum config_scope store_scope = current_parsing_scope;\n> +\n> + opts = *inc->opts;\n> + opts.unconditional_remote_url = 1;\n> +\n> + cf = NULL;\n> + current_config_kvi = NULL;\n> + current_parsing_scope = 0;\n> +\n> + inc->remote_urls = xmalloc(sizeof(*inc->remote_urls));\n> + string_list_init_dup(inc->remote_urls);\n> + config_with_options(add_remote_url, inc->remote_urls, inc->config_source, &opts);\n> +\n> + cf = store_cf;\n> + current_config_kvi = store_kvi;\n> + current_parsing_scope = store_scope;\n> +}\n> +\n> +static int forbid_remote_url(const char *var, const char *value, void *data)\n> +{\n> + const char *remote_name;\n> + size_t remote_name_len;\n> + const char *key;\n> +\n> + if (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n> + &key) &&\n> + remote_name &&\n> + !strcmp(key, \"url\"))\n> + die(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\"));\n> + return 0;\n> +}\n> +\n> +static int at_least_one_url_matches_glob(const char *glob, int glob_len,\n> + struct string_list *remote_urls)\n> +{\n> + struct strbuf pattern = STRBUF_INIT;\n> + struct string_list_item *url_item;\n> + int found = 0;\n> +\n> + strbuf_add(&pattern, glob, glob_len);\n> + for_each_string_list_item(url_item, remote_urls) {\n> + if (!wildmatch(pattern.buf, url_item->string, WM_PATHNAME)) {\n> + found = 1;\n> + break;\n> + }\n> + }\n> + strbuf_release(&pattern);\n> + return found;\n> +}\n> +\n> +static int include_by_remote_url(struct config_include_data *inc,\n> + const char *cond, size_t cond_len)\n> +{\n> + if (inc->opts->unconditional_remote_url)\n> + return 1;\n> + if (!inc->remote_urls)\n> + populate_remote_urls(inc);\n> + return at_least_one_url_matches_glob(cond, cond_len,\n> + inc->remote_urls);\n> +}\n> +\n> +static int include_condition_is_true(struct config_include_data *inc,\n> const char *cond, size_t cond_len)\n> {\n> + const struct config_options *opts = inc->opts;\n>\n> if (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len))\n> return include_by_gitdir(opts, cond, cond_len, 0);\n> @@ -311,6 +400,9 @@ static int include_condition_is_true(const struct config_options *opts,\n> return include_by_gitdir(opts, cond, cond_len, 1);\n> else if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len))\n> return include_by_branch(cond, cond_len);\n> + else if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n> + &cond_len))\n> + return include_by_remote_url(inc, cond, cond_len);\n>\n> /* unknown conditionals are always false */\n> return 0;\n> @@ -335,9 +427,15 @@ static int git_config_include(const char *var, const char *value, void *data)\n> ret = handle_path_include(value, inc);\n>\n> if (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n> - (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n> - !strcmp(key, \"path\"))\n> + cond && include_condition_is_true(inc, cond, cond_len) &&\n> + !strcmp(key, \"path\")) {\n> + config_fn_t old_fn = inc->fn;\n> +\n> + if (inc->opts->unconditional_remote_url)\n> + inc->fn = forbid_remote_url;\n> ret = handle_path_include(value, inc);\n> + inc->fn = old_fn;\n> + }\n>\n> return ret;\n> }\n> @@ -1933,11 +2031,13 @@ int config_with_options(config_fn_t fn, void *data,\n> const struct config_options *opts)\n> {\n> struct config_include_data inc = CONFIG_INCLUDE_INIT;\n> + int ret;\n>\n> if (opts->respect_includes) {\n> inc.fn = fn;\n> inc.data = data;\n> inc.opts = opts;\n> + inc.config_source = config_source;\n> fn = git_config_include;\n> data = &inc;\n> }\n> @@ -1950,17 +2050,23 @@ int config_with_options(config_fn_t fn, void *data,\n> * regular lookup sequence.\n> */\n> if (config_source && config_source->use_stdin) {\n> - return git_config_from_stdin(fn, data);\n> + ret = git_config_from_stdin(fn, data);\n> } else if (config_source && config_source->file) {\n> - return git_config_from_file(fn, config_source->file, data);\n> + ret = git_config_from_file(fn, config_source->file, data);\n> } else if (config_source && config_source->blob) {\n> struct repository *repo = config_source->repo ?\n> config_source->repo : the_repository;\n> - return git_config_from_blob_ref(fn, repo, config_source->blob,\n> + ret = git_config_from_blob_ref(fn, repo, config_source->blob,\n> data);\n> + } else {\n> + ret = do_git_config_sequence(opts, fn, data);\n> }\n>\n> - return do_git_config_sequence(opts, fn, data);\n> + if (inc.remote_urls) {\n> + string_list_clear(inc.remote_urls, 0);\n> + FREE_AND_NULL(inc.remote_urls);\n> + }\n> + return ret;\n> }\n>\n> static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)\n> diff --git a/config.h b/config.h\n> index 48a5e472ca..ab0106d287 100644\n> --- a/config.h\n> +++ b/config.h\n> @@ -89,6 +89,15 @@ struct config_options {\n> unsigned int ignore_worktree : 1;\n> unsigned int ignore_cmdline : 1;\n> unsigned int system_gently : 1;\n> +\n> + /*\n> + * For internal use. Include all includeif.hasremoteurl paths without\n> + * checking if the repo has that remote URL, and when doing so, verify\n> + * that files included in this way do not configure any remote URLs\n> + * themselves.\n> + */\n> + unsigned int unconditional_remote_url : 1;\n> +\n> const char *commondir;\n> const char *git_dir;\n> config_parser_event_fn_t event_fn;\n> diff --git a/t/t1300-config.sh b/t/t1300-config.sh\n> index 9ff46f3b04..8310562b84 100755\n> --- a/t/t1300-config.sh\n> +++ b/t/t1300-config.sh\n> @@ -2387,4 +2387,122 @@ test_expect_success '--get and --get-all with --fixed-value' '\n> test_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n> '\n>\n> +test_expect_success 'includeIf.hasconfig:remote.*.url' '\n> + git init hasremoteurlTest &&\n> + test_when_finished \"rm -rf hasremoteurlTest\" &&\n> +\n> + cat >include-this <<-\\EOF &&\n> + [user]\n> + this = this-is-included\n> + EOF\n> + cat >dont-include-that <<-\\EOF &&\n> + [user]\n> + that = that-is-not-included\n> + EOF\n> + cat >>hasremoteurlTest/.git/config <<-EOF &&\n> + [includeIf \"hasconfig:remote.*.url:foo\"]\n> + path = \"$(pwd)/include-this\"\n> + [includeIf \"hasconfig:remote.*.url:bar\"]\n> + path = \"$(pwd)/dont-include-that\"\n> + [remote \"foo\"]\n> + url = foo\n\nWhich \"foo\" is relevant here? The remote name, or the url value?\nCould they be given different values so that the testcase is a bit\neasier to read and understand?\n\n> + EOF\n> +\n> + echo this-is-included >expect-this &&\n> + git -C hasremoteurlTest config --get user.this >actual-this &&\n> + test_cmp expect-this actual-this &&\n> +\n> + test_must_fail git -C hasremoteurlTest config --get user.that\n> +'\n> +\n> +test_expect_success 'includeIf.hasconfig:remote.*.url respects last-config-wins' '\n> + git init hasremoteurlTest &&\n> + test_when_finished \"rm -rf hasremoteurlTest\" &&\n> +\n> + cat >include-two-three <<-\\EOF &&\n> + [user]\n> + two = included-config\n> + three = included-config\n> + EOF\n> + cat >>hasremoteurlTest/.git/config <<-EOF &&\n> + [remote \"foo\"]\n> + url = foo\n\n...similarly here.\n\n> + [user]\n> + one = main-config\n> + two = main-config\n> + [includeIf \"hasconfig:remote.*.url:foo\"]\n> + path = \"$(pwd)/include-two-three\"\n> + [user]\n> + three = main-config\n> + EOF\n> +\n> + echo main-config >expect-main-config &&\n> + echo included-config >expect-included-config &&\n> +\n> + git -C hasremoteurlTest config --get user.one >actual &&\n> + test_cmp expect-main-config actual &&\n> +\n> + git -C hasremoteurlTest config --get user.two >actual &&\n> + test_cmp expect-included-config actual &&\n> +\n> + git -C hasremoteurlTest config --get user.three >actual &&\n> + test_cmp expect-main-config actual\n> +'\n> +\n> +test_expect_success 'includeIf.hasconfig:remote.*.url globs' '\n> + git init hasremoteurlTest &&\n> + test_when_finished \"rm -rf hasremoteurlTest\" &&\n> +\n> + printf \"[user]\\ndss = yes\\n\" >double-star-start &&\n> + printf \"[user]\\ndse = yes\\n\" >double-star-end &&\n> + printf \"[user]\\ndsm = yes\\n\" >double-star-middle &&\n> + printf \"[user]\\nssm = yes\\n\" >single-star-middle &&\n> + printf \"[user]\\nno = no\\n\" >no &&\n> +\n> + cat >>hasremoteurlTest/.git/config <<-EOF &&\n> + [remote \"foo\"]\n> + url = https://foo/bar/baz\n\nThis example is nicer as the below matches make it clearer that it's\nabout the url value.\n\n> + [includeIf \"hasconfig:remote.*.url:**/baz\"]\n> + path = \"$(pwd)/double-star-start\"\n> + [includeIf \"hasconfig:remote.*.url:**/nomatch\"]\n> + path = \"$(pwd)/no\"\n> + [includeIf \"hasconfig:remote.*.url:https:/**\"]\n> + path = \"$(pwd)/double-star-end\"\n> + [includeIf \"hasconfig:remote.*.url:nomatch:/**\"]\n> + path = \"$(pwd)/no\"\n> + [includeIf \"hasconfig:remote.*.url:https:/**/baz\"]\n> + path = \"$(pwd)/double-star-middle\"\n> + [includeIf \"hasconfig:remote.*.url:https:/**/nomatch\"]\n> + path = \"$(pwd)/no\"\n> + [includeIf \"hasconfig:remote.*.url:https://*/bar/baz\"]\n> + path = \"$(pwd)/single-star-middle\"\n> + [includeIf \"hasconfig:remote.*.url:https://*/baz\"]\n> + path = \"$(pwd)/no\"\n> + EOF\n> +\n> + git -C hasremoteurlTest config --get user.dss &&\n> + git -C hasremoteurlTest config --get user.dse &&\n> + git -C hasremoteurlTest config --get user.dsm &&\n> + git -C hasremoteurlTest config --get user.ssm &&\n> + test_must_fail git -C hasremoteurlTest config --get user.no\n> +'\n> +\n> +test_expect_success 'includeIf.hasconfig:remote.*.url forbids remote url in such included files' '\n> + git init hasremoteurlTest &&\n> + test_when_finished \"rm -rf hasremoteurlTest\" &&\n> +\n> + cat >include-with-url <<-\\EOF &&\n> + [remote \"bar\"]\n> + url = bar\n> + EOF\n> + cat >>hasremoteurlTest/.git/config <<-EOF &&\n> + [includeIf \"hasconfig:remote.*.url:foo\"]\n> + path = \"$(pwd)/include-with-url\"\n> + EOF\n> +\n> + # test with any Git command\n> + test_must_fail git -C hasremoteurlTest status 2>err &&\n> + grep \"fatal: remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\" err\n> +'\n> +\n> test_done\n> --\n> 2.34.1.173.g76aa8bc2d0-goog\n\nThe testcases are very helpful. I found myself confused when reading\njust the documentation about how it would be used. Perhaps an example\nor two should be added to the documentation?\n"},{"id":"445050","messageId":"CABPp-BFLNqLuJ8o_6YZDYgd=Ft+wc9EjBPX+RRzwAdASKSW2bw@mail.gmail.com","threadId":"56696","inReplyTo":"cover.1639509048.git.jonathantanmy@google.com","subject":"Re: [PATCH v7 0/2] Conditional config includes based on remote URL","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-12-28T01:13:39Z","receivedAt":"2021-12-28T01:13:54Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Dec 15, 2021 at 7:25 AM Jonathan Tan wrote:\n>\n> Thanks, everyone, for your comments. I've followed Glen's code\n> suggestion and Junio's documentation suggestion, as you can see in the\n> range-diff.\n\nSo, the basic idea is, in a setting like Google's, you can have users\ninstall additional files on their system out-of-band, and have the\nusers specify a simple line in their configuration to make use of\nthose additional files -- or portions thereof. It's a way of easily\nproviding potentially large blocks of pre-vetted configuration for\nusers.\n\nSeems to make sense. (and I've read over the code lightly, so feel\nfree to take this as an Acked-by.)\n\n\nBut can I back up and comment on a bigger picture item?\n\nThis mechanism requires somehow getting additional files to the user\nseparately; projects that span companies (git.git, linux.git, etc.)\nwon't likely be able to make use of this.\n\nScalar also has a mechanism for providing potentially large blocks of\npre-vetted configuration for users. It does so as part of a new\ntop-level command. And it does so with a very opinionated set of\nvalues that are not configurable. Thus, while I'd like to use it,\nthey use a configuration option that would break things badly at my\n$DAYJOB. (Too many gradle plugins using jgit, which doesn't\nunderstand index.version=4 and will blow up with a very suboptimal\nerror message when they see it.) And, it's very specific to scalar;\nwe probably don't want to add a new toplevel command everytime someone\nwants common configuration to be easily grabbed by some user.\n\nIt would be nice if we could find some more generic solution.\nGranted, I can't think of any, and I don't think this comment should\nblock this particular series (nor the scalar one), but I am worrying a\nlittle bit that we're getting multiple completely different solutions\nfor the same general problem, and each brings caveats big enough to\npreclude many (most?) potential users. I don't know what to do about\nthat, especially since configuration that is too easy to propagate\ncomes with big security problems, but I wanted to at least raise the\nissue and hope others have good ideas. If nothing else, I want to\nraise awareness to avoid proliferation of similar\npre-vetted-configuration-deployment mechanisms. I'm CC'ing a couple\nscalar folks as well for that point.\n"},{"id":"445150","messageId":"kl6lee5w5nng.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56696","inReplyTo":"CABPp-BFLNqLuJ8o_6YZDYgd=Ft+wc9EjBPX+RRzwAdASKSW2bw@mail.gmail.com","subject":"Re: [PATCH v7 0/2] Conditional config includes based on remote URL","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2021-12-28T23:13:07Z","receivedAt":"2021-12-28T23:13:21Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Elijah Newren writes:\n\n> But can I back up and comment on a bigger picture item?\n>\n> This mechanism requires somehow getting additional files to the user\n> separately; projects that span companies (git.git, linux.git, etc.)\n> won't likely be able to make use of this.\n>\n> Scalar also has a mechanism for providing potentially large blocks of\n> pre-vetted configuration for users. It does so as part of a new\n> top-level command. And it does so with a very opinionated set of\n> values that are not configurable. Thus, while I'd like to use it,\n> they use a configuration option that would break things badly at my\n> $DAYJOB. (Too many gradle plugins using jgit, which doesn't\n> understand index.version=4 and will blow up with a very suboptimal\n> error message when they see it.) And, it's very specific to scalar;\n> we probably don't want to add a new toplevel command everytime someone\n> wants common configuration to be easily grabbed by some user.\n>\n> It would be nice if we could find some more generic solution.\n> Granted, I can't think of any, and I don't think this comment should\n> block this particular series (nor the scalar one), but I am worrying a\n> little bit that we're getting multiple completely different solutions\n> for the same general problem, and each brings caveats big enough to\n> preclude many (most?) potential users. I don't know what to do about\n> that, especially since configuration that is too easy to propagate\n> comes with big security problems, but I wanted to at least raise the\n> issue and hope others have good ideas. If nothing else, I want to\n> raise awareness to avoid proliferation of similar\n> pre-vetted-configuration-deployment mechanisms. I'm CC'ing a couple\n> scalar folks as well for that point.\n\nYes, that's an accurate description. To reiterate what Jonathan said in\nhis first cover letter [1], the primary motivation is that we want to be\nable to 'suggest' hooks to users. There was an RFC for this\n'remote-suggested hooks feature' (docs [2], RFC implementation [3]) but\nit ultimately stalled due to security concerns I believe (this was\nbefore I joined the team, so I'm not the most familiar with this).\n\nIt might be worth re-reading those threads since they tread on pretty\nmuch the same ground of shipping pre-vetted config (this is directed at\nme too, since I haven't read through those in detail). I've also been\ntold that we're (aka Google) still looking for feedback on [2], so feel\nfree to share any thoughts on that thread too.\n\n[1] https://lore.kernel.org/git/cover.1634077795.git.jonathantanmy@google.com\n[2] https://lore.kernel.org/git/pull.908.v4.git.1620241892929.gitgitgadget@gmail.com/\n[3] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n"},{"id":"445858","messageId":"20220110185830.1115343-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"CABPp-BF+_33zR-vwh-tMTG=vxuah5jyuJVXssa-vAH51qdsrDQ@mail.gmail.com","subject":"Re: [PATCH v7 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2022-01-10T18:58:30Z","receivedAt":"2022-01-10T18:58:36Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Elijah Newren writes:\n> > +test_expect_success 'includeIf.hasconfig:remote.*.url' '\n> > + git init hasremoteurlTest &&\n> > + test_when_finished \"rm -rf hasremoteurlTest\" &&\n> > +\n> > + cat >include-this <<-\\EOF &&\n> > + [user]\n> > + this = this-is-included\n> > + EOF\n> > + cat >dont-include-that <<-\\EOF &&\n> > + [user]\n> > + that = that-is-not-included\n> > + EOF\n> > + cat >>hasremoteurlTest/.git/config <<-EOF &&\n> > + [includeIf \"hasconfig:remote.*.url:foo\"]\n> > + path = \"$(pwd)/include-this\"\n> > + [includeIf \"hasconfig:remote.*.url:bar\"]\n> > + path = \"$(pwd)/dont-include-that\"\n> > + [remote \"foo\"]\n> > + url = foo\n> \n> Which \"foo\" is relevant here? The remote name, or the url value?\n> Could they be given different values so that the testcase is a bit\n> easier to read and understand?\n\nThanks for taking a look. Sorry for the late reply - I just got back\nfrom vacation.\n\nThis is a good point - I'll change one of them.\n\n> > +test_expect_success 'includeIf.hasconfig:remote.*.url respects last-config-wins' '\n> > + git init hasremoteurlTest &&\n> > + test_when_finished \"rm -rf hasremoteurlTest\" &&\n> > +\n> > + cat >include-two-three <<-\\EOF &&\n> > + [user]\n> > + two = included-config\n> > + three = included-config\n> > + EOF\n> > + cat >>hasremoteurlTest/.git/config <<-EOF &&\n> > + [remote \"foo\"]\n> > + url = foo\n> \n> ...similarly here.\n\nNoted.\n\n> The testcases are very helpful. I found myself confused when reading\n> just the documentation about how it would be used. Perhaps an example\n> or two should be added to the documentation?\n\nWill do. I notice that there is a section with examples - I'll add it\nthere.\n"},{"id":"445868","messageId":"20220110192246.1124991-1-jonathantanmy@google.com","threadId":"56696","inReplyTo":"CABPp-BFLNqLuJ8o_6YZDYgd=Ft+wc9EjBPX+RRzwAdASKSW2bw@mail.gmail.com","subject":"Re: [PATCH v7 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2022-01-10T19:22:46Z","receivedAt":"2022-01-10T19:22:52Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Elijah Newren writes:\n> On Wed, Dec 15, 2021 at 7:25 AM Jonathan Tan wrote:\n> >\n> > Thanks, everyone, for your comments. I've followed Glen's code\n> > suggestion and Junio's documentation suggestion, as you can see in the\n> > range-diff.\n> \n> So, the basic idea is, in a setting like Google's, you can have users\n> install additional files on their system out-of-band, and have the\n> users specify a simple line in their configuration to make use of\n> those additional files -- or portions thereof. It's a way of easily\n> providing potentially large blocks of pre-vetted configuration for\n> users.\n> \n> Seems to make sense. (and I've read over the code lightly, so feel\n> free to take this as an Acked-by.)\n\nThanks.\n\n> But can I back up and comment on a bigger picture item?\n> \n> This mechanism requires somehow getting additional files to the user\n> separately; projects that span companies (git.git, linux.git, etc.)\n> won't likely be able to make use of this.\n\nYes, they would also need to use a separate mechanism in addition to\nGit.\n\n> Scalar also has a mechanism for providing potentially large blocks of\n> pre-vetted configuration for users. It does so as part of a new\n> top-level command. And it does so with a very opinionated set of\n> values that are not configurable. Thus, while I'd like to use it,\n> they use a configuration option that would break things badly at my\n> $DAYJOB. (Too many gradle plugins using jgit, which doesn't\n> understand index.version=4 and will blow up with a very suboptimal\n> error message when they see it.) And, it's very specific to scalar;\n> we probably don't want to add a new toplevel command everytime someone\n> wants common configuration to be easily grabbed by some user.\n\nDo you have more information on this? The closest thing I've seen is\n\"Scalar Config\" under \"Modifying Configuration Values\" in [1], which\nseems to be more about bundling additional tools (which may change\nconfig, of course).\n\nUnless you're referring to the config bundled in the Scalar tool itself,\nin which case this patch set seems orthogonal and potentially\ncomplementary - I was envisioning config being provided by a package\nmanager package, but Scalar could provide some too for users to use at\ntheir own discretion.\n\n[1] https://github.com/microsoft/git/blob/7a514b4c2d5df7fdd2f66f048010d8ddcb412d0b/contrib/scalar/docs/troubleshooting.md\n\n> It would be nice if we could find some more generic solution.\n> Granted, I can't think of any, and I don't think this comment should\n> block this particular series (nor the scalar one), but I am worrying a\n> little bit that we're getting multiple completely different solutions\n> for the same general problem, and each brings caveats big enough to\n> preclude many (most?) potential users. I don't know what to do about\n> that, especially since configuration that is too easy to propagate\n> comes with big security problems, but I wanted to at least raise the\n> issue and hope others have good ideas. If nothing else, I want to\n> raise awareness to avoid proliferation of similar\n> pre-vetted-configuration-deployment mechanisms. I'm CC'ing a couple\n> scalar folks as well for that point.\n\nThat's a good point. As Glen said [2], it seems like transmitting config\nitself (or, at least, hooks) through Git is something that we (the Git\nproject) don't want to do, so I have been working from the basis that\nGit should just make use of config/hooks delivered through a non-Git\nmechanism, and not deliver the config/hooks itself.\n\n[2] https://lore.kernel.org/git/kl6lee5w5nng.fsf@chooglen-macbookpro.roam.corp.google.com/\n"},{"id":"445873","messageId":"CABPp-BGtA1KwV8kdN52r5=XTCZrS_e3sPYfTxMnoEVyVV8Hc_w@mail.gmail.com","threadId":"56696","inReplyTo":"20220110192246.1124991-1-jonathantanmy@google.com","subject":"Re: [PATCH v7 0/2] Conditional config includes based on remote URL","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-01-10T20:17:02Z","receivedAt":"2022-01-10T20:17:16Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Jan 10, 2022 at 11:22 AM Jonathan Tan wrote:\n>\n> Elijah Newren writes:\n> > On Wed, Dec 15, 2021 at 7:25 AM Jonathan Tan wrote:\n> > >\n> > > Thanks, everyone, for your comments. I've followed Glen's code\n> > > suggestion and Junio's documentation suggestion, as you can see in the\n> > > range-diff.\n> >\n> > So, the basic idea is, in a setting like Google's, you can have users\n> > install additional files on their system out-of-band, and have the\n> > users specify a simple line in their configuration to make use of\n> > those additional files -- or portions thereof. It's a way of easily\n> > providing potentially large blocks of pre-vetted configuration for\n> > users.\n> >\n> > Seems to make sense. (and I've read over the code lightly, so feel\n> > free to take this as an Acked-by.)\n>\n> Thanks.\n>\n> > But can I back up and comment on a bigger picture item?\n> >\n> > This mechanism requires somehow getting additional files to the user\n> > separately; projects that span companies (git.git, linux.git, etc.)\n> > won't likely be able to make use of this.\n>\n> Yes, they would also need to use a separate mechanism in addition to\n> Git.\n>\n> > Scalar also has a mechanism for providing potentially large blocks of\n> > pre-vetted configuration for users. It does so as part of a new\n> > top-level command. And it does so with a very opinionated set of\n> > values that are not configurable. Thus, while I'd like to use it,\n> > they use a configuration option that would break things badly at my\n> > $DAYJOB. (Too many gradle plugins using jgit, which doesn't\n> > understand index.version=4 and will blow up with a very suboptimal\n> > error message when they see it.) And, it's very specific to scalar;\n> > we probably don't want to add a new toplevel command everytime someone\n> > wants common configuration to be easily grabbed by some user.\n>\n> Do you have more information on this? The closest thing I've seen is\n> \"Scalar Config\" under \"Modifying Configuration Values\" in [1], which\n> seems to be more about bundling additional tools (which may change\n> config, of course).\n>\n> Unless you're referring to the config bundled in the Scalar tool itself,\n> in which case this patch set seems orthogonal and potentially\n> complementary - I was envisioning config being provided by a package\n> manager package, but Scalar could provide some too for users to use at\n> their own discretion.\n>\n> [1] https://github.com/microsoft/git/blob/7a514b4c2d5df7fdd2f66f048010d8ddcb412d0b/contrib/scalar/docs/troubleshooting.md\n\nYes, I was referring to the config hardcoded in the Scalar tool itself\n(see set_recommended_config() in\nhttps://lore.kernel.org/git/4439ab4de0bc3f48a6bdcf4b5165b16fad792ebd.1638538470.git.gitgitgadget@gmail.com/).\n\nI agree they are different solutions to \"help others setup config in a\npre-vetted way\", that the two don't seem to conflict, and one can't be\nimplemented in terms of the other. It might even be possible for\nsomeone somewhere to simultaneously take advantage of both (not sure\nif anyone would try, but I don't forsee problems in doing so, except\nin the narrow case that both schemes try to set the same config and\nthere are worries about which one \"wins\", which might boil down to\nwhether the include directive came first in the config file or the\nspecific config value that scalar set).\n\n> > It would be nice if we could find some more generic solution.\n> > Granted, I can't think of any, and I don't think this comment should\n> > block this particular series (nor the scalar one), but I am worrying a\n> > little bit that we're getting multiple completely different solutions\n> > for the same general problem, and each brings caveats big enough to\n> > preclude many (most?) potential users. I don't know what to do about\n> > that, especially since configuration that is too easy to propagate\n> > comes with big security problems, but I wanted to at least raise the\n> > issue and hope others have good ideas. If nothing else, I want to\n> > raise awareness to avoid proliferation of similar\n> > pre-vetted-configuration-deployment mechanisms. I'm CC'ing a couple\n> > scalar folks as well for that point.\n>\n> That's a good point. As Glen said [2], it seems like transmitting config\n> itself (or, at least, hooks) through Git is something that we (the Git\n> project) don't want to do, so I have been working from the basis that\n> Git should just make use of config/hooks delivered through a non-Git\n> mechanism, and not deliver the config/hooks itself.\n>\n> [2] https://lore.kernel.org/git/kl6lee5w5nng.fsf@chooglen-macbookpro.roam.corp.google.com/\n\nYeah, makes sense. And I don't know any better solutions. I guess\nall I'm really saying is that if a third narrowly targetted way to\nprovide pre-vetted configuration shows up on the list, it may be time\nto ask folks to step back and try to find a more generic solution.\n"},{"id":"446434","messageId":"cover.1642527965.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1634077795.git.jonathantanmy@google.com","subject":"[PATCH v8 0/2] Conditional config includes based on remote URL","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2022-01-18T17:47:38Z","receivedAt":"2022-01-18T17:47:46Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Thanks everyone for your review. v8 follows Elijah Newren's comments\nabout making URLs more identifiable in tests, and also including an\nexample in documentation.\n\nJonathan Tan (2):\n config: make git_config_include() static\n config: include file if remote URL matches a glob\n\n Documentation/config.txt | 35 +++++++++++\n config.c | 132 ++++++++++++++++++++++++++++++++++++---\n config.h | 46 ++++----------\n t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++\n 4 files changed, 290 insertions(+), 41 deletions(-)\n\nRange-diff against v7:\n1: b2dcae03ed = 1: b2dcae03ed config: make git_config_include() static\n2: 7c70089074 ! 2: 6691e39c82 config: include file if remote URL matches a glob\n @@ Documentation/config.txt: all branches that begin with `foo/`. This is useful if\n A few more notes on matching via `gitdir` and `gitdir/i`:\n \n * Symlinks in `$GIT_DIR` are not resolved before matching.\n +@@ Documentation/config.txt: Example\n + ; currently checked out\n + [includeIf \"onbranch:foo-branch\"]\n + \tpath = foo.inc\n ++\n ++; include only if a remote with the given URL exists (note\n ++; that such a URL may be provided later in a file or in a\n ++; file read after this file is read, as seen in this example)\n ++[includeIf \"hasconfig:remote.*.url:https://example.com/**\"]\n ++\tpath = foo.inc\n ++[remote \"origin\"]\n ++\turl = https://example.com/git\n + ----\n + \n + Values\n \n ## config.c ##\n @@ config.c: struct config_include_data {\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\t\tthat = that-is-not-included\n +\tEOF\n +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n -+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n ++\t[includeIf \"hasconfig:remote.*.url:foourl\"]\n +\t\tpath = \"$(pwd)/include-this\"\n -+\t[includeIf \"hasconfig:remote.*.url:bar\"]\n ++\t[includeIf \"hasconfig:remote.*.url:barurl\"]\n +\t\tpath = \"$(pwd)/dont-include-that\"\n +\t[remote \"foo\"]\n -+\t\turl = foo\n ++\t\turl = foourl\n +\tEOF\n +\n +\techo this-is-included >expect-this &&\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\tEOF\n +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n +\t[remote \"foo\"]\n -+\t\turl = foo\n ++\t\turl = foourl\n +\t[user]\n +\t\tone = main-config\n +\t\ttwo = main-config\n -+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n ++\t[includeIf \"hasconfig:remote.*.url:foourl\"]\n +\t\tpath = \"$(pwd)/include-two-three\"\n +\t[user]\n +\t\tthree = main-config\n @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n +\n +\tcat >include-with-url <<-\\EOF &&\n +\t[remote \"bar\"]\n -+\t\turl = bar\n ++\t\turl = barurl\n +\tEOF\n +\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n -+\t[includeIf \"hasconfig:remote.*.url:foo\"]\n ++\t[includeIf \"hasconfig:remote.*.url:foourl\"]\n +\t\tpath = \"$(pwd)/include-with-url\"\n +\tEOF\n +\n-- \n2.34.1.703.g22d0c6ccf7-goog\n\n"},{"id":"446435","messageId":"b2dcae03ed3016f96a5368db925fa3736c3d7c58.1642527965.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1642527965.git.jonathantanmy@google.com","subject":"[PATCH v8 1/2] config: make git_config_include() static","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2022-01-18T17:47:39Z","receivedAt":"2022-01-18T17:47:48Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"It is not used from outside the file in which it is declared.\n\nSigned-off-by: Jonathan Tan \n---\n config.c | 12 +++++++++++-\n config.h | 37 ++++---------------------------------\n 2 files changed, 15 insertions(+), 34 deletions(-)\n\ndiff --git a/config.c b/config.c\nindex 2dcbe901b6..94ad5ce913 100644\n--- a/config.c\n+++ b/config.c\n@@ -120,6 +120,16 @@ static long config_buf_ftell(struct config_source *conf)\n \treturn conf->u.buf.pos;\n }\n \n+struct config_include_data {\n+\tint depth;\n+\tconfig_fn_t fn;\n+\tvoid *data;\n+\tconst struct config_options *opts;\n+};\n+#define CONFIG_INCLUDE_INIT { 0 }\n+\n+static int git_config_include(const char *var, const char *value, void *data);\n+\n #define MAX_INCLUDE_DEPTH 10\n static const char include_depth_advice[] = N_(\n \"exceeded maximum include depth (%d) while including\\n\"\n@@ -306,7 +316,7 @@ static int include_condition_is_true(const struct config_options *opts,\n \treturn 0;\n }\n \n-int git_config_include(const char *var, const char *value, void *data)\n+static int git_config_include(const char *var, const char *value, void *data)\n {\n \tstruct config_include_data *inc = data;\n \tconst char *cond, *key;\ndiff --git a/config.h b/config.h\nindex f119de0130..48a5e472ca 100644\n--- a/config.h\n+++ b/config.h\n@@ -126,6 +126,8 @@ int git_default_config(const char *, const char *, void *);\n /**\n * Read a specific file in git-config format.\n * This function takes the same callback and data parameters as `git_config`.\n+ *\n+ * Unlike git_config(), this function does not respect includes.\n */\n int git_config_from_file(config_fn_t fn, const char *, void *);\n \n@@ -158,6 +160,8 @@ void read_very_early_config(config_fn_t cb, void *data);\n * will first feed the user-wide one to the callback, and then the\n * repo-specific one; by overwriting, the higher-priority repo-specific\n * value is left at the end).\n+ *\n+ * Unlike git_config_from_file(), this function respects includes.\n */\n void git_config(config_fn_t fn, void *);\n \n@@ -338,39 +342,6 @@ const char *current_config_origin_type(void);\n const char *current_config_name(void);\n int current_config_line(void);\n \n-/**\n- * Include Directives\n- * ------------------\n- *\n- * By default, the config parser does not respect include directives.\n- * However, a caller can use the special `git_config_include` wrapper\n- * callback to support them. To do so, you simply wrap your \"real\" callback\n- * function and data pointer in a `struct config_include_data`, and pass\n- * the wrapper to the regular config-reading functions. For example:\n- *\n- * -------------------------------------------\n- * int read_file_with_include(const char *file, config_fn_t fn, void *data)\n- * {\n- * struct config_include_data inc = CONFIG_INCLUDE_INIT;\n- * inc.fn = fn;\n- * inc.data = data;\n- * return git_config_from_file(git_config_include, file, &inc);\n- * }\n- * -------------------------------------------\n- *\n- * `git_config` respects includes automatically. The lower-level\n- * `git_config_from_file` does not.\n- *\n- */\n-struct config_include_data {\n-\tint depth;\n-\tconfig_fn_t fn;\n-\tvoid *data;\n-\tconst struct config_options *opts;\n-};\n-#define CONFIG_INCLUDE_INIT { 0 }\n-int git_config_include(const char *name, const char *value, void *data);\n-\n /*\n * Match and parse a config key of the form:\n *\n-- \n2.34.1.703.g22d0c6ccf7-goog\n\n"},{"id":"446436","messageId":"6691e39c822ae2d12d0d6fbc4b1f69cf10fb7a54.1642527965.git.jonathantanmy@google.com","threadId":"56696","inReplyTo":"cover.1642527965.git.jonathantanmy@google.com","subject":"[PATCH v8 2/2] config: include file if remote URL matches a glob","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2022-01-18T17:47:40Z","receivedAt":"2022-01-18T17:47:50Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This is a feature that supports config file inclusion conditional on\nwhether the repo has a remote with a URL that matches a glob.\n\nSimilar to my previous work on remote-suggested hooks [1], the main\nmotivation is to allow remote repo administrators to provide recommended\nconfigs in a way that can be consumed more easily (e.g. through a\npackage installable by a package manager - it could, for example,\ncontain a file to be included conditionally and a post-install script\nthat adds the include directive to the system-wide config file).\n\nIn order to do this, Git reruns the config parsing mechanism upon\nnoticing the first URL-conditional include in order to find all remote\nURLs, and these remote URLs are then used to determine if that first and\nall subsequent includes are executed. Remote URLs are not allowed to be\nconfigued in any URL-conditionally-included file.\n\n[1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/\n\nSigned-off-by: Jonathan Tan \n---\n Documentation/config.txt | 35 ++++++++++++\n config.c | 120 ++++++++++++++++++++++++++++++++++++---\n config.h | 9 +++\n t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++++++\n 4 files changed, 275 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 0c0e6b859f..5a5205952e 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -159,6 +159,33 @@ all branches that begin with `foo/`. This is useful if your branches are\n organized hierarchically and you would like to apply a configuration to\n all the branches in that hierarchy.\n \n+`hasconfig:remote.*.url:`::\n+\tThe data that follows this keyword is taken to\n+\tbe a pattern with standard globbing wildcards and two\n+\tadditional ones, `**/` and `/**`, that can match multiple\n+\tcomponents. The first time this keyword is seen, the rest of\n+\tthe config files will be scanned for remote URLs (without\n+\tapplying any values). If there exists at least one remote URL\n+\tthat matches this pattern, the include condition is met.\n++\n+Files included by this option (directly or indirectly) are not allowed\n+to contain remote URLs.\n++\n+Note that unlike other includeIf conditions, resolving this condition\n+relies on information that is not yet known at the point of reading the\n+condition. A typical use case is this option being present as a\n+system-level or global-level config, and the remote URL being in a\n+local-level config; hence the need to scan ahead when resolving this\n+condition. In order to avoid the chicken-and-egg problem in which\n+potentially-included files can affect whether such files are potentially\n+included, Git breaks the cycle by prohibiting these files from affecting\n+the resolution of these conditions (thus, prohibiting them from\n+declaring remote URLs).\n++\n+As for the naming of this keyword, it is for forwards compatibiliy with\n+a naming scheme that supports more variable-based include conditions,\n+but currently Git only supports the exact keyword described above.\n+\n A few more notes on matching via `gitdir` and `gitdir/i`:\n \n * Symlinks in `$GIT_DIR` are not resolved before matching.\n@@ -226,6 +253,14 @@ Example\n ; currently checked out\n [includeIf \"onbranch:foo-branch\"]\n \tpath = foo.inc\n+\n+; include only if a remote with the given URL exists (note\n+; that such a URL may be provided later in a file or in a\n+; file read after this file is read, as seen in this example)\n+[includeIf \"hasconfig:remote.*.url:https://example.com/**\"]\n+\tpath = foo.inc\n+[remote \"origin\"]\n+\turl = https://example.com/git\n ----\n \n Values\ndiff --git a/config.c b/config.c\nindex 94ad5ce913..ac4534ecf2 100644\n--- a/config.c\n+++ b/config.c\n@@ -125,6 +125,12 @@ struct config_include_data {\n \tconfig_fn_t fn;\n \tvoid *data;\n \tconst struct config_options *opts;\n+\tstruct git_config_source *config_source;\n+\n+\t/*\n+\t * All remote URLs discovered when reading all config files.\n+\t */\n+\tstruct string_list *remote_urls;\n };\n #define CONFIG_INCLUDE_INIT { 0 }\n \n@@ -301,9 +307,92 @@ static int include_by_branch(const char *cond, size_t cond_len)\n \treturn ret;\n }\n \n-static int include_condition_is_true(const struct config_options *opts,\n+static int add_remote_url(const char *var, const char *value, void *data)\n+{\n+\tstruct string_list *remote_urls = data;\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tstring_list_append(remote_urls, value);\n+\treturn 0;\n+}\n+\n+static void populate_remote_urls(struct config_include_data *inc)\n+{\n+\tstruct config_options opts;\n+\n+\tstruct config_source *store_cf = cf;\n+\tstruct key_value_info *store_kvi = current_config_kvi;\n+\tenum config_scope store_scope = current_parsing_scope;\n+\n+\topts = *inc->opts;\n+\topts.unconditional_remote_url = 1;\n+\n+\tcf = NULL;\n+\tcurrent_config_kvi = NULL;\n+\tcurrent_parsing_scope = 0;\n+\n+\tinc->remote_urls = xmalloc(sizeof(*inc->remote_urls));\n+\tstring_list_init_dup(inc->remote_urls);\n+\tconfig_with_options(add_remote_url, inc->remote_urls, inc->config_source, &opts);\n+\n+\tcf = store_cf;\n+\tcurrent_config_kvi = store_kvi;\n+\tcurrent_parsing_scope = store_scope;\n+}\n+\n+static int forbid_remote_url(const char *var, const char *value, void *data)\n+{\n+\tconst char *remote_name;\n+\tsize_t remote_name_len;\n+\tconst char *key;\n+\n+\tif (!parse_config_key(var, \"remote\", &remote_name, &remote_name_len,\n+\t\t\t &key) &&\n+\t remote_name &&\n+\t !strcmp(key, \"url\"))\n+\t\tdie(_(\"remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\"));\n+\treturn 0;\n+}\n+\n+static int at_least_one_url_matches_glob(const char *glob, int glob_len,\n+\t\t\t\t\t struct string_list *remote_urls)\n+{\n+\tstruct strbuf pattern = STRBUF_INIT;\n+\tstruct string_list_item *url_item;\n+\tint found = 0;\n+\n+\tstrbuf_add(&pattern, glob, glob_len);\n+\tfor_each_string_list_item(url_item, remote_urls) {\n+\t\tif (!wildmatch(pattern.buf, url_item->string, WM_PATHNAME)) {\n+\t\t\tfound = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tstrbuf_release(&pattern);\n+\treturn found;\n+}\n+\n+static int include_by_remote_url(struct config_include_data *inc,\n+\t\tconst char *cond, size_t cond_len)\n+{\n+\tif (inc->opts->unconditional_remote_url)\n+\t\treturn 1;\n+\tif (!inc->remote_urls)\n+\t\tpopulate_remote_urls(inc);\n+\treturn at_least_one_url_matches_glob(cond, cond_len,\n+\t\t\t\t\t inc->remote_urls);\n+}\n+\n+static int include_condition_is_true(struct config_include_data *inc,\n \t\t\t\t const char *cond, size_t cond_len)\n {\n+\tconst struct config_options *opts = inc->opts;\n \n \tif (skip_prefix_mem(cond, cond_len, \"gitdir:\", &cond, &cond_len))\n \t\treturn include_by_gitdir(opts, cond, cond_len, 0);\n@@ -311,6 +400,9 @@ static int include_condition_is_true(const struct config_options *opts,\n \t\treturn include_by_gitdir(opts, cond, cond_len, 1);\n \telse if (skip_prefix_mem(cond, cond_len, \"onbranch:\", &cond, &cond_len))\n \t\treturn include_by_branch(cond, cond_len);\n+\telse if (skip_prefix_mem(cond, cond_len, \"hasconfig:remote.*.url:\", &cond,\n+\t\t\t\t &cond_len))\n+\t\treturn include_by_remote_url(inc, cond, cond_len);\n \n \t/* unknown conditionals are always false */\n \treturn 0;\n@@ -335,9 +427,15 @@ static int git_config_include(const char *var, const char *value, void *data)\n \t\tret = handle_path_include(value, inc);\n \n \tif (!parse_config_key(var, \"includeif\", &cond, &cond_len, &key) &&\n-\t (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&\n-\t !strcmp(key, \"path\"))\n+\t cond && include_condition_is_true(inc, cond, cond_len) &&\n+\t !strcmp(key, \"path\")) {\n+\t\tconfig_fn_t old_fn = inc->fn;\n+\n+\t\tif (inc->opts->unconditional_remote_url)\n+\t\t\tinc->fn = forbid_remote_url;\n \t\tret = handle_path_include(value, inc);\n+\t\tinc->fn = old_fn;\n+\t}\n \n \treturn ret;\n }\n@@ -1933,11 +2031,13 @@ int config_with_options(config_fn_t fn, void *data,\n \t\t\tconst struct config_options *opts)\n {\n \tstruct config_include_data inc = CONFIG_INCLUDE_INIT;\n+\tint ret;\n \n \tif (opts->respect_includes) {\n \t\tinc.fn = fn;\n \t\tinc.data = data;\n \t\tinc.opts = opts;\n+\t\tinc.config_source = config_source;\n \t\tfn = git_config_include;\n \t\tdata = &inc;\n \t}\n@@ -1950,17 +2050,23 @@ int config_with_options(config_fn_t fn, void *data,\n \t * regular lookup sequence.\n \t */\n \tif (config_source && config_source->use_stdin) {\n-\t\treturn git_config_from_stdin(fn, data);\n+\t\tret = git_config_from_stdin(fn, data);\n \t} else if (config_source && config_source->file) {\n-\t\treturn git_config_from_file(fn, config_source->file, data);\n+\t\tret = git_config_from_file(fn, config_source->file, data);\n \t} else if (config_source && config_source->blob) {\n \t\tstruct repository *repo = config_source->repo ?\n \t\t\tconfig_source->repo : the_repository;\n-\t\treturn git_config_from_blob_ref(fn, repo, config_source->blob,\n+\t\tret = git_config_from_blob_ref(fn, repo, config_source->blob,\n \t\t\t\t\t\tdata);\n+\t} else {\n+\t\tret = do_git_config_sequence(opts, fn, data);\n \t}\n \n-\treturn do_git_config_sequence(opts, fn, data);\n+\tif (inc.remote_urls) {\n+\t\tstring_list_clear(inc.remote_urls, 0);\n+\t\tFREE_AND_NULL(inc.remote_urls);\n+\t}\n+\treturn ret;\n }\n \n static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)\ndiff --git a/config.h b/config.h\nindex 48a5e472ca..ab0106d287 100644\n--- a/config.h\n+++ b/config.h\n@@ -89,6 +89,15 @@ struct config_options {\n \tunsigned int ignore_worktree : 1;\n \tunsigned int ignore_cmdline : 1;\n \tunsigned int system_gently : 1;\n+\n+\t/*\n+\t * For internal use. Include all includeif.hasremoteurl paths without\n+\t * checking if the repo has that remote URL, and when doing so, verify\n+\t * that files included in this way do not configure any remote URLs\n+\t * themselves.\n+\t */\n+\tunsigned int unconditional_remote_url : 1;\n+\n \tconst char *commondir;\n \tconst char *git_dir;\n \tconfig_parser_event_fn_t event_fn;\ndiff --git a/t/t1300-config.sh b/t/t1300-config.sh\nindex 9ff46f3b04..c6b5911c4d 100755\n--- a/t/t1300-config.sh\n+++ b/t/t1300-config.sh\n@@ -2387,4 +2387,122 @@ test_expect_success '--get and --get-all with --fixed-value' '\n \ttest_must_fail git config --file=config --get-regexp --fixed-value fixed+ non-existent\n '\n \n+test_expect_success 'includeIf.hasconfig:remote.*.url' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >include-this <<-\\EOF &&\n+\t[user]\n+\t\tthis = this-is-included\n+\tEOF\n+\tcat >dont-include-that <<-\\EOF &&\n+\t[user]\n+\t\tthat = that-is-not-included\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasconfig:remote.*.url:foourl\"]\n+\t\tpath = \"$(pwd)/include-this\"\n+\t[includeIf \"hasconfig:remote.*.url:barurl\"]\n+\t\tpath = \"$(pwd)/dont-include-that\"\n+\t[remote \"foo\"]\n+\t\turl = foourl\n+\tEOF\n+\n+\techo this-is-included >expect-this &&\n+\tgit -C hasremoteurlTest config --get user.this >actual-this &&\n+\ttest_cmp expect-this actual-this &&\n+\n+\ttest_must_fail git -C hasremoteurlTest config --get user.that\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url respects last-config-wins' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >include-two-three <<-\\EOF &&\n+\t[user]\n+\t\ttwo = included-config\n+\t\tthree = included-config\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = foourl\n+\t[user]\n+\t\tone = main-config\n+\t\ttwo = main-config\n+\t[includeIf \"hasconfig:remote.*.url:foourl\"]\n+\t\tpath = \"$(pwd)/include-two-three\"\n+\t[user]\n+\t\tthree = main-config\n+\tEOF\n+\n+\techo main-config >expect-main-config &&\n+\techo included-config >expect-included-config &&\n+\n+\tgit -C hasremoteurlTest config --get user.one >actual &&\n+\ttest_cmp expect-main-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.two >actual &&\n+\ttest_cmp expect-included-config actual &&\n+\n+\tgit -C hasremoteurlTest config --get user.three >actual &&\n+\ttest_cmp expect-main-config actual\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url globs' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tprintf \"[user]\\ndss = yes\\n\" >double-star-start &&\n+\tprintf \"[user]\\ndse = yes\\n\" >double-star-end &&\n+\tprintf \"[user]\\ndsm = yes\\n\" >double-star-middle &&\n+\tprintf \"[user]\\nssm = yes\\n\" >single-star-middle &&\n+\tprintf \"[user]\\nno = no\\n\" >no &&\n+\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[remote \"foo\"]\n+\t\turl = https://foo/bar/baz\n+\t[includeIf \"hasconfig:remote.*.url:**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-start\"\n+\t[includeIf \"hasconfig:remote.*.url:**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**\"]\n+\t\tpath = \"$(pwd)/double-star-end\"\n+\t[includeIf \"hasconfig:remote.*.url:nomatch:/**\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**/baz\"]\n+\t\tpath = \"$(pwd)/double-star-middle\"\n+\t[includeIf \"hasconfig:remote.*.url:https:/**/nomatch\"]\n+\t\tpath = \"$(pwd)/no\"\n+\t[includeIf \"hasconfig:remote.*.url:https://*/bar/baz\"]\n+\t\tpath = \"$(pwd)/single-star-middle\"\n+\t[includeIf \"hasconfig:remote.*.url:https://*/baz\"]\n+\t\tpath = \"$(pwd)/no\"\n+\tEOF\n+\n+\tgit -C hasremoteurlTest config --get user.dss &&\n+\tgit -C hasremoteurlTest config --get user.dse &&\n+\tgit -C hasremoteurlTest config --get user.dsm &&\n+\tgit -C hasremoteurlTest config --get user.ssm &&\n+\ttest_must_fail git -C hasremoteurlTest config --get user.no\n+'\n+\n+test_expect_success 'includeIf.hasconfig:remote.*.url forbids remote url in such included files' '\n+\tgit init hasremoteurlTest &&\n+\ttest_when_finished \"rm -rf hasremoteurlTest\" &&\n+\n+\tcat >include-with-url <<-\\EOF &&\n+\t[remote \"bar\"]\n+\t\turl = barurl\n+\tEOF\n+\tcat >>hasremoteurlTest/.git/config <<-EOF &&\n+\t[includeIf \"hasconfig:remote.*.url:foourl\"]\n+\t\tpath = \"$(pwd)/include-with-url\"\n+\tEOF\n+\n+\t# test with any Git command\n+\ttest_must_fail git -C hasremoteurlTest status 2>err &&\n+\tgrep \"fatal: remote URLs cannot be configured in file directly or indirectly included by includeIf.hasconfig:remote.*.url\" err\n+'\n+\n test_done\n-- \n2.34.1.703.g22d0c6ccf7-goog\n\n"},{"id":"446450","messageId":"CABPp-BE+CCqZxSZnhn7drqYsA9XhqGaE++kPwRiAtbm4pvSFVA@mail.gmail.com","threadId":"56696","inReplyTo":"cover.1642527965.git.jonathantanmy@google.com","subject":"Re: [PATCH v8 0/2] Conditional config includes based on remote URL","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-01-18T20:54:40Z","receivedAt":"2022-01-18T20:55:02Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jan 18, 2022 at 9:47 AM Jonathan Tan wrote:\n>\n> Thanks everyone for your review. v8 follows Elijah Newren's comments\n> about making URLs more identifiable in tests, and also including an\n> example in documentation.\n\nThanks for that; this version looks good to me.\n\n> Jonathan Tan (2):\n> config: make git_config_include() static\n> config: include file if remote URL matches a glob\n>\n> Documentation/config.txt | 35 +++++++++++\n> config.c | 132 ++++++++++++++++++++++++++++++++++++---\n> config.h | 46 ++++----------\n> t/t1300-config.sh | 118 ++++++++++++++++++++++++++++++++++\n> 4 files changed, 290 insertions(+), 41 deletions(-)\n>\n> Range-diff against v7:\n> 1: b2dcae03ed = 1: b2dcae03ed config: make git_config_include() static\n> 2: 7c70089074 ! 2: 6691e39c82 config: include file if remote URL matches a glob\n> @@ Documentation/config.txt: all branches that begin with `foo/`. This is useful if\n> A few more notes on matching via `gitdir` and `gitdir/i`:\n>\n> * Symlinks in `$GIT_DIR` are not resolved before matching.\n> +@@ Documentation/config.txt: Example\n> + ; currently checked out\n> + [includeIf \"onbranch:foo-branch\"]\n> + path = foo.inc\n> ++\n> ++; include only if a remote with the given URL exists (note\n> ++; that such a URL may be provided later in a file or in a\n> ++; file read after this file is read, as seen in this example)\n> ++[includeIf \"hasconfig:remote.*.url:https://example.com/**\"]\n> ++ path = foo.inc\n> ++[remote \"origin\"]\n> ++ url = https://example.com/git\n> + ----\n> +\n> + Values\n>\n> ## config.c ##\n> @@ config.c: struct config_include_data {\n> @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n> + that = that-is-not-included\n> + EOF\n> + cat >>hasremoteurlTest/.git/config <<-EOF &&\n> -+ [includeIf \"hasconfig:remote.*.url:foo\"]\n> ++ [includeIf \"hasconfig:remote.*.url:foourl\"]\n> + path = \"$(pwd)/include-this\"\n> -+ [includeIf \"hasconfig:remote.*.url:bar\"]\n> ++ [includeIf \"hasconfig:remote.*.url:barurl\"]\n> + path = \"$(pwd)/dont-include-that\"\n> + [remote \"foo\"]\n> -+ url = foo\n> ++ url = foourl\n> + EOF\n> +\n> + echo this-is-included >expect-this &&\n> @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n> + EOF\n> + cat >>hasremoteurlTest/.git/config <<-EOF &&\n> + [remote \"foo\"]\n> -+ url = foo\n> ++ url = foourl\n> + [user]\n> + one = main-config\n> + two = main-config\n> -+ [includeIf \"hasconfig:remote.*.url:foo\"]\n> ++ [includeIf \"hasconfig:remote.*.url:foourl\"]\n> + path = \"$(pwd)/include-two-three\"\n> + [user]\n> + three = main-config\n> @@ t/t1300-config.sh: test_expect_success '--get and --get-all with --fixed-value'\n> +\n> + cat >include-with-url <<-\\EOF &&\n> + [remote \"bar\"]\n> -+ url = bar\n> ++ url = barurl\n> + EOF\n> + cat >>hasremoteurlTest/.git/config <<-EOF &&\n> -+ [includeIf \"hasconfig:remote.*.url:foo\"]\n> ++ [includeIf \"hasconfig:remote.*.url:foourl\"]\n> + path = \"$(pwd)/include-with-url\"\n> + EOF\n> +\n> --\n> 2.34.1.703.g22d0c6ccf7-goog\n>\n"},{"id":"446845","messageId":"nycvar.QRO.7.76.6.2201251424530.2121@tvgsbejvaqbjf.bet","threadId":"56696","inReplyTo":"CABPp-BGtA1KwV8kdN52r5=XTCZrS_e3sPYfTxMnoEVyVV8Hc_w@mail.gmail.com","subject":"Scalar vs JGit, was Re: [PATCH v7 0/2] Conditional config includes based on remote URL","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-01-25T13:26:29Z","receivedAt":"2022-01-25T13:29:04Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Elijah,\n\nOn Mon, 10 Jan 2022, Elijah Newren wrote:\n\n> On Mon, Jan 10, 2022 at 11:22 AM Jonathan Tan wrote:\n> >\n> > Elijah Newren writes:\n> >\n> > > Scalar also has a mechanism for providing potentially large blocks\n> > > of pre-vetted configuration for users. It does so as part of a new\n> > > top-level command. And it does so with a very opinionated set of\n> > > values that are not configurable. Thus, while I'd like to use it,\n> > > they use a configuration option that would break things badly at my\n> > > $DAYJOB. (Too many gradle plugins using jgit, which doesn't\n> > > understand index.version=4 and will blow up with a very suboptimal\n> > > error message when they see it.) And, it's very specific to scalar;\n> > > we probably don't want to add a new toplevel command everytime\n> > > someone wants common configuration to be easily grabbed by some\n> > > user.\n> >\n> > Do you have more information on this? The closest thing I've seen is\n> > \"Scalar Config\" under \"Modifying Configuration Values\" in [1], which\n> > seems to be more about bundling additional tools (which may change\n> > config, of course).\n> >\n> > Unless you're referring to the config bundled in the Scalar tool itself,\n> > in which case this patch set seems orthogonal and potentially\n> > complementary - I was envisioning config being provided by a package\n> > manager package, but Scalar could provide some too for users to use at\n> > their own discretion.\n> >\n> > [1] https://github.com/microsoft/git/blob/7a514b4c2d5df7fdd2f66f048010d8ddcb412d0b/contrib/scalar/docs/troubleshooting.md\n>\n> Yes, I was referring to the config hardcoded in the Scalar tool itself\n> (see set_recommended_config() in\n> https://lore.kernel.org/git/4439ab4de0bc3f48a6bdcf4b5165b16fad792ebd.1638538470.git.gitgitgadget@gmail.com/).\n\nI was kind of thinking that such problems might be solved via introducing\ne.g. `scalar.ensureJGitCompatibility = true` (which should be a relatively\ntrivial patch to write).\n\nWhat do you think?\n\nCiao,\nDscho\n"}]}