Re: [PATCH v2 0/8] Auto-configure advertised remotes via URL allowlist
- From
Christian Couder <christian.couder@gmail.com>
- Date
- Apr 27, 2026, 13:00 UTC
- Message-ID
- <CAP8UFD2b+AHD0rfR9PxBcANAGg823LRk0S6zWCJWKMVWEGHYjQ@mail.gmail.com>
- In-Reply-To
- <20260427124108.3524129-1-christian.couder@gmail.com>
Sorry, it looks like I sent this series in reply to:
https://lore.kernel.org/git/20251223111113.47473-1-christian.couder@gmail.com/
instead of:
https://lore.kernel.org/git/20260323080520.887550-1-christian.couder@gmail.com/
I will try to do better next time.
Also I forgot to say that this series is based on a merge of 'master' @ v2.54.0 and 'cc/promisor-auto-config-url' (which is in 'next' but is marked with "Will merge to master" in the last "What's cooking..." email).
On Mon, Apr 27, 2026 at 2:41 PM Christian Couder <christian.couder@gmail.com> wrote:
Show 1023 quoted lines
>
> Currently, the "promisor-remote" protocol capability allows a server
> to advertise promisor remotes (and their tokens/filters), but the
> client's `promisor.acceptFromServer` mechanism requires these remotes
> to already exist in the config.
>
> This is a significant burden for users and administrators who have to
> pre-configure remotes.
>
> This patch series improves on this by introducing a new
> `promisor.acceptFromServerUrl` config option, which provides an
> additive, URL-based security allowlist.
>
> Multiple `promisor.acceptFromServerUrl` config options can be provided
> in different config files. Each one should contain a URL glob pattern
> which can optionally be prefixed with a remote name in the
> "[<name>=]<pattern>" format.
>
> The goal is for something like a simple:
>
> git config set --global promisor.acceptFromServerUrl "https://my-org.com/*"
>
> to be all that is needed for internal work in many organizations.
>
> With this new config option:
>
> - The server can update fields (like tokens) for known remotes,
> provided their URL matches the allowlist, even if
> `acceptFromServer` is set to `None`.
>
> - Unknown remotes advertised by the server can be automatically
> configured on the client if their URL matches the allowlist.
>
> - If there is no `<name>` prefix before the glob pattern matched, the
> auto-configured remote is named using the
> "promisor-auto-<sanitized-url>" format. So the same auto-configured
> remote config entry will be reused for the same URL.
>
> - If a `<name>` prefix is provided, it will be used for the
> auto-configured remote config entry.
>
> - If the chosen name (auto-generated or prefixed) already exists but
> points to a different URL, overwriting the existing config is
> prevented by appending a numeric suffix (e.g., -1, -2) to the name
> and auto-configuring using that name.
>
> - The server's originally advertised name is always saved in the
> `remote.<name>.advertisedAs` config variable of the auto-configured
> remote for tracing and debugging.
>
> Security considerations:
>
> - Advertised URLs and glob patterns are routed through
> url_normalize() / url_normalize_pattern() before matching, to
> prevent percent-encoding, case variation, or path-traversal (..)
> bypasses.
>
> - URL matching is done component by component: scheme and port
> must match exactly (no wildcards), the host is matched with
> WM_PATHNAME so a '*' cannot cross the '/' boundary into the
> path, and the path is matched without WM_PATHNAME so '*' can
> still span multi-level paths.
>
> - Auto-generated remote names are sanitized (non-alphanumeric
> characters are replaced with '-', runs of '-' are collapsed)
> and prefixed with 'promisor-auto-'. User-supplied names (from
> the 'name=<pattern>' syntax) are validated with
> valid_remote_name(). Together, these prevent a server from
> maliciously overwriting standard remotes (like 'origin').
>
> - If the auto-generated or user-supplied name collides with an
> existing remote configured to a different URL, a numeric
> suffix ('-1', '-2', ...) is appended, up to a bounded limit,
> so a server cannot hijack an existing remote by name.
>
> - Known remotes are still subject to URL consistency checks:
> even if an advertised URL matches the allowlist, it is only
> accepted for a known remote if it matches the URL already
> configured locally for that remote.
>
> - The documentation explains in detail how to write secure glob
> patterns in `promisor.acceptFromServerUrl`, and highlights the
> risks of overly broad patterns on shared hosting platforms.
>
> High level description of the patches
> =====================================
>
> - Patch 1/8 is new. It is a very small preparatory patch that
> simplifies some tests a bit.
>
> - Patches 2/8 and 3/8 expose and adapt a url_normalize_pattern()
> helper function in the urlmatch API.
>
> - Patch 4/8 adapts `struct promisor_info` by adding a new
> `local_name` member to it to prepare for the next patches.
>
> - Patches 5/8 to 7/8 implement the core feature. They introduce the
> parsing machinery, add the additive allowlist for known remotes
> (with url_normalize() security), and finally implement the
> auto-creation and collision resolution for unknown remotes.
>
> - Patch 8/8 cleans up and modernizes the existing
> `promisor.acceptFromServer` documentation.
>
> Changes compared to v1
> ======================
>
> Thanks to Patrick and Junio for reviewing the previous versions of
> this series and of the preparatory series.
>
> - A lot of preparatory patches have been moved to a preparatory series
> that has already been merged. See:
>
> https://lore.kernel.org/git/20260407115243.358642-1-christian.couder@gmail.com/
>
> This is why this v2 contains only 8 patches compared to 16 patches
> in v1.
>
> - Everywhere in this series "whitelist" as been replaced with
> "allowlist".
>
> - In the tests added in this series, the new $TRASH_DIRECTORY_URL and
> $ENCODED_TRASH_DIRECTORY_URL introduced by the preparatory series
> are used instead of the previous $PWD_URL and $ENCODED_PWD_URL.
>
> - Patch 1/8 ("t5710: simplify 'mkdir X' followed by 'git -C X init'")
> is new.
>
> - Patch 3/8 ("urlmatch: add url_normalize_pattern() helper") replaces
> patch 3/16 ("urlmatch: add url_is_valid_pattern() helper") because
> in subsequent patches we now normalize patterns to validate them
> and match them component by component against URLs.
>
> - In patch 5/8, previously 13/16, ("promisor-remote: introduce
> promisor.acceptFromServerUrl"):
>
> - We add a `struct url_info pattern_info;` to `struct allowed_url`,
> so we can validate patterns using url_normalize_pattern() and, in
> a subsequent patch, match URLs component by component. This
> requires a new allowed_url_free() function that is passed to
> string_list_clear_func() to clear the `struct allowed_url`
> instances.
>
> - We don't use a `static struct string_list` to store the URL
> patterns we accept. Instead we load them from the config into a
> `struct string_list` passed as argument. The function doing this
> is renamed accordingly from accept_from_server_url() to
> load_accept_from_server_url().
>
> - A "clone with invalid promisor.acceptFromServerUrl" test is moved
> from patch 15/16 to this patch as it's more relevant in this
> patch (where we validate the content of the
> `promisor.acceptFromServerUrl` environment variable).
>
> - In patch 6/8, previously 14/16, ("promisor-remote: trust known
> remotes matching acceptFromServerUrl"):
>
> - In the commit message, an example, which shows how the new
> "acceptFromServerUrl" config option can be useful, is added.
>
> - The matching of URLs advertised by the server to URLs patterns
> from the config, is now performed component by component. This is
> reflected in the commit message, the documentation and the
> code. This ensures a `*` in the host pattern cannot cross into
> the path.
>
> - In the code, we add a new match_one_url() function to perform the
> matching.
>
> - In patch 7/8, previously 15/16 ("promisor-remote: auto-configure
> unknown remotes"):
>
> - In the doc, the unclear "considered trusted by the client" is
> clarified using "a client is allowed to act on" and subsequent
> explanations. In general the doc is also improved a bit.
>
> - In the tests, parsing the "remote.<name>.advertisedAs" config
> option is now more careful about the possibility that more than
> one such options exist.
>
> - The test that was moved to patch 5/8 is still enhanced a bit in
> this commit by checking that no "remote.<name>.advertisedAs"
> config option has been added.
>
> CI tests
> ========
>
> They all pass, see:
>
> https://github.com/chriscool/git/actions/runs/24992478331
>
> Range diff since v1
> ===================
>
> 1: b2894eb33a < -: ---------- promisor-remote: try accepted remotes before others in get_direct()
> -: ---------- > 1: 44e9a16455 t5710: simplify 'mkdir X' followed by 'git -C X init'
> 2: a3206a6ae9 = 2: 42f174910c urlmatch: change 'allow_globs' arg to bool
> 3: 51bbf65c52 < -: ---------- urlmatch: add url_is_valid_pattern() helper
> 4: f367beef72 < -: ---------- promisor-remote: clarify that a remote is ignored
> 5: 1faf74cb3f < -: ---------- promisor-remote: refactor has_control_char()
> 6: 40cf0af639 < -: ---------- promisor-remote: refactor accept_from_server()
> 7: b75dca8037 < -: ---------- promisor-remote: keep accepted promisor_info structs alive
> 8: f5e55dc407 < -: ---------- promisor-remote: remove the 'accepted' strvec
> -: ---------- > 3: 8088374458 urlmatch: add url_normalize_pattern() helper
> 9: 63c1db30de ! 4: 6bfda89a79 promisor-remote: add 'local_name' to 'struct promisor_info'
> @@ Commit message
> In a following commit, we will store promisor remote information under
> a remote name different than the one the server advertised.
>
> - To prepare for this change, let's add a new 'char* local_name' member
> + To prepare for this change, let's add a new 'char *local_name' member
> to 'struct promisor_info', and let's update the related functions.
>
> While at it, let's also add a small promisor_info_internal_name()
> @@ Commit message
>
> ## promisor-remote.c ##
> @@ promisor-remote.c: static struct string_list *fields_stored(void)
> -
> - /*
> * Struct for promisor remotes involved in the "promisor-remote"
> -- * protocol capability.
> -+ * protocol capability:
> + * protocol capability.
> *
> - * Except for "name", each <member> in this struct and its <value>
> - * should correspond (either on the client side or on the server side)
> - * to a "remote.<name>.<member>" config variable set to <value> where
> - * "<name>" is a promisor remote name.
> -+ * - "name" is the name the server advertised.
> -+ * - "local_name" is the name we use locally (may be auto-generated).
> -+ *
> + * Except for "name" and "local_name", each <member> in this struct
> + * and its <value> should correspond (either on the client side or on
> + * the server side) to a "remote.<name>.<member>" config variable set
> + * to <value> where "<name>" is a promisor remote name.
> */
> struct promisor_info {
> - const char *name;
> -+ const char *local_name;
> +- const char *name;
> ++ const char *name; /* name the server advertised */
> ++ const char *local_name; /* name used locally (may be auto-generated) */
> const char *url;
> const char *filter;
> const char *token;
> 10: e9b8a64ab8 < -: ---------- promisor-remote: pass config entry to all_fields_match() directly
> 11: 2e1260190a < -: ---------- promisor-remote: refactor should_accept_remote() control flow
> 12: b33f06173a < -: ---------- t5710: use proper file:// URIs for absolute paths
> 13: 681b03e248 ! 5: fefa17e6dd promisor-remote: introduce promisor.acceptFromServerUrl
> @@ promisor-remote.c: static bool has_control_char(const char *s)
> +struct allowed_url {
> + char *remote_name;
> + char *url_pattern;
> ++ struct url_info pattern_info;
> +};
> +
> ++static void allowed_url_free(void *util, const char *str UNUSED)
> ++{
> ++ struct allowed_url *allowed = util;
> ++
> ++ if (!allowed)
> ++ return;
> ++
> ++ /* Depending on prefix, free either remote_name or url_pattern */
> ++ free(allowed->remote_name ? allowed->remote_name : allowed->url_pattern);
> ++ free(allowed->pattern_info.url);
> ++ free(allowed);
> ++}
> ++
> +static struct allowed_url *valid_accept_url(const char *url)
> +{
> + char *dup, *p;
> @@ promisor-remote.c: static bool has_control_char(const char *s)
> + p = dup;
> + }
> +
> -+ if (has_control_char(p) || !url_is_valid_pattern(p)) {
> ++ if (has_control_char(p)) {
> + warning(_("invalid url pattern '%s' "
> + "in '%s' from promisor.acceptFromServerUrl config"), p, url);
> + free(dup);
> @@ promisor-remote.c: static bool has_control_char(const char *s)
> + allowed = xmalloc(sizeof(*allowed));
> + allowed->remote_name = (p == dup) ? NULL : dup;
> + allowed->url_pattern = p;
> ++ allowed->pattern_info.url = url_normalize_pattern(p, &allowed->pattern_info);
> ++ if (!allowed->pattern_info.url) {
> ++ warning(_("invalid url pattern '%s' "
> ++ "in '%s' from promisor.acceptFromServerUrl config"), p, url);
> ++ free(dup);
> ++ free(allowed);
> ++ return NULL;
> ++ }
> +
> + return allowed;
> +}
> +
> -+static struct string_list *accept_from_server_url(struct repository *repo)
> ++static void load_accept_from_server_url(struct repository *repo,
> ++ struct string_list *accept_urls)
> +{
> -+ static struct string_list accept_urls = STRING_LIST_INIT_DUP;
> -+ static int initialized;
> + const struct string_list *config_urls;
> +
> -+ if (initialized)
> -+ return &accept_urls;
> -+
> -+ initialized = 1;
> -+
> + if (!repo_config_get_string_multi(repo, "promisor.acceptfromserverurl", &config_urls)) {
> + struct string_list_item *item;
> +
> @@ promisor-remote.c: static bool has_control_char(const char *s)
> + struct allowed_url *allowed = valid_accept_url(item->string);
> + if (allowed) {
> + struct string_list_item *new;
> -+ new = string_list_append(&accept_urls, item->string);
> ++ new = string_list_append(accept_urls, item->string);
> + new->util = allowed;
> + }
> + }
> + }
> -+
> -+ return &accept_urls;
> +}
> +
> static int should_accept_remote(enum accept_promisor accept,
> @@ promisor-remote.c: static void filter_promisor_remote(struct repository *repo,
> struct string_list_item *item;
> bool reload_config = false;
> enum accept_promisor accept = accept_from_server(repo);
> -+ /* Pre-load and validate the acceptFromServerUrl config */
> -+ (void)accept_from_server_url(repo);
> ++ struct string_list accept_urls = STRING_LIST_INIT_DUP;
> ++
> ++ /* Load and validate the acceptFromServerUrl config */
> ++ load_accept_from_server_url(repo, &accept_urls);
>
> if (accept == ACCEPT_NONE)
> return;
> +@@ promisor-remote.c: static void filter_promisor_remote(struct repository *repo,
> + }
> + }
> +
> ++ string_list_clear_func(&accept_urls, allowed_url_free);
> + promisor_info_list_clear(&config_info);
> + string_list_clear(&remote_info, 0);
> + store_info_free(store_info);
> +
> + ## t/t5710-promisor-remote-capability.sh ##
> +@@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'KnownUrl' and empty url, so not advertised" '
> + check_missing_objects server 1 "$oid"
> + '
> +
> ++test_expect_success "clone with invalid promisor.acceptFromServerUrl" '
> ++ git -C server config promisor.advertise true &&
> ++ test_when_finished "rm -rf client" &&
> ++
> ++ # As "bad name" contains a space, which is not a valid remote name,
> ++ # the pattern should be rejected with a warning and no remote created.
> ++ GIT_NO_LAZY_FETCH=0 git clone \
> ++ -c promisor.acceptfromserver=None \
> ++ -c "promisor.acceptFromServerUrl=bad name=https://example.com/*" \
> ++ --no-local --filter="blob:limit=5k" server client 2>err &&
> ++
> ++ # Check that a warning was emitted
> ++ test_grep "invalid remote name '\''bad name'\''" err &&
> ++
> ++ # Check that the largest object is not missing on the server
> ++ check_missing_objects server 0 "" &&
> ++
> ++ # Reinitialize server so that the largest object is missing again
> ++ initialize_server 1 "$oid"
> ++'
> ++
> + test_expect_success "clone with promisor.sendFields" '
> + git -C server config promisor.advertise true &&
> + test_when_finished "rm -rf client" &&
> 14: 8c04e48d66 ! 6: 2f238d0a7a promisor-remote: trust known remotes matching acceptFromServerUrl
> @@ Commit message
>
> To enable such targeted updates for trusted URLs, let's use the URL
> patterns from `promisor.acceptFromServerUrl` as an additional URL
> - based whitelist.
> + based allowlist.
>
> Concretely, let's check the advertised URLs against the URL glob
> patterns by introducing a new small helper function called
> url_matches_accept_list(), which iterates over the glob patterns and
> returns the first matching allowed_url entry (or NULL).
>
> - (Before matching, the advertised URL is passed through url_normalize()
> - so that case variations in the scheme/host, percent-encoding tricks,
> - and ".." path segments cannot bypass the whitelist.)
> + The URL matching is done component by component: scheme and port are
> + compared exactly, the host is matched with wildmatch() using the
> + WM_PATHNAME flag (so '*' cannot cross the '/' boundary into the path),
> + and the path is matched with wildmatch() without WM_PATHNAME (so '*'
> + can still match multi-level paths). Before matching, the advertised
> + URL is passed through url_normalize() so that case variations in the
> + scheme/host, percent-encoding tricks, and ".." path segments cannot
> + bypass the allowlist.
>
> Let's then use this helper at the tail of should_accept_remote() so
> that, when `accept == ACCEPT_NONE`, a known remote whose URL matches
> - the whitelist is still accepted.
> + the allowlist is still accepted.
>
> To prepare for this new logic, let's also:
>
> @@ Commit message
> and relax its early return so that the function is entered when
> `accept_urls` has entries even if `accept == ACCEPT_NONE`.
>
> + With this, many organizations may only need something like:
> +
> + git config set --global \
> + promisor.acceptFromServerUrl "https://my-org.com/*"
> +
> + to accept only their own remotes. And if they need to accept additional
> + remotes in some specific repos, they can also set:
> +
> + git config set promisor.acceptFromServer knownUrl
> +
> + and configure the additional remote manually only in the repos where
> + they are needed.
> +
> Let's then properly document `promisor.acceptFromServerUrl` in
> - "promisor.adoc" as an additive security whitelist for known remotes,
> - including the URL normalization behavior, and let's mention it in
> - "gitprotocol-v2.adoc".
> + "promisor.adoc" as an additive security allowlist for known remotes,
> + including the URL normalization behavior and the component-wise
> + matching, and let's mention it in "gitprotocol-v2.adoc".
>
> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
>
> @@ Documentation/config/promisor.adoc: promisor.acceptFromServer::
> comparisons are case sensitive. See linkgit:gitprotocol-v2[5].
>
> +promisor.acceptFromServerUrl::
> -+ A glob pattern to specify which URLs advertised by a server
> -+ are considered trusted by the client. This option acts as an
> -+ additive security whitelist that works in conjunction with
> -+ `promisor.acceptFromServer`.
> ++ A glob pattern to specify which server-advertised URLs a
> ++ client is allowed to act on. When a URL matches, the client
> ++ will accept the advertised remote as a promisor remote and may
> ++ automatically accept field updates (such as authentication
> ++ tokens) from the server, even if `promisor.acceptFromServer`
> ++ is set to `none` (the default).
> ++
> +This option can appear multiple times in config files. An advertised
> +URL will be accepted if it matches _ANY_ glob pattern specified by
> +this option in _ANY_ config file read by Git.
> ++
> -+Be _VERY_ careful with these glob patterns, as it can be a big
> -+security hole to allow any advertised remote to be auto-configured!
> ++Be _VERY_ careful with these patterns: `*` matches any sequence of
> ++characters within the 'host' and 'path' parts of a URL (but cannot
> ++cross part boundaries). An overly broad pattern is a major security
> ++risk, as a matching URL allows a server to update fields (such as
> ++authentication tokens) on known remotes without further confirmation.
> +To minimize security risks, follow these guidelines:
> ++
> +1. Start with a secure protocol scheme, like `https://` or `ssh://`.
> @@ Documentation/config/promisor.adoc: promisor.acceptFromServer::
> + your specific organization or namespace (e.g.,
> + `https://gitlab.com/your-org/*`).
> ++
> -+3. Don't use globs (`*`) in the domain name. For example
> -+ `https://cdn.example.com/*` is much safer than
> -+ `https://*.example.com/*`, because the latter matches
> -+ `https://evil-hacker.net/fake.example.com/repo`.
> ++3. Never use globs at the end of domain names. For example,
> ++ `https://cdn.your-org.com/*` might be safe, but
> ++ `https://cdn.your-org.com*/*` is a major security risk because
> ++ the latter matches `https://cdn.your-org.com.hacker.net/repo`.
> ++
> -+4. Make sure to have a `/` at the end of the domain name (or the end
> -+ of specific directories). For example `https://cdn.example.com/*`
> -+ is much safer than `https://cdn.example.com*`, because the latter
> -+ matches `https://cdn.example.com.hacker.net/repo`.
> ++4. Be careful using globs at the beginning of domain names. While the
> ++ code ensures a `*` in the host cannot cross into the path, a
> ++ pattern like `https://*.example.com/*` will still match any
> ++ subdomain. This is extremely dangerous on shared hosting platforms
> ++ (e.g., `https://*.github.io/*` trusts every user's site on the
> ++ entire platform).
> ++
> -+Before matching, the advertised URL is normalized: the scheme and
> -+host are lowercased, percent-encoded characters are decoded where
> -+possible, and path segments like `..` are resolved. Glob patterns
> -+are matched against this normalized URL as-is, so patterns should
> -+be written in normalized form (e.g., lowercase scheme and host).
> ++Before matching, both the advertised URL and the pattern are
> ++normalized: the scheme and host are lowercased, percent-encoded
> ++characters are decoded where possible, and path segments like `..`
> ++are resolved. The port must also match exactly (e.g.,
> ++`https://example.com:8080/*` will not match a URL advertised on
> ++port 9999).
> ++
> -+Even if `promisor.acceptFromServer` is set to `None` (the default),
> -+Git will still accept field updates (like tokens) for known remotes,
> -+provided their URLs match a pattern in
> -+`promisor.acceptFromServerUrl`. See linkgit:gitprotocol-v2[5] for
> -+details on the protocol.
> ++For the security implications of accepting a promisor remote, see the
> ++documentation of `promisor.acceptFromServer`. For details on the
> ++protocol, see linkgit:gitprotocol-v2[5].
> +
> promisor.checkFields::
> A comma or space separated list of additional remote related
> @@ promisor-remote.c
>
> struct promisor_remote_config {
> struct promisor_remote *promisors;
> -@@ promisor-remote.c: static struct string_list *accept_from_server_url(struct repository *repo)
> - return &accept_urls;
> +@@ promisor-remote.c: static void load_accept_from_server_url(struct repository *repo,
> + }
> }
>
> ++static bool match_one_url(const struct url_info *pi, const struct url_info *ui)
> ++{
> ++ const char *pat = pi->url;
> ++ const char *url = ui->url;
> ++ char *p_str, *u_str;
> ++ bool res;
> ++
> ++ /*
> ++ * Schemes must match exactly. They are case-folded by
> ++ * url_normalize(), so strncmp() suffices.
> ++ */
> ++ if (pi->scheme_len != ui->scheme_len || strncmp(pat, url, pi->scheme_len))
> ++ return false;
> ++
> ++ /*
> ++ * Ports must match exactly. url_normalize() strips default
> ++ * ports (like 443 for https), so length and content
> ++ * comparisons are sufficient.
> ++ */
> ++ if (pi->port_len != ui->port_len ||
> ++ strncmp(pat + pi->port_off, url + ui->port_off, pi->port_len))
> ++ return false;
> ++
> ++ /*
> ++ * Match host and path separately to prevent a '*' in the host
> ++ * portion of the pattern from matching across the '/'
> ++ * boundary into the path. Use WM_PATHNAME for the host so '*'
> ++ * cannot cross '/' there, and 0 for the path so '*' can still
> ++ * match multi-level paths.
> ++ */
> ++
> ++ p_str = xstrndup(pat + pi->host_off, pi->host_len);
> ++ u_str = xstrndup(url + ui->host_off, ui->host_len);
> ++ res = !wildmatch(p_str, u_str, WM_PATHNAME);
> ++ free(p_str);
> ++ free(u_str);
> ++
> ++ if (!res)
> ++ return false;
> ++
> ++ p_str = xstrndup(pat + pi->path_off, pi->path_len);
> ++ u_str = xstrndup(url + ui->path_off, ui->path_len);
> ++ res = !wildmatch(p_str, u_str, 0);
> ++ free(p_str);
> ++ free(u_str);
> ++
> ++ return res;
> ++}
> ++
> +static struct allowed_url *url_matches_accept_list(
> + struct string_list *accept_urls, const char *url)
> +{
> + struct string_list_item *item;
> -+ char *normalized = url_normalize(url, NULL);
> ++ struct url_info url_info;
> ++
> ++ url_info.url = url_normalize(url, &url_info);
> +
> -+ if (!normalized)
> ++ if (!url_info.url)
> + return NULL;
> +
> + for_each_string_list_item(item, accept_urls) {
> + struct allowed_url *allowed = item->util;
> +
> -+ if (!wildmatch(allowed->url_pattern, normalized, 0)) {
> -+ free(normalized);
> ++ if (match_one_url(&allowed->pattern_info, &url_info)) {
> ++ free(url_info.url);
> + return allowed;
> + }
> + }
> +
> -+ free(normalized);
> ++ free(url_info.url);
> + return NULL;
> +}
> +
> @@ promisor-remote.c: static int should_accept_remote(enum accept_promisor accept,
> + /*
> + * Even if accept == ACCEPT_NONE, we MUST trust this known
> + * remote to update its token or other such fields if its URL
> -+ * matches the acceptFromServerUrl whitelist!
> ++ * matches the acceptFromServerUrl allowlist!
> + */
> + if (url_matches_accept_list(accept_urls, remote_url))
> + return all_fields_match(advertised, config_info, p);
> @@ promisor-remote.c: static int should_accept_remote(enum accept_promisor accept,
>
> static int skip_field_name_prefix(const char *elem, const char *field_name, const char **value)
> @@ promisor-remote.c: static void filter_promisor_remote(struct repository *repo,
> - struct string_list_item *item;
> - bool reload_config = false;
> - enum accept_promisor accept = accept_from_server(repo);
> -- /* Pre-load and validate the acceptFromServerUrl config */
> -- (void)accept_from_server_url(repo);
> -+ struct string_list *accept_urls = accept_from_server_url(repo);
> + /* Load and validate the acceptFromServerUrl config */
> + load_accept_from_server_url(repo, &accept_urls);
>
> - if (accept == ACCEPT_NONE)
> -+ if (accept == ACCEPT_NONE && !accept_urls->nr)
> ++ if (accept == ACCEPT_NONE && !accept_urls.nr)
> return;
>
> /* Parse remote info received */
> @@ promisor-remote.c: static void filter_promisor_remote(struct repository *repo,
> }
>
> - if (should_accept_remote(accept, advertised, &config_info)) {
> -+ if (should_accept_remote(accept, advertised, accept_urls, &config_info)) {
> ++ if (should_accept_remote(accept, advertised, &accept_urls, &config_info)) {
> if (!store_info)
> store_info = store_info_new(repo);
> if (promisor_store_advertised_fields(advertised, store_info))
> @@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'KnownUrl
> check_missing_objects server 1 "$oid"
> '
>
> -+test_expect_success "clone with 'None' but URL whitelisted" '
> ++test_expect_success "clone with 'None' but URL allowlisted" '
> + git -C server config promisor.advertise true &&
> + test_when_finished "rm -rf client" &&
> +
> + GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \
> + -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \
> -+ -c remote.lop.url="$PWD_URL/lop" \
> ++ -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \
> + -c promisor.acceptfromserver=None \
> -+ -c promisor.acceptFromServerUrl="$ENCODED_PWD_URL/*" \
> ++ -c promisor.acceptFromServerUrl="$ENCODED_TRASH_DIRECTORY_URL/*" \
> + --no-local --filter="blob:limit=5k" server client &&
> +
> + # Check that the largest object is still missing on the server
> + check_missing_objects server 1 "$oid"
> +'
> +
> -+test_expect_success "clone with 'None' but URL not in whitelist" '
> ++test_expect_success "clone with 'None' but URL not in allowlist" '
> + git -C server config promisor.advertise true &&
> + test_when_finished "rm -rf client" &&
> +
> + GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \
> + -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \
> -+ -c remote.lop.url="$PWD_URL/lop" \
> ++ -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \
> + -c promisor.acceptfromserver=None \
> + -c promisor.acceptFromServerUrl="https://example.com/*" \
> + --no-local --filter="blob:limit=5k" server client &&
> @@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'KnownUrl
> + initialize_server 1 "$oid"
> +'
> +
> -+test_expect_success "clone with 'None' but URL whitelisted in one pattern out of two" '
> ++test_expect_success "clone with 'None' but URL allowlisted in one pattern out of two" '
> + git -C server config promisor.advertise true &&
> + test_when_finished "rm -rf client" &&
> +
> + GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \
> + -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \
> -+ -c remote.lop.url="$PWD_URL/lop" \
> ++ -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \
> + -c promisor.acceptfromserver=None \
> + -c promisor.acceptFromServerUrl="https://example.com/*" \
> -+ -c promisor.acceptFromServerUrl="$ENCODED_PWD_URL/*" \
> ++ -c promisor.acceptFromServerUrl="$ENCODED_TRASH_DIRECTORY_URL/*" \
> + --no-local --filter="blob:limit=5k" server client &&
> +
> + # Check that the largest object is still missing on the server
> + check_missing_objects server 1 "$oid"
> +'
> +
> -+test_expect_success "clone with 'None', URL whitelisted, but client has different URL" '
> ++test_expect_success "clone with 'None', URL allowlisted, but client has different URL" '
> + git -C server config promisor.advertise true &&
> + test_when_finished "rm -rf client" &&
> +
> + # The client configures "lop" with a different URL (serverTwo) than
> + # what the server advertises (lop). Even though the advertised URL
> -+ # matches the whitelist, the remote is rejected because the
> ++ # matches the allowlist, the remote is rejected because the
> + # configured URL does not match the advertised one.
> + GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \
> + -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \
> -+ -c remote.lop.url="$PWD_URL/serverTwo" \
> ++ -c remote.lop.url="$TRASH_DIRECTORY_URL/serverTwo" \
> + -c promisor.acceptfromserver=None \
> -+ -c promisor.acceptFromServerUrl="$ENCODED_PWD_URL/*" \
> ++ -c promisor.acceptFromServerUrl="$ENCODED_TRASH_DIRECTORY_URL/*" \
> + --no-local --filter="blob:limit=5k" server client &&
> +
> + # Check that the largest object is not missing on the server
> @@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'KnownUrl
> + initialize_server 1 "$oid"
> +'
> +
> - test_expect_success "clone with promisor.sendFields" '
> + test_expect_success "clone with invalid promisor.acceptFromServerUrl" '
> git -C server config promisor.advertise true &&
> test_when_finished "rm -rf client" &&
> -@@ t/t5710-promisor-remote-capability.sh: test_expect_success "subsequent fetch from a client when promisor.advertise is f
> - check_missing_objects server 1 "$oid"
> - '
> -
> -+
> -+
> - test_done
> 15: 314150a860 ! 7: a077f33df4 promisor-remote: auto-configure unknown remotes
> @@ Commit message
> promisor-remote: auto-configure unknown remotes
>
> Previous commits have introduced the `promisor.acceptFromServerUrl`
> - config variable to whitelist some URLs advertised by a server through
> + config variable to allowlist some URLs advertised by a server through
> the "promisor-remote" protocol capability.
>
> However the new `promisor.acceptFromServerUrl` mechanism, like the old
> @@ Commit message
>
> ## Documentation/config/promisor.adoc ##
> @@ Documentation/config/promisor.adoc: promisor.acceptFromServer::
> -
> promisor.acceptFromServerUrl::
> - A glob pattern to specify which URLs advertised by a server
> -- are considered trusted by the client. This option acts as an
> -- additive security whitelist that works in conjunction with
> -- `promisor.acceptFromServer`.
> -+ are allowed to be auto-configured (created and persisted) on
> -+ the client side. Unlike `promisor.acceptFromServer`, which
> -+ only accepts already configured remotes, a match against this
> -+ option instructs Git to write a new `[remote "<name>"]`
> -+ section to the client's configuration.
> + A glob pattern to specify which server-advertised URLs a
> + client is allowed to act on. When a URL matches, the client
> +- will accept the advertised remote as a promisor remote and may
> ++ will accept the advertised remote as a promisor remote, may
> ++ automatically create a new remote configuration for it and may
> + automatically accept field updates (such as authentication
> + tokens) from the server, even if `promisor.acceptFromServer`
> + is set to `none` (the default).
> +@@ Documentation/config/promisor.adoc: this option in _ANY_ config file read by Git.
> + Be _VERY_ careful with these patterns: `*` matches any sequence of
> + characters within the 'host' and 'path' parts of a URL (but cannot
> + cross part boundaries). An overly broad pattern is a major security
> +-risk, as a matching URL allows a server to update fields (such as
> +-authentication tokens) on known remotes without further confirmation.
> +-To minimize security risks, follow these guidelines:
> ++risk, as a matching URL allows a server to auto-configure new remotes
> ++and to update fields (such as authentication tokens) on known remotes
> ++without further confirmation. To minimize security risks, follow these
> ++guidelines:
> + +
> + 1. Start with a secure protocol scheme, like `https://` or `ssh://`.
> +
> - This option can appear multiple times in config files. An advertised
> - URL will be accepted if it matches _ANY_ glob pattern specified by
> -@@ Documentation/config/promisor.adoc: possible, and path segments like `..` are resolved. Glob patterns
> - are matched against this normalized URL as-is, so patterns should
> - be written in normalized form (e.g., lowercase scheme and host).
> +@@ Documentation/config/promisor.adoc: are resolved. The port must also match exactly (e.g.,
> + `https://example.com:8080/*` will not match a URL advertised on
> + port 9999).
> +
> --Even if `promisor.acceptFromServer` is set to `None` (the default),
> --Git will still accept field updates (like tokens) for known remotes,
> --provided their URLs match a pattern in
> --`promisor.acceptFromServerUrl`. See linkgit:gitprotocol-v2[5] for
> --details on the protocol.
> +The glob pattern can optionally be prefixed with a remote name and an
> +equals sign (e.g., `cdn=https://cdn.example.com/*`). If such a prefix
> +is provided, accepted remotes will be saved under that name. If no
> +such prefix is provided, a safe remote name will be automatically
> +generated by sanitizing the URL and prefixing it with
> -+`promisor-auto-`. If a remote with the chosen name already exists but
> -+points to a different URL, Git will append a numeric suffix (e.g.,
> -+`-1`, `-2`) to the name to prevent overwriting existing
> -+configurations. You should make sure that this doesn't happen often
> -+though, as remotes will be rejected if the numeric suffix increases
> -+too much. In all cases, the original name advertised by the server is
> -+recorded in the `remote.<name>.advertisedAs` configuration variable
> -+for tracing and debugging purposes.
> ++`promisor-auto-`.
> ++
> -+Note that this option acts as an additive security whitelist. It works
> -+in conjunction with `promisor.acceptFromServer` (see the documentation
> -+of that option for the implications of accepting a promisor
> -+remote). Even if `promisor.acceptFromServer` is set to `None` (the
> -+default), Git will still automatically configure new remotes, and
> -+accept field updates (like tokens) for known remotes, provided their
> -+URLs match a pattern in `promisor.acceptFromServerUrl`. See
> -+linkgit:gitprotocol-v2[5] for details on the protocol.
> -
> - promisor.checkFields::
> - A comma or space separated list of additional remote related
> ++If a remote with the chosen name already exists but points to a
> ++different URL, Git will append a numeric suffix (e.g., `-1`, `-2`) to
> ++the name to prevent overwriting existing configurations. You should
> ++make sure that this doesn't happen often though, as remotes will be
> ++rejected if the numeric suffix increases too much. In all cases, the
> ++original name advertised by the server is recorded in the
> ++`remote.<name>.advertisedAs` configuration variable for tracing and
> ++debugging purposes.
> +++
> + For the security implications of accepting a promisor remote, see the
> + documentation of `promisor.acceptFromServer`. For details on the
> + protocol, see linkgit:gitprotocol-v2[5].
>
> ## Documentation/config/remote.adoc ##
> @@ Documentation/config/remote.adoc: remote.<name>.promisor::
> @@ promisor-remote.c: static void filter_promisor_remote(struct repository *repo,
> string_list_sort(&config_info);
> }
>
> -- if (should_accept_remote(accept, advertised, accept_urls, &config_info)) {
> -+ if (should_accept_remote(repo, accept, advertised, accept_urls,
> +- if (should_accept_remote(accept, advertised, &accept_urls, &config_info)) {
> ++ if (should_accept_remote(repo, accept, advertised, &accept_urls,
> + &config_info, &reload_config)) {
> if (!store_info)
> store_info = store_info_new(repo);
> if (promisor_store_advertised_fields(advertised, store_info))
>
> ## t/t5710-promisor-remote-capability.sh ##
> -@@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'None', URL whitelisted, but client has differen
> +@@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'None', URL allowlisted, but client has differen
> initialize_server 1 "$oid"
> '
>
> -+test_expect_success "clone with URL whitelisted and no remote already configured" '
> ++test_expect_success "clone with URL allowlisted and no remote already configured" '
> + git -C server config promisor.advertise true &&
> + test_when_finished "rm -rf client" &&
> ++ test_when_finished "rm -f full_names" &&
> +
> + GIT_NO_LAZY_FETCH=0 git clone \
> + -c promisor.acceptfromserver=None \
> -+ -c promisor.acceptFromServerUrl="$ENCODED_PWD_URL/*" \
> ++ -c promisor.acceptFromServerUrl="$ENCODED_TRASH_DIRECTORY_URL/*" \
> + --no-local --filter="blob:limit=5k" server client &&
> +
> -+ # Check that a remote has been auto-created with the right fields.
> -+ # The remote is identified by "remote.<name>.advertisedAs" == "lop".
> -+ FULL_NAME=$(git -C client config --name-only --get-regexp "remote\..*\.advertisedas" "^lop$") &&
> -+ REMOTE_NAME=$(echo "$FULL_NAME" | sed "s/remote\.\(.*\)\.advertisedas/\1/") &&
> ++ # Check that exactly one remote has been auto-created, identified
> ++ # by "remote.<name>.advertisedAs" == "lop".
> ++ git -C client config get --all --show-names --regexp \
> ++ "remote\..*\.advertisedas" >full_names &&
> ++ test_line_count = 1 full_names &&
> ++ REMOTE_NAME=$(sed "s/^remote\.\(.*\)\.advertisedas .*$/\1/" full_names) &&
> +
> + # Check ".url" and ".promisor" values
> -+ printf "%s\n" "$PWD_URL/lop" "true" >expect &&
> ++ printf "%s\n" "$TRASH_DIRECTORY_URL/lop" "true" >expect &&
> + git -C client config "remote.$REMOTE_NAME.url" >actual &&
> + git -C client config "remote.$REMOTE_NAME.promisor" >>actual &&
> + test_cmp expect actual &&
> @@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'None', U
> + check_missing_objects server 1 "$oid"
> +'
> +
> -+test_expect_success "clone with named URL whitelisted and no pre-configured remote" '
> ++test_expect_success "clone with named URL allowlisted and no pre-configured remote" '
> + git -C server config promisor.advertise true &&
> + test_when_finished "rm -rf client" &&
> +
> + GIT_NO_LAZY_FETCH=0 git clone \
> + -c promisor.acceptfromserver=None \
> -+ -c promisor.acceptFromServerUrl="cdn=$ENCODED_PWD_URL/*" \
> ++ -c promisor.acceptFromServerUrl="cdn=$ENCODED_TRASH_DIRECTORY_URL/*" \
> + --no-local --filter="blob:limit=5k" server client &&
> +
> + # Check that a remote has been auto-created with the right "cdn" name and fields.
> -+ printf "%s\n" "$PWD_URL/lop" "true" "lop" >expect &&
> ++ printf "%s\n" "$TRASH_DIRECTORY_URL/lop" "true" "lop" >expect &&
> + git -C client config "remote.cdn.url" >actual &&
> + git -C client config "remote.cdn.promisor" >>actual &&
> + git -C client config "remote.cdn.advertisedAs" >>actual &&
> @@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'None', U
> + check_missing_objects server 1 "$oid"
> +'
> +
> -+test_expect_success "clone with URL whitelisted but colliding name" '
> ++test_expect_success "clone with URL allowlisted but colliding name" '
> + git -C server config promisor.advertise true &&
> + test_when_finished "rm -rf client" &&
> +
> @@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'None', U
> + -c remote.cdn.fetch="+refs/heads/*:refs/remotes/lop/*" \
> + -c remote.cdn.url="https://example.com/cdn" \
> + -c promisor.acceptfromserver=None \
> -+ -c promisor.acceptFromServerUrl="cdn=$ENCODED_PWD_URL/*" \
> ++ -c promisor.acceptFromServerUrl="cdn=$ENCODED_TRASH_DIRECTORY_URL/*" \
> + --no-local --filter="blob:limit=5k" server client &&
> +
> + # Check that a remote has been auto-created with the right "cdn-1" name and fields.
> -+ printf "%s\n" "$PWD_URL/lop" "true" "lop" >expect &&
> ++ printf "%s\n" "$TRASH_DIRECTORY_URL/lop" "true" "lop" >expect &&
> + git -C client config "remote.cdn-1.url" >actual &&
> + git -C client config "remote.cdn-1.promisor" >>actual &&
> + git -C client config "remote.cdn-1.advertisedAs" >>actual &&
> @@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'None', U
> + check_missing_objects server 1 "$oid"
> +'
> +
> -+test_expect_success "clone with URL whitelisted and reusable remote" '
> ++test_expect_success "clone with URL allowlisted and reusable remote" '
> + git -C server config promisor.advertise true &&
> + test_when_finished "rm -rf client" &&
> +
> + GIT_NO_LAZY_FETCH=0 git clone \
> + -c remote.cdn.fetch="+refs/heads/*:refs/remotes/lop/*" \
> -+ -c remote.cdn.url="$PWD_URL/lop" \
> ++ -c remote.cdn.url="$TRASH_DIRECTORY_URL/lop" \
> + -c promisor.acceptfromserver=None \
> -+ -c promisor.acceptFromServerUrl="cdn=$ENCODED_PWD_URL/*" \
> ++ -c promisor.acceptFromServerUrl="cdn=$ENCODED_TRASH_DIRECTORY_URL/*" \
> + --no-local --filter="blob:limit=5k" server client &&
> +
> + # Check that the existing "cdn" remote has been properly updated.
> -+ printf "%s\n" "$PWD_URL/lop" "true" "lop" "+refs/heads/*:refs/remotes/lop/*" >expect &&
> ++ printf "%s\n" "$TRASH_DIRECTORY_URL/lop" "true" "lop" "+refs/heads/*:refs/remotes/lop/*" >expect &&
> + git -C client config "remote.cdn.url" >actual &&
> + git -C client config "remote.cdn.promisor" >>actual &&
> + git -C client config "remote.cdn.advertisedAs" >>actual &&
> @@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with 'None', U
> + check_missing_objects server 1 "$oid"
> +'
> +
> -+test_expect_success "clone with invalid promisor.acceptFromServerUrl" '
> -+ git -C server config promisor.advertise true &&
> -+ test_when_finished "rm -rf client" &&
> -+
> -+ # As "bad name" contains a space, which is not a valid remote name,
> -+ # the pattern should be rejected with a warning and no remote created.
> -+ GIT_NO_LAZY_FETCH=0 git clone \
> -+ -c promisor.acceptfromserver=None \
> -+ -c "promisor.acceptFromServerUrl=bad name=https://example.com/*" \
> -+ --no-local --filter="blob:limit=5k" server client 2>err &&
> -+
> -+ # Check that a warning was emitted
> -+ test_grep "invalid remote name '\''bad name'\''" err &&
> -+
> -+ # Check that no remote was auto-created
> -+ test_must_fail git -C client config --get-regexp "remote\..*\.advertisedas" &&
> -+
> -+ # Check that the largest object is not missing on the server
> -+ check_missing_objects server 0 "" &&
> -+
> -+ # Reinitialize server so that the largest object is missing again
> -+ initialize_server 1 "$oid"
> -+'
> -+
> - test_expect_success "clone with promisor.sendFields" '
> + test_expect_success "clone with invalid promisor.acceptFromServerUrl" '
> git -C server config promisor.advertise true &&
> test_when_finished "rm -rf client" &&
> +@@ t/t5710-promisor-remote-capability.sh: test_expect_success "clone with invalid promisor.acceptFromServerUrl" '
> + # Check that a warning was emitted
> + test_grep "invalid remote name '\''bad name'\''" err &&
> +
> ++ # Check that no remote was auto-created
> ++ test_must_fail git -C client config get --regexp "remote\..*\.advertisedas" &&
> ++
> + # Check that the largest object is not missing on the server
> + check_missing_objects server 0 "" &&
> +
> 16: 20f70b52bb ! 8: b68b9497aa doc: promisor: improve acceptFromServer entry
> @@ Documentation/config/promisor.adoc: variable is set to "true", and the "name" an
> +for protocol details.
>
> promisor.acceptFromServerUrl::
> - A glob pattern to specify which URLs advertised by a server
> + A glob pattern to specify which server-advertised URLs a
>
>
> Christian Couder (8):
> t5710: simplify 'mkdir X' followed by 'git -C X init'
> urlmatch: change 'allow_globs' arg to bool
> urlmatch: add url_normalize_pattern() helper
> promisor-remote: add 'local_name' to 'struct promisor_info'
> promisor-remote: introduce promisor.acceptFromServerUrl
> promisor-remote: trust known remotes matching acceptFromServerUrl
> promisor-remote: auto-configure unknown remotes
> doc: promisor: improve acceptFromServer entry
>
> Documentation/config/promisor.adoc | 123 ++++++--
> Documentation/config/remote.adoc | 9 +
> Documentation/gitprotocol-v2.adoc | 9 +-
> promisor-remote.c | 410 ++++++++++++++++++++++++--
> t/t5710-promisor-remote-capability.sh | 202 ++++++++++++-
> urlmatch.c | 11 +-
> urlmatch.h | 12 +
> 7 files changed, 730 insertions(+), 46 deletions(-)
>
> --
> 2.54.0.19.gb68b9497aa
>