# [PATCH v3 2/4] urlmatch: enable normalization of URLs with globs 10 messages from 2017-01-25 to 2017-01-27. Participants: Patrick Steinhardt, Junio C Hamano. Thread: https://gitlist.dev/t/44966 ## Patrick Steinhardt, 2017-01-25 09:56 Subject: [PATCH v3 2/4] urlmatch: enable normalization of URLs with globs Message-ID: <20170125095648.4116-3-patrick.steinhardt@elego.de> URL: https://gitlist.dev/e/20170125095648.4116-3-patrick.steinhardt%40elego.de In-Reply-To: <20170125095648.4116-1-patrick.steinhardt@elego.de> ``` The `url_normalize` function is used to validate and normalize URLs. As such, it does not allow for some special characters to be part of the URLs that are to be normalized. As we want to allow using globs in some configuration keys making use of URLs, namely `http..`, but still normalize them, we need to somehow enable some additional allowed characters. To do this without having to change all callers of `url_normalize`, where most do not actually want globbing at all, we split off another function `url_normalize_1`. This function accepts an additional parameter `allow_globs`, which is subsequently called by `url_normalize` with `allow_globs=0`. As of now, this function is not used with globbing enabled. A caller will be added in the following commit. Signed-off-by: Patrick Steinhardt --- urlmatch.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/urlmatch.c b/urlmatch.c index 132d342bc..d350478c0 100644 --- a/urlmatch.c +++ b/urlmatch.c @@ -63,7 +63,7 @@ static int append_normalized_escapes(struct strbuf *buf, return 1; } -char *url_normalize(const char *url, struct url_info *out_info) +static char *url_normalize_1(const char *url, struct url_info *out_info, char allow_globs) { /* * Normalize NUL-terminated url using the following rules: @@ -191,7 +191,12 @@ char *url_normalize(const char *url, struct url_info *out_info) strbuf_release(&norm); return NULL; } - spanned = strspn(url, URL_HOST_CHARS); + + if (allow_globs) + spanned = strspn(url, URL_HOST_CHARS "*"); + else + spanned = strspn(url, URL_HOST_CHARS); + if (spanned < colon_ptr - url) { /* Host name has invalid characters */ if (out_info) { @@ -380,6 +385,11 @@ char *url_normalize(const char *url, struct url_info *out_info) return result; } +char *url_normalize(const char *url, struct url_info *out_info) +{ + return url_normalize_1(url, out_info, 0); +} + static size_t url_match_prefix(const char *url, const char *url_prefix, size_t url_prefix_len) -- 2.11.0 ``` ## Patrick Steinhardt, 2017-01-25 09:56 Subject: [PATCH v3 0/4] urlmatch: allow regexp-based matches Message-ID: <20170125095648.4116-1-patrick.steinhardt@elego.de> URL: https://gitlist.dev/e/20170125095648.4116-1-patrick.steinhardt%40elego.de In-Reply-To: <20170123130635.29577-1-patrick.steinhardt@elego.de> ``` Hi, This is version three of my patch series. The previous version can be found at [1]. The use case is to be able to configure an HTTP proxy for all subdomains of a domain where there are hundreds of subdomains. This version addresses a comment by Philip Oakley regarding the documentation. You can find the interdiff below. Regards Patrick [1]: http://public-inbox.org/git/20170124170031.18069-1-patrick.steinhardt@elego.de/T/#u Patrick Steinhardt (4): mailmap: add Patrick Steinhardt's work address urlmatch: enable normalization of URLs with globs urlmatch: split host and port fields in `struct url_info` urlmatch: allow globbing for the URL host part .mailmap | 1 + Documentation/config.txt | 5 +++- t/t1300-repo-config.sh | 36 +++++++++++++++++++++++++ urlmatch.c | 68 +++++++++++++++++++++++++++++++++++++++++------- urlmatch.h | 9 ++++--- 5 files changed, 104 insertions(+), 15 deletions(-) diff --git a/Documentation/config.txt b/Documentation/config.txt index a78921c2b..078e9b490 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -1915,9 +1915,9 @@ http..*:: . Host/domain name (e.g., `example.com` in `https://example.com/`). This field must match between the config key and the URL. It is - possible to use globs in the config key to match all subdomains, e.g. - `https://*.example.com/` to match all subdomains of `example.com`. Note - that a glob only every matches a single part of the hostname. + possible to specify a `*` as part of the host name to match all subdomains + at this level. `https://*.example.com/` for example would match + `https://foo.example.com/`, but not `https://foo.bar.example.com/`. . Port number (e.g., `8080` in `http://example.com:8080/`). This field must match exactly between the config key and the URL. -- 2.11.0 ``` ## Patrick Steinhardt, 2017-01-25 09:56 Subject: [PATCH v3 4/4] urlmatch: allow globbing for the URL host part Message-ID: <20170125095648.4116-5-patrick.steinhardt@elego.de> URL: https://gitlist.dev/e/20170125095648.4116-5-patrick.steinhardt%40elego.de In-Reply-To: <20170125095648.4116-1-patrick.steinhardt@elego.de> ``` The URL matching function computes for two URLs whether they match not. The match is performed by splitting up the URL into different parts and then doing an exact comparison with the to-be-matched URL. The main user of `urlmatch` is the configuration subsystem. It allows to set certain configurations based on the URL which is being connected to via keys like `http..*`. A common use case for this is to set proxies for only some remotes which match the given URL. Unfortunately, having exact matches for all parts of the URL can become quite tedious in some setups. Imagine for example a corporate network where there are dozens or even hundreds of subdomains, which would have to be configured individually. This commit introduces the ability to use globbing in the host-part of the URLs. A user can simply specify a `*` as part of the host name to match all subdomains at this level. For example adding a configuration key `http.https://*.example.com.proxy` will match all subdomains of `https://example.com`. Signed-off-by: Patrick Steinhardt --- Documentation/config.txt | 5 ++++- t/t1300-repo-config.sh | 36 ++++++++++++++++++++++++++++++++++++ urlmatch.c | 38 ++++++++++++++++++++++++++++++++++---- 3 files changed, 74 insertions(+), 5 deletions(-) diff --git a/Documentation/config.txt b/Documentation/config.txt index 506431267..078e9b490 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -1914,7 +1914,10 @@ http..*:: must match exactly between the config key and the URL. . Host/domain name (e.g., `example.com` in `https://example.com/`). - This field must match exactly between the config key and the URL. + This field must match between the config key and the URL. It is + possible to specify a `*` as part of the host name to match all subdomains + at this level. `https://*.example.com/` for example would match + `https://foo.example.com/`, but not `https://foo.bar.example.com/`. . Port number (e.g., `8080` in `http://example.com:8080/`). This field must match exactly between the config key and the URL. diff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh index 923bfc5a2..ec545e092 100755 --- a/t/t1300-repo-config.sh +++ b/t/t1300-repo-config.sh @@ -1177,6 +1177,42 @@ test_expect_success 'urlmatch' ' test_cmp expect actual ' +test_expect_success 'glob-based urlmatch' ' + cat >.git/config <<-\EOF && + [http] + sslVerify + [http "https://*.example.com"] + sslVerify = false + cookieFile = /tmp/cookie.txt + EOF + + test_expect_code 1 git config --bool --get-urlmatch doesnt.exist https://good.example.com >actual && + test_must_be_empty actual && + + echo true >expect && + git config --bool --get-urlmatch http.SSLverify https://example.com >actual && + test_cmp expect actual && + + echo true >expect && + git config --bool --get-urlmatch http.SSLverify https://good-example.com >actual && + test_cmp expect actual && + + echo true >expect && + git config --bool --get-urlmatch http.sslverify https://deep.nested.example.com >actual && + test_cmp expect actual && + + echo false >expect && + git config --bool --get-urlmatch http.sslverify https://good.example.com >actual && + test_cmp expect actual && + + { + echo http.cookiefile /tmp/cookie.txt && + echo http.sslverify false + } >expect && + git config --get-urlmatch HTTP https://good.example.com >actual && + test_cmp expect actual +' + # good section hygiene test_expect_failure 'unsetting the last key in a section removes header' ' cat >.git/config <<-\EOF && diff --git a/urlmatch.c b/urlmatch.c index e328905eb..53ff972a6 100644 --- a/urlmatch.c +++ b/urlmatch.c @@ -63,6 +63,38 @@ static int append_normalized_escapes(struct strbuf *buf, return 1; } +static int match_host(const struct url_info *url_info, + const struct url_info *pattern_info) +{ + char *url = xmemdupz(url_info->url + url_info->host_off, url_info->host_len); + char *pat = xmemdupz(pattern_info->url + pattern_info->host_off, pattern_info->host_len); + char *url_tok, *pat_tok, *url_save, *pat_save; + int matching; + + url_tok = strtok_r(url, ".", &url_save); + pat_tok = strtok_r(pat, ".", &pat_save); + + for (; url_tok && pat_tok; url_tok = strtok_r(NULL, ".", &url_save), + pat_tok = strtok_r(NULL, ".", &pat_save)) { + if (!strcmp(pat_tok, "*")) + continue; /* a simple glob matches everything */ + + if (strcmp(url_tok, pat_tok)) { + /* subdomains do not match */ + matching = 0; + break; + } + } + + /* matching if both URL and pattern are at their ends */ + matching = (url_tok == NULL && pat_tok == NULL); + + free(url); + free(pat); + + return matching; +} + static char *url_normalize_1(const char *url, struct url_info *out_info, char allow_globs) { /* @@ -467,9 +499,7 @@ static int match_urls(const struct url_info *url, } /* check the host */ - if (url_prefix->host_len != url->host_len || - strncmp(url->url + url->host_off, - url_prefix->url + url_prefix->host_off, url->host_len)) + if (!match_host(url, url_prefix)) return 0; /* host names do not match */ /* check the port */ @@ -512,7 +542,7 @@ int urlmatch_config_entry(const char *var, const char *value, void *cb) struct url_info norm_info; config_url = xmemdupz(key, dot - key); - norm_url = url_normalize(config_url, &norm_info); + norm_url = url_normalize_1(config_url, &norm_info, 1); free(config_url); if (!norm_url) return 0; -- 2.11.0 ``` ## Patrick Steinhardt, 2017-01-25 09:56 Subject: [PATCH v3 1/4] mailmap: add Patrick Steinhardt's work address Message-ID: <20170125095648.4116-2-patrick.steinhardt@elego.de> URL: https://gitlist.dev/e/20170125095648.4116-2-patrick.steinhardt%40elego.de In-Reply-To: <20170125095648.4116-1-patrick.steinhardt@elego.de> ``` Signed-off-by: Patrick Steinhardt --- .mailmap | 1 + 1 file changed, 1 insertion(+) diff --git a/.mailmap b/.mailmap index 9c87a3840..ea59205b9 100644 --- a/.mailmap +++ b/.mailmap @@ -177,6 +177,7 @@ Paolo Bonzini Pascal Obry Pascal Obry Pat Notz +Patrick Steinhardt Paul Mackerras Paul Mackerras Peter Baumann

-- 2.11.0 ``` ## Patrick Steinhardt, 2017-01-25 09:56 Subject: [PATCH v3 3/4] urlmatch: split host and port fields in `struct url_info` Message-ID: <20170125095648.4116-4-patrick.steinhardt@elego.de> URL: https://gitlist.dev/e/20170125095648.4116-4-patrick.steinhardt%40elego.de In-Reply-To: <20170125095648.4116-1-patrick.steinhardt@elego.de> ``` The `url_info` structure contains information about a normalized URL with the URL's components being represented by different fields. The host and port part though are to be accessed by the same `host` field, so that getting the host and/or port separately becomes more involved than really necessary. To make the port more readily accessible, split up the host and port fields. Namely, the `host_len` will not include the port length anymore and a new `port_off` field has been added which includes the offset to the port, if available. The only user of these fields is `url_normalize_1`. This change makes it easier later on to treat host and port differently when introducing globs for domains. Signed-off-by: Patrick Steinhardt --- urlmatch.c | 16 ++++++++++++---- urlmatch.h | 9 +++++---- 2 files changed, 17 insertions(+), 8 deletions(-) diff --git a/urlmatch.c b/urlmatch.c index d350478c0..e328905eb 100644 --- a/urlmatch.c +++ b/urlmatch.c @@ -104,7 +104,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al struct strbuf norm; size_t spanned; size_t scheme_len, user_off=0, user_len=0, passwd_off=0, passwd_len=0; - size_t host_off=0, host_len=0, port_len=0, path_off, path_len, result_len; + size_t host_off=0, host_len=0, port_off=0, port_len=0, path_off, path_len, result_len; const char *slash_ptr, *at_ptr, *colon_ptr, *path_start; char *result; @@ -263,6 +263,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al return NULL; } strbuf_addch(&norm, ':'); + port_off = norm.len; strbuf_add(&norm, url, slash_ptr - url); port_len = slash_ptr - url; } @@ -270,7 +271,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al url = slash_ptr; } if (host_off) - host_len = norm.len - host_off; + host_len = norm.len - host_off - (port_len ? port_len + 1 : 0); /* @@ -378,6 +379,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al out_info->passwd_len = passwd_len; out_info->host_off = host_off; out_info->host_len = host_len; + out_info->port_off = port_off; out_info->port_len = port_len; out_info->path_off = path_off; out_info->path_len = path_len; @@ -464,11 +466,17 @@ static int match_urls(const struct url_info *url, usermatched = 1; } - /* check the host and port */ + /* check the host */ if (url_prefix->host_len != url->host_len || strncmp(url->url + url->host_off, url_prefix->url + url_prefix->host_off, url->host_len)) - return 0; /* host names and/or ports do not match */ + return 0; /* host names do not match */ + + /* check the port */ + if (url_prefix->port_len != url->port_len || + strncmp(url->url + url->port_off, + url_prefix->url + url_prefix->port_off, url->port_len)) + return 0; /* ports do not match */ /* check the path */ pathmatchlen = url_match_prefix( diff --git a/urlmatch.h b/urlmatch.h index 528862adc..0ea812b03 100644 --- a/urlmatch.h +++ b/urlmatch.h @@ -18,11 +18,12 @@ struct url_info { size_t passwd_len; /* length of passwd; if passwd_off != 0 but passwd_len == 0, an empty passwd was given */ size_t host_off; /* offset into url to start of host name (0 => none) */ - size_t host_len; /* length of host name; this INCLUDES any ':portnum'; + size_t host_len; /* length of host name; * file urls may have host_len == 0 */ - size_t port_len; /* if a portnum is present (port_len != 0), it has - * this length (excluding the leading ':') at the - * end of the host name (always 0 for file urls) */ + size_t port_off; /* offset into url to start of port number (0 => none) */ + size_t port_len; /* if a portnum is present (port_off != 0), it has + * this length (excluding the leading ':') starting + * from port_off (always 0 for file urls) */ size_t path_off; /* offset into url to the start of the url path; * this will always point to a '/' character * after the url has been normalized */ -- 2.11.0 ``` ## Junio C Hamano, 2017-01-26 20:43 Subject: Re: [PATCH v3 4/4] urlmatch: allow globbing for the URL host part Message-ID: URL: https://gitlist.dev/e/xmqq7f5h4kng.fsf%40gitster.mtv.corp.google.com In-Reply-To: <20170125095648.4116-5-patrick.steinhardt@elego.de> ``` Patrick Steinhardt writes: > The URL matching function computes for two URLs whether they match not. > The match is performed by splitting up the URL into different parts and > then doing an exact comparison with the to-be-matched URL. > > The main user of `urlmatch` is the configuration subsystem. It allows to > set certain configurations based on the URL which is being connected to > via keys like `http..*`. A common use case for this is to set > proxies for only some remotes which match the given URL. Unfortunately, > having exact matches for all parts of the URL can become quite tedious > in some setups. Imagine for example a corporate network where there are > dozens or even hundreds of subdomains, which would have to be configured > individually. > > This commit introduces the ability to use globbing in the host-part of > the URLs. A user can simply specify a `*` as part of the host name to > match all subdomains at this level. For example adding a configuration > key `http.https://*.example.com.proxy` will match all subdomains of > `https://example.com`. This is probably a useful improvement. Having said that, when I mentioned "glob", I meant to also support something like this: https://www[1-4].ibm.com/ And when people read "glob", that is what they expect. So calling this "the ability to use globbing" is misleading. The last paragraph in the log message above needs a bit of tweaking, perhaps like this: Allow users to write an asterisk '*' in place of any 'host' or 'subdomain' label as part of the host name. For example, "http.https://*.example.com.proxy" sets "http.proxy" for all direct subdomains of "https://example.com", e.g. "https://foo.example.com", but not "https://foo.bar.example.com". Fortunately, your update to config.txt, which is facing the end users, does not misuse the word and instead is explicit that the only thing the matcher does is to match '*' to a single hierarchy. It is clear that even http://www*.ibm.com/ is not supported from the description, which is good. > . Host/domain name (e.g., `example.com` in `https://example.com/`). > - This field must match exactly between the config key and the URL. > + This field must match between the config key and the URL. It is > + possible to specify a `*` as part of the host name to match all subdomains > + at this level. `https://*.example.com/` for example would match > + `https://foo.example.com/`, but not `https://foo.bar.example.com/`. This is good as-is. > . Port number (e.g., `8080` in `http://example.com:8080/`). > This field must match exactly between the config key and the URL. > diff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh > index 923bfc5a2..ec545e092 100755 > --- a/t/t1300-repo-config.sh > +++ b/t/t1300-repo-config.sh > @@ -1177,6 +1177,42 @@ test_expect_success 'urlmatch' ' > test_cmp expect actual > ' > > +test_expect_success 'glob-based urlmatch' ' This is not "glob". A more generic term "wildcard" is OK. > + cat >.git/config <<-\EOF && > + [http] > + sslVerify > ... > +static int match_host(const struct url_info *url_info, > + const struct url_info *pattern_info) > +{ > + char *url = xmemdupz(url_info->url + url_info->host_off, url_info->host_len); > + char *pat = xmemdupz(pattern_info->url + pattern_info->host_off, pattern_info->host_len); > + char *url_tok, *pat_tok, *url_save, *pat_save; > + int matching; > + > + url_tok = strtok_r(url, ".", &url_save); > + pat_tok = strtok_r(pat, ".", &pat_save); Hmph, this will be the first use of strtok_r() in our codebase. Does everybody have it? For a use like this where your delimiter set is a singleton, it may be simpler to do the usual strchrnul() or memchr() based loop. The attached is my attempt to do so on top of this patch. > + > + for (; url_tok && pat_tok; url_tok = strtok_r(NULL, ".", &url_save), > + pat_tok = strtok_r(NULL, ".", &pat_save)) { > + if (!strcmp(pat_tok, "*")) > + continue; /* a simple glob matches everything */ s/glob/asterisk/ Other than that, the patch looks OK. diff --git a/urlmatch.c b/urlmatch.c index 53ff972a60..8dfc7fd28a 100644 --- a/urlmatch.c +++ b/urlmatch.c @@ -63,36 +63,47 @@ static int append_normalized_escapes(struct strbuf *buf, return 1; } +static const char *end_of_token(const char *s, int c, size_t n) +{ + const char *next = memchr(s, c, n); + if (!next) + next = s + n; + return next; +} + static int match_host(const struct url_info *url_info, const struct url_info *pattern_info) { - char *url = xmemdupz(url_info->url + url_info->host_off, url_info->host_len); - char *pat = xmemdupz(pattern_info->url + pattern_info->host_off, pattern_info->host_len); - char *url_tok, *pat_tok, *url_save, *pat_save; - int matching; - - url_tok = strtok_r(url, ".", &url_save); - pat_tok = strtok_r(pat, ".", &pat_save); - - for (; url_tok && pat_tok; url_tok = strtok_r(NULL, ".", &url_save), - pat_tok = strtok_r(NULL, ".", &pat_save)) { - if (!strcmp(pat_tok, "*")) - continue; /* a simple glob matches everything */ - - if (strcmp(url_tok, pat_tok)) { - /* subdomains do not match */ - matching = 0; - break; - } + const char *url = url_info->url + url_info->host_off; + const char *pat = pattern_info->url + pattern_info->host_off; + int url_len = url_info->host_len; + int pat_len = pattern_info->host_len; + + while (url_len && pat_len) { + const char *url_next = end_of_token(url, '.', url_len); + const char *pat_next = end_of_token(pat, '.', pat_len); + + if (pat_next == pat + 1 && pat[0] == '*') + /* wildcard matches anything */ + ; + else if ((pat_next - pat) == (url_next - url) && + !memcmp(url, pat, url_next - url)) + /* the components are the same */ + ; + else + return 0; /* found an unmatch */ + + if (url_next < url + url_len) + url_next++; + url_len -= url_next - url; + url = url_next; + if (pat_next < pat + pat_len) + pat_next++; + pat_len -= pat_next - pat; + pat = pat_next; } - /* matching if both URL and pattern are at their ends */ - matching = (url_tok == NULL && pat_tok == NULL); - - free(url); - free(pat); - - return matching; + return 1; } static char *url_normalize_1(const char *url, struct url_info *out_info, char allow_globs) ``` ## Junio C Hamano, 2017-01-26 20:49 Subject: Re: [PATCH v3 4/4] urlmatch: allow globbing for the URL host part Message-ID: URL: https://gitlist.dev/e/xmqq37g54kdz.fsf%40gitster.mtv.corp.google.com In-Reply-To: ``` Junio C Hamano writes: > + while (url_len && pat_len) { > + const char *url_next = end_of_token(url, '.', url_len); > + const char *pat_next = end_of_token(pat, '.', pat_len); > + ... > } > > + return 1; Embarrassing. The last one must be "have they both run out?" i.e. return (!url_len && !pat_len); ``` ## Junio C Hamano, 2017-01-26 21:12 Subject: Re: [PATCH v3 4/4] urlmatch: allow globbing for the URL host part Message-ID: URL: https://gitlist.dev/e/xmqqo9yt34qw.fsf%40gitster.mtv.corp.google.com In-Reply-To: ``` Junio C Hamano writes: > Junio C Hamano writes: > >> + while (url_len && pat_len) { >> + const char *url_next = end_of_token(url, '.', url_len); >> + const char *pat_next = end_of_token(pat, '.', pat_len); >> + ... >> } >> >> + return 1; > > Embarrassing. The last one must be "have they both run out?" i.e. > > return (!url_len && !pat_len); OK, here is my second try. The added test piece is to catch the silly mistake I made above. diff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh index ec545e0929..33fd59fbb3 100755 --- a/t/t1300-repo-config.sh +++ b/t/t1300-repo-config.sh @@ -1177,7 +1177,7 @@ test_expect_success 'urlmatch' ' test_cmp expect actual ' -test_expect_success 'glob-based urlmatch' ' +test_expect_success 'urlmatch with wildcard' ' cat >.git/config <<-\EOF && [http] sslVerify @@ -1210,6 +1210,10 @@ test_expect_success 'glob-based urlmatch' ' echo http.sslverify false } >expect && git config --get-urlmatch HTTP https://good.example.com >actual && + test_cmp expect actual && + + echo http.sslverify >expect && + git config --get-urlmatch HTTP https://more.example.com.au >actual && test_cmp expect actual ' diff --git a/urlmatch.c b/urlmatch.c index 53ff972a60..0e007a3e07 100644 --- a/urlmatch.c +++ b/urlmatch.c @@ -63,36 +63,47 @@ static int append_normalized_escapes(struct strbuf *buf, return 1; } +static const char *end_of_token(const char *s, int c, size_t n) +{ + const char *next = memchr(s, c, n); + if (!next) + next = s + n; + return next; +} + static int match_host(const struct url_info *url_info, const struct url_info *pattern_info) { - char *url = xmemdupz(url_info->url + url_info->host_off, url_info->host_len); - char *pat = xmemdupz(pattern_info->url + pattern_info->host_off, pattern_info->host_len); - char *url_tok, *pat_tok, *url_save, *pat_save; - int matching; - - url_tok = strtok_r(url, ".", &url_save); - pat_tok = strtok_r(pat, ".", &pat_save); - - for (; url_tok && pat_tok; url_tok = strtok_r(NULL, ".", &url_save), - pat_tok = strtok_r(NULL, ".", &pat_save)) { - if (!strcmp(pat_tok, "*")) - continue; /* a simple glob matches everything */ - - if (strcmp(url_tok, pat_tok)) { - /* subdomains do not match */ - matching = 0; - break; - } + const char *url = url_info->url + url_info->host_off; + const char *pat = pattern_info->url + pattern_info->host_off; + int url_len = url_info->host_len; + int pat_len = pattern_info->host_len; + + while (url_len && pat_len) { + const char *url_next = end_of_token(url, '.', url_len); + const char *pat_next = end_of_token(pat, '.', pat_len); + + if (pat_next == pat + 1 && pat[0] == '*') + /* wildcard matches anything */ + ; + else if ((pat_next - pat) == (url_next - url) && + !memcmp(url, pat, url_next - url)) + /* the components are the same */ + ; + else + return 0; /* found an unmatch */ + + if (url_next < url + url_len) + url_next++; + url_len -= url_next - url; + url = url_next; + if (pat_next < pat + pat_len) + pat_next++; + pat_len -= pat_next - pat; + pat = pat_next; } - /* matching if both URL and pattern are at their ends */ - matching = (url_tok == NULL && pat_tok == NULL); - - free(url); - free(pat); - - return matching; + return (!url_len && !pat_len); } static char *url_normalize_1(const char *url, struct url_info *out_info, char allow_globs) ``` ## Patrick Steinhardt, 2017-01-27 06:21 Subject: Re: [PATCH v3 4/4] urlmatch: allow globbing for the URL host part Message-ID: <20170127062142.GA760@pks-pc> URL: https://gitlist.dev/e/20170127062142.GA760%40pks-pc In-Reply-To: <20170125095648.4116-5-patrick.steinhardt@elego.de> ``` On Thu, Jan 26, 2017 at 12:43:31PM -0800, Junio C Hamano wrote: > Patrick Steinhardt writes: > > > The URL matching function computes for two URLs whether they match not. > > The match is performed by splitting up the URL into different parts and > > then doing an exact comparison with the to-be-matched URL. > > > > The main user of `urlmatch` is the configuration subsystem. It allows to > > set certain configurations based on the URL which is being connected to > > via keys like `http..*`. A common use case for this is to set > > proxies for only some remotes which match the given URL. Unfortunately, > > having exact matches for all parts of the URL can become quite tedious > > in some setups. Imagine for example a corporate network where there are > > dozens or even hundreds of subdomains, which would have to be configured > > individually. > > > > This commit introduces the ability to use globbing in the host-part of > > the URLs. A user can simply specify a `*` as part of the host name to > > match all subdomains at this level. For example adding a configuration > > key `http.https://*.example.com.proxy` will match all subdomains of > > `https://example.com`. > > This is probably a useful improvement. > > Having said that, when I mentioned "glob", I meant to also support > something like this: > > https://www[1-4].ibm.com/ The problem with additional extended syntax like proposed by you is that we would indeed need an escaping mechanism here. '[]' are already allowed inside the host part to enable IPv6 hosts of the form 'https://[2001:0db8:]/', so the syntax is now ambiguous. So we have to be cautios which characters to enable for globbing syntax. As of now, I think we can only safely include '*' and '?' here without escaping mechanisms. If additional use cases come up we might still extend the syntax later on to allow for more special syntax. > And when people read "glob", that is what they expect. > > So calling this "the ability to use globbing" is misleading. > The last paragraph in the log message above needs a bit of > tweaking, perhaps like this: > > Allow users to write an asterisk '*' in place of any 'host' > or 'subdomain' label as part of the host name. For example, > "http.https://*.example.com.proxy" sets "http.proxy" for all > direct subdomains of "https://example.com", > e.g. "https://foo.example.com", but not > "https://foo.bar.example.com". > > Fortunately, your update to config.txt, which is facing the end > users, does not misuse the word and instead is explicit that the > only thing the matcher does is to match '*' to a single hierarchy. > It is clear that even http://www*.ibm.com/ is not supported from > the description, which is good. I agree that globbing is the wrong word here. I'll swap in "wildcard" where applicable. I'll send a version 4 later on. Thanks again for your feedback and improvements. Regards Patrick ``` ## Junio C Hamano, 2017-01-27 17:45 Subject: Re: [PATCH v3 4/4] urlmatch: allow globbing for the URL host part Message-ID: URL: https://gitlist.dev/e/xmqq1svo1jne.fsf%40gitster.mtv.corp.google.com In-Reply-To: <20170127062142.GA760@pks-pc> ``` Patrick Steinhardt writes: >> This is probably a useful improvement. >> >> Having said that, when I mentioned "glob", I meant to also support >> something like this: >> >> https://www[1-4].ibm.com/ > > The problem with additional extended syntax like proposed by you > is that we would indeed need an escaping mechanism here. True. I think a true shell globbing is overkill (so is regexp) and just a simple wildcarding with '*' would be a good first step that is easy to explain and later extend as needed. Thanks. ```