{"thread":{"id":"44952","subject":"[PATCH v1 2/2] urlmatch: allow regex-based URL matching","startedAt":"2017-01-23T13:13:04Z","lastAt":"2017-01-25T09:58:07Z","messageCount":12,"participants":["Patrick Steinhardt","Junio C Hamano","Philip Oakley"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"309974","messageId":"20170123130635.29577-3-patrick.steinhardt@elego.de","threadId":"44952","inReplyTo":"20170123130635.29577-1-patrick.steinhardt@elego.de","subject":"[PATCH v1 2/2] urlmatch: allow regex-based URL matching","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-23T13:06:35Z","receivedAt":"2017-01-23T13:13:04Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The URL matching function computes for two URLs whether they match not.\nThe match is performed by splitting up the URL into different parts and\nthen doing an exact comparison with the to-be-matched URL.\n\nThe main user of `urlmatch` is the configuration subsystem. It allows to\nset certain configurations based on the URL which is being connected to\nvia keys like `http..*`. A common use case for this is to set\nproxies for only some remotes which match the given URL. Unfortunately,\nhaving exact matches for all parts of the URL can become quite tedious\nin some setups. Imagine for example a corporate network where there are\ndozens or even hundreds of subdomains, which would have to be configured\nindividually.\n\nThis commit introduces the ability to have regex-based URL matches. A\nuser can prefix a configuration key's URL with a question mark ('?') to\nuse regular expressions instead of exact matches in order to find all\nmatching URLs. A user can now simply add a key\n`http.?http://.*\\\\.example\\\\.com.proxy` to set a proxy for all\nsubdomains of `example.com`. When no question mark is given as a prefix,\nthen the configuration subsystem will use the old algorithm based on\nexact matches.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/config.txt | 6 ++++-\n t/t1300-repo-config.sh | 31 ++++++++++++++++++++++++++\n urlmatch.c | 57 ++++++++++++++++++++++++++++++++++++++----------\n 3 files changed, 81 insertions(+), 13 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 506431267..23651b19e 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1906,7 +1906,11 @@ http.followRedirects::\n \n http..*::\n \tAny of the http.* options above can be applied selectively to some URLs.\n-\tFor a config key to match a URL, each element of the config key is\n+\tThere are two different modes to match URLs: if the config key's URL is\n+\tprefixed with a `?`, it allows to make use of regular expressions. An\n+\texample for this is `http.?http://.*\\\\.example\\\\.com.*` to match all\n+\tsubdomains of `example.com`.\n+\tIf the key is not prefixed with a `?`, each element of the config key is\n \tcompared to that of the URL, in the following order:\n +\n --\ndiff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh\nindex 923bfc5a2..fbbc58304 100755\n--- a/t/t1300-repo-config.sh\n+++ b/t/t1300-repo-config.sh\n@@ -1177,6 +1177,37 @@ test_expect_success 'urlmatch' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'regex-based urlmatch' '\n+\tcat >.git/config <<-\\EOF &&\n+\t[http]\n+\t\tsslVerify\n+\t[http \"?https://.*\\\\.example\\\\.com\"]\n+\t\tsslVerify = false\n+\t\tcookieFile = /tmp/cookie.txt\n+\tEOF\n+\n+\ttest_expect_code 1 git config --bool --get-urlmatch doesnt.exist https://good.example.com >actual &&\n+\ttest_must_be_empty actual &&\n+\n+\ttest_expect_code 1 git config --bool --get-urlmatch doesnt.exist https://good-example.com >actual &&\n+\ttest_must_be_empty actual &&\n+\n+\techo true >expect &&\n+\tgit config --bool --get-urlmatch http.SSLverify https://example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo false >expect &&\n+\tgit config --bool --get-urlmatch http.sslverify https://subdomain.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\t{\n+\t\techo http.cookiefile /tmp/cookie.txt &&\n+\t\techo http.sslverify false\n+\t} >expect &&\n+\tgit config --get-urlmatch HTTP https://subdomain.example.com >actual &&\n+\ttest_cmp expect actual\n+'\n+\n # good section hygiene\n test_expect_failure 'unsetting the last key in a section removes header' '\n \tcat >.git/config <<-\\EOF &&\ndiff --git a/urlmatch.c b/urlmatch.c\nindex 132d342bc..8ed5047e3 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -490,18 +490,51 @@ int urlmatch_config_entry(const char *var, const char *value, void *cb)\n \t}\n \tdot = strrchr(key, '.');\n \tif (dot) {\n-\t\tchar *config_url, *norm_url;\n-\t\tstruct url_info norm_info;\n-\n-\t\tconfig_url = xmemdupz(key, dot - key);\n-\t\tnorm_url = url_normalize(config_url, &norm_info);\n-\t\tfree(config_url);\n-\t\tif (!norm_url)\n-\t\t\treturn 0;\n-\t\tmatched_len = match_urls(url, &norm_info, &user_matched);\n-\t\tfree(norm_url);\n-\t\tif (!matched_len)\n-\t\t\treturn 0;\n+\t\t/*\n+\t\t * When the configuration key's URL is prefixed\n+\t\t * with a '?', regular expressions are enabled to\n+\t\t * match the URL instead of the exact-match\n+\t\t * algorithm.\n+\t\t */\n+\t\tif (starts_with(key, \"?\")) {\n+\t\t\tchar *config_url;\n+\t\t\tregex_t reg;\n+\t\t\tint status;\n+\n+\t\t\tconfig_url = xmemdupz(key + 1, dot - key - 1);\n+\t\t\tif (regcomp(®, config_url, REG_EXTENDED)) {\n+\t\t\t\twarning(_(\"Cannot prepare URL regexp %s\"),\n+\t\t\t\t\tconfig_url);\n+\t\t\t\tfree(config_url);\n+\t\t\t\treturn 0;\n+\t\t\t}\n+\n+\t\t\tstatus = regexec(®, url->url, 0, NULL, 0);\n+\t\t\tfree(config_url);\n+\t\t\tregfree(®);\n+\n+\t\t\tif (status) {\n+\t\t\t\t if (status != REG_NOMATCH)\n+\t\t\t\t\twarning(_(\"regexec returned %d for input '%s'\"),\n+\t\t\t\t\t\tstatus, url->url);\n+\n+\t\t\t\treturn 0;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tchar *config_url, *norm_url;\n+\t\t\tstruct url_info norm_info;\n+\n+\t\t\tconfig_url = xmemdupz(key, dot - key);\n+\t\t\tnorm_url = url_normalize(config_url, &norm_info);\n+\t\t\tfree(config_url);\n+\t\t\tif (!norm_url)\n+\t\t\t\treturn 0;\n+\t\t\tmatched_len = match_urls(url, &norm_info, &user_matched);\n+\t\t\tfree(norm_url);\n+\t\t\tif (!matched_len)\n+\t\t\t\treturn 0;\n+\t\t}\n+\n \t\tkey = dot + 1;\n \t}\n \n-- \n2.11.0\n\n"},{"id":"309975","messageId":"20170123130635.29577-2-patrick.steinhardt@elego.de","threadId":"44952","inReplyTo":"20170123130635.29577-1-patrick.steinhardt@elego.de","subject":"[PATCH v1 1/2] mailmap: add Patrick Steinhardt's work address","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-23T13:06:34Z","receivedAt":"2017-01-23T13:13:08Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Signed-off-by: Patrick Steinhardt \n---\n .mailmap | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/.mailmap b/.mailmap\nindex 9c87a3840..ea59205b9 100644\n--- a/.mailmap\n+++ b/.mailmap\n@@ -177,6 +177,7 @@ Paolo Bonzini \n Pascal Obry \n Pascal Obry \n Pat Notz \n+Patrick Steinhardt \n Paul Mackerras \n Paul Mackerras \n Peter Baumann \n-- \n2.11.0\n\n"},{"id":"309976","messageId":"20170123130635.29577-1-patrick.steinhardt@elego.de","threadId":"44952","inReplyTo":null,"subject":"[PATCH v1 0/2] urlmatch: allow regexp-based matches","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-23T13:06:33Z","receivedAt":"2017-01-23T13:13:09Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nShort disclaimer: this patch results from work for a client at my\nday job at elego Software Solutions GmbH. As such, I'm using my\nwork mail address and added a new mailmap entry. I wasn't exactly\ncertain if the mailmap entry should've been created in a separate\ncommit series, as it has nothing to do with the actual topic --\nI can re-send it separately if requested.\n\nThis patch is mostly a request for comments. The use case is to\nbe able to configure an HTTP proxy for all subdomains of a\ncertain domain where there are hundreds of subdomains. The most\nflexible way I could imagine was by using regular expressions for\nthe matching, which is how I implemented it for now. So users can\nnow create a configuration key like\n`http.?http://.*\\\\.example\\\\.com.*` to apply settings for all\nsubdomains of `example.com`.\n\nI tried to make this feature as backwards-compatible as it can be\nby having the '?' prefix. Older clients will barf when trying to\nnormalize the URL as '?' is not in the set of allowed characters\nfor a URL, and for newer clients there will be no change in\nbehavior for previously configured `http..*` keys.\n\nRegards\nPatrick Steinhardt\n\nPatrick Steinhardt (2):\n mailmap: add Patrick Steinhardt's work address\n urlmatch: allow regex-based URL matching\n\n .mailmap | 1 +\n Documentation/config.txt | 6 ++++-\n t/t1300-repo-config.sh | 31 ++++++++++++++++++++++++++\n urlmatch.c | 57 ++++++++++++++++++++++++++++++++++++++----------\n 4 files changed, 82 insertions(+), 13 deletions(-)\n\n-- \n2.11.0\n\n"},{"id":"310004","messageId":"xmqqsho9r1rs.fsf@gitster.mtv.corp.google.com","threadId":"44952","inReplyTo":"20170123130635.29577-1-patrick.steinhardt@elego.de","subject":"Re: [PATCH v1 0/2] urlmatch: allow regexp-based matches","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-01-23T19:53:43Z","receivedAt":"2017-01-23T19:53:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> This patch is mostly a request for comments. The use case is to\n> be able to configure an HTTP proxy for all subdomains of a\n> certain domain where there are hundreds of subdomains. The most\n> flexible way I could imagine was by using regular expressions for\n> the matching, which is how I implemented it for now. So users can\n> now create a configuration key like\n> `http.?http://.*\\\\.example\\\\.com.*` to apply settings for all\n> subdomains of `example.com`.\n\nWhile reading 2/2, I got an impression that this is \"too\" flexible\nand possibly operates at a wrong level. I would have expected that\nthe wildcarding to be limited to the host part only and hook into\nmatch_urls(), allowing the users of the new feature to still take\nadvantage of the existing support of \"http://me@example.com\" that\nlimits the match to the case that the connection is authenticated\nfor a user, for example, by newly allowing \"http://me@*.example.com\"\nor something like that.\n\nBecause you cannot have a literal '*' in your hostname, I would\nimagine that supporting a match pattern \"http://me@*.example.com\"\nwould be already backward compatible without requiring a leading\nquestion-mark.\n\nI also personally would prefer these textual matching to be done\nwith glob not with regexp, by the way, as the above description of\nmine shows.\n\nThanks.\n\n\n"},{"id":"310084","messageId":"20170124112952.GA2603@pks-pc","threadId":"44952","inReplyTo":"xmqqsho9r1rs.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v1 0/2] urlmatch: allow regexp-based matches","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-24T11:29:52Z","receivedAt":"2017-01-24T11:35:51Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Jan 23, 2017 at 11:53:43AM -0800, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> > This patch is mostly a request for comments. The use case is to\n> > be able to configure an HTTP proxy for all subdomains of a\n> > certain domain where there are hundreds of subdomains. The most\n> > flexible way I could imagine was by using regular expressions for\n> > the matching, which is how I implemented it for now. So users can\n> > now create a configuration key like\n> > `http.?http://.*\\\\.example\\\\.com.*` to apply settings for all\n> > subdomains of `example.com`.\n> \n> While reading 2/2, I got an impression that this is \"too\" flexible\n> and possibly operates at a wrong level. I would have expected that\n> the wildcarding to be limited to the host part only and hook into\n> match_urls(), allowing the users of the new feature to still take\n> advantage of the existing support of \"http://me@example.com\" that\n> limits the match to the case that the connection is authenticated\n> for a user, for example, by newly allowing \"http://me@*.example.com\"\n> or something like that.\n> \n> Because you cannot have a literal '*' in your hostname, I would\n> imagine that supporting a match pattern \"http://me@*.example.com\"\n> would be already backward compatible without requiring a leading\n> question-mark.\n> \n> I also personally would prefer these textual matching to be done\n> with glob not with regexp, by the way, as the above description of\n> mine shows.\n> \n> Thanks.\n\nThanks for your feedback. Using globs in the hostname only was my\nfirst intent, as well. I later on took regular expressions\ninstead so as to allow further flexibility for the user. The\nreasoning was that there might be other use cases which cannot\nactually be solved with using globs only, even if I myself wasn't\naware of different ones. So this might be indeed over-engineered\nwhen using regular expressions.\n\nThere are several questions though regarding semantics with\nglobs, where I'd like to have additional opinions on.\n\n- should a glob only be allowed for actual subdomains, allowing\n \"http://*.example.com\" but not \"http://*example.com\"?\n\n- should a glob also match multiple nestings of subdomains? E.g.\n \"http://*.example.com\" would match \"http://foo.example.com\" but\n not \"http://foo.bar.example.com\"?\n\nI'll send a version 2 soon-ish.\n\nRegards\nPatrick\n"},{"id":"310092","messageId":"20170124170031.18069-1-patrick.steinhardt@elego.de","threadId":"44952","inReplyTo":"20170123130635.29577-1-patrick.steinhardt@elego.de","subject":"[PATCH v2 0/4] urlmatch: allow regexp-based matches","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-24T17:00:27Z","receivedAt":"2017-01-24T17:00:57Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nThis is version two of my patch series.\n\nThe use case is to be able to configure an HTTP proxy for all\nsubdomains of a domain where there are hundreds of subdomains.\n\nPreviously, I have been using complete regular expressions with\nan escape-mechanism to match the configuration key's URLs.\nAccording to Junio's comments, I changed this mechanism to a much\nsimpler one, where the user is only allowed to use globbing for\nthe host part of the URL. That is a user can now specify a key\n`http.https://*.example.com` to match all sub-domains of\n`example.com`. For now I've decided to implement it such that a\nsingle `*` matches a single subdomain only, so for example\n`https://foo.bar.example.com` would not match in this case. This\nis similar to how shell-globbing works usually, so it should not\nbe of much surprise. It's also highlighted in the documentation.\n\nI did not include an interdiff as too much has changed between\nthe two versions.\n\nRegards\nPatrick\n\nPatrick Steinhardt (4):\n mailmap: add Patrick Steinhardt's work address\n urlmatch: enable normalization of URLs with globs\n urlmatch: split host and port fields in `struct url_info`\n urlmatch: allow globbing for the URL host part\n\n .mailmap | 1 +\n Documentation/config.txt | 5 +++-\n t/t1300-repo-config.sh | 36 +++++++++++++++++++++++++\n urlmatch.c | 68 +++++++++++++++++++++++++++++++++++++++++-------\n urlmatch.h | 9 ++++---\n 5 files changed, 104 insertions(+), 15 deletions(-)\n\n-- \n2.11.0\n\n"},{"id":"310093","messageId":"20170124170031.18069-2-patrick.steinhardt@elego.de","threadId":"44952","inReplyTo":"20170124170031.18069-1-patrick.steinhardt@elego.de","subject":"[PATCH v2 1/4] mailmap: add Patrick Steinhardt's work address","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-24T17:00:28Z","receivedAt":"2017-01-24T17:01:02Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Signed-off-by: Patrick Steinhardt \n---\n .mailmap | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/.mailmap b/.mailmap\nindex 9c87a3840..ea59205b9 100644\n--- a/.mailmap\n+++ b/.mailmap\n@@ -177,6 +177,7 @@ Paolo Bonzini \n Pascal Obry \n Pascal Obry \n Pat Notz \n+Patrick Steinhardt \n Paul Mackerras \n Paul Mackerras \n Peter Baumann \n-- \n2.11.0\n\n"},{"id":"310094","messageId":"20170124170031.18069-3-patrick.steinhardt@elego.de","threadId":"44952","inReplyTo":"20170124170031.18069-1-patrick.steinhardt@elego.de","subject":"[PATCH v2 2/4] urlmatch: enable normalization of URLs with globs","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-24T17:00:29Z","receivedAt":"2017-01-24T17:01:06Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `url_normalize` function is used to validate and normalize URLs. As\nsuch, it does not allow for some special characters to be part of the\nURLs that are to be normalized. As we want to allow using globs in some\nconfiguration keys making use of URLs, namely `http..`, but\nstill normalize them, we need to somehow enable some additional allowed\ncharacters.\n\nTo do this without having to change all callers of `url_normalize`,\nwhere most do not actually want globbing at all, we split off another\nfunction `url_normalize_1`. This function accepts an additional\nparameter `allow_globs`, which is subsequently called by `url_normalize`\nwith `allow_globs=0`.\n\nAs of now, this function is not used with globbing enabled. A caller\nwill be added in the following commit.\n\nSigned-off-by: Patrick Steinhardt \n---\n urlmatch.c | 14 ++++++++++++--\n 1 file changed, 12 insertions(+), 2 deletions(-)\n\ndiff --git a/urlmatch.c b/urlmatch.c\nindex 132d342bc..d350478c0 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -63,7 +63,7 @@ static int append_normalized_escapes(struct strbuf *buf,\n \treturn 1;\n }\n \n-char *url_normalize(const char *url, struct url_info *out_info)\n+static char *url_normalize_1(const char *url, struct url_info *out_info, char allow_globs)\n {\n \t/*\n \t * Normalize NUL-terminated url using the following rules:\n@@ -191,7 +191,12 @@ char *url_normalize(const char *url, struct url_info *out_info)\n \t\tstrbuf_release(&norm);\n \t\treturn NULL;\n \t}\n-\tspanned = strspn(url, URL_HOST_CHARS);\n+\n+\tif (allow_globs)\n+\t\tspanned = strspn(url, URL_HOST_CHARS \"*\");\n+\telse\n+\t\tspanned = strspn(url, URL_HOST_CHARS);\n+\n \tif (spanned < colon_ptr - url) {\n \t\t/* Host name has invalid characters */\n \t\tif (out_info) {\n@@ -380,6 +385,11 @@ char *url_normalize(const char *url, struct url_info *out_info)\n \treturn result;\n }\n \n+char *url_normalize(const char *url, struct url_info *out_info)\n+{\n+\treturn url_normalize_1(url, out_info, 0);\n+}\n+\n static size_t url_match_prefix(const char *url,\n \t\t\t const char *url_prefix,\n \t\t\t size_t url_prefix_len)\n-- \n2.11.0\n\n"},{"id":"310095","messageId":"20170124170031.18069-4-patrick.steinhardt@elego.de","threadId":"44952","inReplyTo":"20170124170031.18069-1-patrick.steinhardt@elego.de","subject":"[PATCH v2 3/4] urlmatch: split host and port fields in `struct url_info`","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-24T17:00:30Z","receivedAt":"2017-01-24T17:01:11Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `url_info` structure contains information about a normalized URL\nwith the URL's components being represented by different fields. The\nhost and port part though are to be accessed by the same `host` field,\nso that getting the host and/or port separately becomes more involved\nthan really necessary.\n\nTo make the port more readily accessible, split up the host and port\nfields. Namely, the `host_len` will not include the port length anymore\nand a new `port_off` field has been added which includes the offset to\nthe port, if available.\n\nThe only user of these fields is `url_normalize_1`. This change makes it\neasier later on to treat host and port differently when introducing\nglobs for domains.\n\nSigned-off-by: Patrick Steinhardt \n---\n urlmatch.c | 16 ++++++++++++----\n urlmatch.h | 9 +++++----\n 2 files changed, 17 insertions(+), 8 deletions(-)\n\ndiff --git a/urlmatch.c b/urlmatch.c\nindex d350478c0..e328905eb 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -104,7 +104,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al\n \tstruct strbuf norm;\n \tsize_t spanned;\n \tsize_t scheme_len, user_off=0, user_len=0, passwd_off=0, passwd_len=0;\n-\tsize_t host_off=0, host_len=0, port_len=0, path_off, path_len, result_len;\n+\tsize_t host_off=0, host_len=0, port_off=0, port_len=0, path_off, path_len, result_len;\n \tconst char *slash_ptr, *at_ptr, *colon_ptr, *path_start;\n \tchar *result;\n \n@@ -263,6 +263,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al\n \t\t\t\treturn NULL;\n \t\t\t}\n \t\t\tstrbuf_addch(&norm, ':');\n+\t\t\tport_off = norm.len;\n \t\t\tstrbuf_add(&norm, url, slash_ptr - url);\n \t\t\tport_len = slash_ptr - url;\n \t\t}\n@@ -270,7 +271,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al\n \t\turl = slash_ptr;\n \t}\n \tif (host_off)\n-\t\thost_len = norm.len - host_off;\n+\t\thost_len = norm.len - host_off - (port_len ? port_len + 1 : 0);\n \n \n \t/*\n@@ -378,6 +379,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al\n \t\tout_info->passwd_len = passwd_len;\n \t\tout_info->host_off = host_off;\n \t\tout_info->host_len = host_len;\n+\t\tout_info->port_off = port_off;\n \t\tout_info->port_len = port_len;\n \t\tout_info->path_off = path_off;\n \t\tout_info->path_len = path_len;\n@@ -464,11 +466,17 @@ static int match_urls(const struct url_info *url,\n \t\tusermatched = 1;\n \t}\n \n-\t/* check the host and port */\n+\t/* check the host */\n \tif (url_prefix->host_len != url->host_len ||\n \t strncmp(url->url + url->host_off,\n \t\t url_prefix->url + url_prefix->host_off, url->host_len))\n-\t\treturn 0; /* host names and/or ports do not match */\n+\t\treturn 0; /* host names do not match */\n+\n+\t/* check the port */\n+\tif (url_prefix->port_len != url->port_len ||\n+\t strncmp(url->url + url->port_off,\n+\t\t url_prefix->url + url_prefix->port_off, url->port_len))\n+\t\treturn 0; /* ports do not match */\n \n \t/* check the path */\n \tpathmatchlen = url_match_prefix(\ndiff --git a/urlmatch.h b/urlmatch.h\nindex 528862adc..0ea812b03 100644\n--- a/urlmatch.h\n+++ b/urlmatch.h\n@@ -18,11 +18,12 @@ struct url_info {\n \tsize_t passwd_len;\t/* length of passwd; if passwd_off != 0 but\n \t\t\t\t passwd_len == 0, an empty passwd was given */\n \tsize_t host_off;\t/* offset into url to start of host name (0 => none) */\n-\tsize_t host_len;\t/* length of host name; this INCLUDES any ':portnum';\n+\tsize_t host_len;\t/* length of host name;\n \t\t\t\t * file urls may have host_len == 0 */\n-\tsize_t port_len;\t/* if a portnum is present (port_len != 0), it has\n-\t\t\t\t * this length (excluding the leading ':') at the\n-\t\t\t\t * end of the host name (always 0 for file urls) */\n+\tsize_t port_off;\t/* offset into url to start of port number (0 => none) */\n+\tsize_t port_len;\t/* if a portnum is present (port_off != 0), it has\n+\t\t\t\t * this length (excluding the leading ':') starting\n+\t\t\t\t * from port_off (always 0 for file urls) */\n \tsize_t path_off;\t/* offset into url to the start of the url path;\n \t\t\t\t * this will always point to a '/' character\n \t\t\t\t * after the url has been normalized */\n-- \n2.11.0\n\n"},{"id":"310096","messageId":"20170124170031.18069-5-patrick.steinhardt@elego.de","threadId":"44952","inReplyTo":"20170124170031.18069-1-patrick.steinhardt@elego.de","subject":"[PATCH v2 4/4] urlmatch: allow globbing for the URL host part","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-24T17:00:31Z","receivedAt":"2017-01-24T17:01:13Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The URL matching function computes for two URLs whether they match not.\nThe match is performed by splitting up the URL into different parts and\nthen doing an exact comparison with the to-be-matched URL.\n\nThe main user of `urlmatch` is the configuration subsystem. It allows to\nset certain configurations based on the URL which is being connected to\nvia keys like `http..*`. A common use case for this is to set\nproxies for only some remotes which match the given URL. Unfortunately,\nhaving exact matches for all parts of the URL can become quite tedious\nin some setups. Imagine for example a corporate network where there are\ndozens or even hundreds of subdomains, which would have to be configured\nindividually.\n\nThis commit introduces the ability to use globbing in the host-part of\nthe URLs. A user can simply specify a `*` as part of the host name to\nmatch all subdomains at this level. For example adding a configuration\nkey `http.https://*.example.com.proxy` will match all subdomains of\n`https://example.com`.\n\nSigned-off-by: Patrick Steinhardt \n---\n Documentation/config.txt | 5 ++++-\n t/t1300-repo-config.sh | 36 ++++++++++++++++++++++++++++++++++++\n urlmatch.c | 38 ++++++++++++++++++++++++++++++++++----\n 3 files changed, 74 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 506431267..a78921c2b 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1914,7 +1914,10 @@ http..*::\n must match exactly between the config key and the URL.\n \n . Host/domain name (e.g., `example.com` in `https://example.com/`).\n- This field must match exactly between the config key and the URL.\n+ This field must match between the config key and the URL. It is\n+ possible to use globs in the config key to match all subdomains, e.g.\n+ `https://*.example.com/` to match all subdomains of `example.com`. Note\n+ that a glob only every matches a single part of the hostname.\n \n . Port number (e.g., `8080` in `http://example.com:8080/`).\n This field must match exactly between the config key and the URL.\ndiff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh\nindex 923bfc5a2..ec545e092 100755\n--- a/t/t1300-repo-config.sh\n+++ b/t/t1300-repo-config.sh\n@@ -1177,6 +1177,42 @@ test_expect_success 'urlmatch' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'glob-based urlmatch' '\n+\tcat >.git/config <<-\\EOF &&\n+\t[http]\n+\t\tsslVerify\n+\t[http \"https://*.example.com\"]\n+\t\tsslVerify = false\n+\t\tcookieFile = /tmp/cookie.txt\n+\tEOF\n+\n+\ttest_expect_code 1 git config --bool --get-urlmatch doesnt.exist https://good.example.com >actual &&\n+\ttest_must_be_empty actual &&\n+\n+\techo true >expect &&\n+\tgit config --bool --get-urlmatch http.SSLverify https://example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo true >expect &&\n+\tgit config --bool --get-urlmatch http.SSLverify https://good-example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo true >expect &&\n+\tgit config --bool --get-urlmatch http.sslverify https://deep.nested.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo false >expect &&\n+\tgit config --bool --get-urlmatch http.sslverify https://good.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\t{\n+\t\techo http.cookiefile /tmp/cookie.txt &&\n+\t\techo http.sslverify false\n+\t} >expect &&\n+\tgit config --get-urlmatch HTTP https://good.example.com >actual &&\n+\ttest_cmp expect actual\n+'\n+\n # good section hygiene\n test_expect_failure 'unsetting the last key in a section removes header' '\n \tcat >.git/config <<-\\EOF &&\ndiff --git a/urlmatch.c b/urlmatch.c\nindex e328905eb..53ff972a6 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -63,6 +63,38 @@ static int append_normalized_escapes(struct strbuf *buf,\n \treturn 1;\n }\n \n+static int match_host(const struct url_info *url_info,\n+\t\t const struct url_info *pattern_info)\n+{\n+\tchar *url = xmemdupz(url_info->url + url_info->host_off, url_info->host_len);\n+\tchar *pat = xmemdupz(pattern_info->url + pattern_info->host_off, pattern_info->host_len);\n+\tchar *url_tok, *pat_tok, *url_save, *pat_save;\n+\tint matching;\n+\n+\turl_tok = strtok_r(url, \".\", &url_save);\n+\tpat_tok = strtok_r(pat, \".\", &pat_save);\n+\n+\tfor (; url_tok && pat_tok; url_tok = strtok_r(NULL, \".\", &url_save),\n+\t\t\t\t pat_tok = strtok_r(NULL, \".\", &pat_save)) {\n+\t\tif (!strcmp(pat_tok, \"*\"))\n+\t\t\tcontinue; /* a simple glob matches everything */\n+\n+\t\tif (strcmp(url_tok, pat_tok)) {\n+\t\t\t/* subdomains do not match */\n+\t\t\tmatching = 0;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\n+\t/* matching if both URL and pattern are at their ends */\n+\tmatching = (url_tok == NULL && pat_tok == NULL);\n+\n+\tfree(url);\n+\tfree(pat);\n+\n+\treturn matching;\n+}\n+\n static char *url_normalize_1(const char *url, struct url_info *out_info, char allow_globs)\n {\n \t/*\n@@ -467,9 +499,7 @@ static int match_urls(const struct url_info *url,\n \t}\n \n \t/* check the host */\n-\tif (url_prefix->host_len != url->host_len ||\n-\t strncmp(url->url + url->host_off,\n-\t\t url_prefix->url + url_prefix->host_off, url->host_len))\n+\tif (!match_host(url, url_prefix))\n \t\treturn 0; /* host names do not match */\n \n \t/* check the port */\n@@ -512,7 +542,7 @@ int urlmatch_config_entry(const char *var, const char *value, void *cb)\n \t\tstruct url_info norm_info;\n \n \t\tconfig_url = xmemdupz(key, dot - key);\n-\t\tnorm_url = url_normalize(config_url, &norm_info);\n+\t\tnorm_url = url_normalize_1(config_url, &norm_info, 1);\n \t\tfree(config_url);\n \t\tif (!norm_url)\n \t\t\treturn 0;\n-- \n2.11.0\n\n"},{"id":"310099","messageId":"5BF60E6DF2C04DF081466BC3BD3F954F@PhilipOakley","threadId":"44952","inReplyTo":"20170124170031.18069-5-patrick.steinhardt@elego.de","subject":"Re: [PATCH v2 4/4] urlmatch: allow globbing for the URL host part","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":null,"receivedAt":"2017-01-24T17:52:45Z","isPatch":true,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Patrick Steinhardt\" \n\na quick comment on the documentation part ..\n\n> The URL matching function computes for two URLs whether they match not.\n> The match is performed by splitting up the URL into different parts and\n> then doing an exact comparison with the to-be-matched URL.\n>\n> The main user of `urlmatch` is the configuration subsystem. It allows to\n> set certain configurations based on the URL which is being connected to\n> via keys like `http..*`. A common use case for this is to set\n> proxies for only some remotes which match the given URL. Unfortunately,\n> having exact matches for all parts of the URL can become quite tedious\n> in some setups. Imagine for example a corporate network where there are\n> dozens or even hundreds of subdomains, which would have to be configured\n> individually.\n>\n> This commit introduces the ability to use globbing in the host-part of\n> the URLs. A user can simply specify a `*` as part of the host name to\n> match all subdomains at this level. For example adding a configuration\n> key `http.https://*.example.com.proxy` will match all subdomains of\n> `https://example.com`.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> Documentation/config.txt | 5 ++++-\n> t/t1300-repo-config.sh | 36 ++++++++++++++++++++++++++++++++++++\n> urlmatch.c | 38 ++++++++++++++++++++++++++++++++++----\n> 3 files changed, 74 insertions(+), 5 deletions(-)\n>\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 506431267..a78921c2b 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -1914,7 +1914,10 @@ http..*::\n> must match exactly between the config key and the URL.\n>\n> . Host/domain name (e.g., `example.com` in `https://example.com/`).\n> - This field must match exactly between the config key and the URL.\n> + This field must match between the config key and the URL. It is\n> + possible to use globs in the config key to match all subdomains, e.g.\n> + `https://*.example.com/` to match all subdomains of `example.com`. Note\n> + that a glob only every matches a single part of the hostname.\n\n[s/every/ever/ ?]\n\nthe \"match all subdomains\" appears to contradict the \"a glob only ever \nmatches a single part \".\n\nMaybe borrow the example from the 0/4 cover letter\n\"so for example `https://foo.bar.example.com` would not match in the case of \n`http.https://*.example.com` \" (If I understood it correctly.\n\nA simple example often clarifies much better than more words.\n--\nPhilip\n\n>\n> . Port number (e.g., `8080` in `http://example.com:8080/`).\n> This field must match exactly between the config key and the URL.\n> diff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh\n> index 923bfc5a2..ec545e092 100755\n> --- a/t/t1300-repo-config.sh\n> +++ b/t/t1300-repo-config.sh\n> @@ -1177,6 +1177,42 @@ test_expect_success 'urlmatch' '\n> test_cmp expect actual\n> '\n>\n> +test_expect_success 'glob-based urlmatch' '\n> + cat >.git/config <<-\\EOF &&\n> + [http]\n> + sslVerify\n> + [http \"https://*.example.com\"]\n> + sslVerify = false\n> + cookieFile = /tmp/cookie.txt\n> + EOF\n> +\n> + test_expect_code 1 git config --bool --get-urlmatch doesnt.exist \n> https://good.example.com >actual &&\n> + test_must_be_empty actual &&\n> +\n> + echo true >expect &&\n> + git config --bool --get-urlmatch http.SSLverify https://example.com \n> >actual &&\n> + test_cmp expect actual &&\n> +\n> + echo true >expect &&\n> + git config --bool --get-urlmatch http.SSLverify https://good-example.com \n> >actual &&\n> + test_cmp expect actual &&\n> +\n> + echo true >expect &&\n> + git config --bool --get-urlmatch http.sslverify \n> https://deep.nested.example.com >actual &&\n> + test_cmp expect actual &&\n> +\n> + echo false >expect &&\n> + git config --bool --get-urlmatch http.sslverify https://good.example.com \n> >actual &&\n> + test_cmp expect actual &&\n> +\n> + {\n> + echo http.cookiefile /tmp/cookie.txt &&\n> + echo http.sslverify false\n> + } >expect &&\n> + git config --get-urlmatch HTTP https://good.example.com >actual &&\n> + test_cmp expect actual\n> +'\n> +\n> # good section hygiene\n> test_expect_failure 'unsetting the last key in a section removes header' '\n> cat >.git/config <<-\\EOF &&\n> diff --git a/urlmatch.c b/urlmatch.c\n> index e328905eb..53ff972a6 100644\n> --- a/urlmatch.c\n> +++ b/urlmatch.c\n> @@ -63,6 +63,38 @@ static int append_normalized_escapes(struct strbuf \n> *buf,\n> return 1;\n> }\n>\n> +static int match_host(const struct url_info *url_info,\n> + const struct url_info *pattern_info)\n> +{\n> + char *url = xmemdupz(url_info->url + url_info->host_off, \n> url_info->host_len);\n> + char *pat = xmemdupz(pattern_info->url + pattern_info->host_off, \n> pattern_info->host_len);\n> + char *url_tok, *pat_tok, *url_save, *pat_save;\n> + int matching;\n> +\n> + url_tok = strtok_r(url, \".\", &url_save);\n> + pat_tok = strtok_r(pat, \".\", &pat_save);\n> +\n> + for (; url_tok && pat_tok; url_tok = strtok_r(NULL, \".\", &url_save),\n> + pat_tok = strtok_r(NULL, \".\", &pat_save)) {\n> + if (!strcmp(pat_tok, \"*\"))\n> + continue; /* a simple glob matches everything */\n> +\n> + if (strcmp(url_tok, pat_tok)) {\n> + /* subdomains do not match */\n> + matching = 0;\n> + break;\n> + }\n> + }\n> +\n> + /* matching if both URL and pattern are at their ends */\n> + matching = (url_tok == NULL && pat_tok == NULL);\n> +\n> + free(url);\n> + free(pat);\n> +\n> + return matching;\n> +}\n> +\n> static char *url_normalize_1(const char *url, struct url_info *out_info, \n> char allow_globs)\n> {\n> /*\n> @@ -467,9 +499,7 @@ static int match_urls(const struct url_info *url,\n> }\n>\n> /* check the host */\n> - if (url_prefix->host_len != url->host_len ||\n> - strncmp(url->url + url->host_off,\n> - url_prefix->url + url_prefix->host_off, url->host_len))\n> + if (!match_host(url, url_prefix))\n> return 0; /* host names do not match */\n>\n> /* check the port */\n> @@ -512,7 +542,7 @@ int urlmatch_config_entry(const char *var, const char \n> *value, void *cb)\n> struct url_info norm_info;\n>\n> config_url = xmemdupz(key, dot - key);\n> - norm_url = url_normalize(config_url, &norm_info);\n> + norm_url = url_normalize_1(config_url, &norm_info, 1);\n> free(config_url);\n> if (!norm_url)\n> return 0;\n> -- \n> 2.11.0\n>\n> \n\n"},{"id":"310149","messageId":"20170125095754.GA4096@pks-pc","threadId":"44952","inReplyTo":"5BF60E6DF2C04DF081466BC3BD3F954F@PhilipOakley","subject":"Re: [PATCH v2 4/4] urlmatch: allow globbing for the URL host part","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-25T09:57:54Z","receivedAt":"2017-01-25T09:58:07Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Jan 24, 2017 at 05:52:39PM -0000, Philip Oakley wrote:\n> From: \"Patrick Steinhardt\" \n> \n> a quick comment on the documentation part ..\n> \n> > The URL matching function computes for two URLs whether they match not.\n> > The match is performed by splitting up the URL into different parts and\n> > then doing an exact comparison with the to-be-matched URL.\n> >\n> > The main user of `urlmatch` is the configuration subsystem. It allows to\n> > set certain configurations based on the URL which is being connected to\n> > via keys like `http..*`. A common use case for this is to set\n> > proxies for only some remotes which match the given URL. Unfortunately,\n> > having exact matches for all parts of the URL can become quite tedious\n> > in some setups. Imagine for example a corporate network where there are\n> > dozens or even hundreds of subdomains, which would have to be configured\n> > individually.\n> >\n> > This commit introduces the ability to use globbing in the host-part of\n> > the URLs. A user can simply specify a `*` as part of the host name to\n> > match all subdomains at this level. For example adding a configuration\n> > key `http.https://*.example.com.proxy` will match all subdomains of\n> > `https://example.com`.\n> >\n> > Signed-off-by: Patrick Steinhardt \n> > ---\n> > Documentation/config.txt | 5 ++++-\n> > t/t1300-repo-config.sh | 36 ++++++++++++++++++++++++++++++++++++\n> > urlmatch.c | 38 ++++++++++++++++++++++++++++++++++----\n> > 3 files changed, 74 insertions(+), 5 deletions(-)\n> >\n> > diff --git a/Documentation/config.txt b/Documentation/config.txt\n> > index 506431267..a78921c2b 100644\n> > --- a/Documentation/config.txt\n> > +++ b/Documentation/config.txt\n> > @@ -1914,7 +1914,10 @@ http..*::\n> > must match exactly between the config key and the URL.\n> >\n> > . Host/domain name (e.g., `example.com` in `https://example.com/`).\n> > - This field must match exactly between the config key and the URL.\n> > + This field must match between the config key and the URL. It is\n> > + possible to use globs in the config key to match all subdomains, e.g.\n> > + `https://*.example.com/` to match all subdomains of `example.com`. Note\n> > + that a glob only every matches a single part of the hostname.\n> \n> [s/every/ever/ ?]\n> \n> the \"match all subdomains\" appears to contradict the \"a glob only ever \n> matches a single part \".\n> \n> Maybe borrow the example from the 0/4 cover letter\n> \"so for example `https://foo.bar.example.com` would not match in the case of \n> `http.https://*.example.com` \" (If I understood it correctly.\n> \n> A simple example often clarifies much better than more words.\n\nThanks! (Hopefully) improved this in v3.\n\nRegards\nPatrick\n"}]}