{"thread":{"id":"44986","subject":"[PATCH v4 1/5] mailmap: add Patrick Steinhardt's work address","startedAt":"2017-01-27T10:35:32Z","lastAt":"2017-01-31T08:34:41Z","messageCount":9,"participants":["Patrick Steinhardt","Junio C Hamano"],"isPatch":true,"patchVersion":4,"patchTotal":5},"messages":[{"id":"310394","messageId":"a17ec3bede760c62afec4058fecc6b40b433acc2.1485512626.git.patrick.steinhardt@elego.de","threadId":"44986","inReplyTo":"cover.1485512626.git.patrick.steinhardt@elego.de","subject":"[PATCH v4 1/5] mailmap: add Patrick Steinhardt's work address","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-27T10:32:14Z","receivedAt":"2017-01-27T10:35:32Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Signed-off-by: Patrick Steinhardt \n---\n .mailmap | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/.mailmap b/.mailmap\nindex 9c87a3840..ea59205b9 100644\n--- a/.mailmap\n+++ b/.mailmap\n@@ -177,6 +177,7 @@ Paolo Bonzini \n Pascal Obry \n Pascal Obry \n Pat Notz \n+Patrick Steinhardt \n Paul Mackerras \n Paul Mackerras \n Peter Baumann \n-- \n2.11.0\n\n"},{"id":"310395","messageId":"3f0b0ba5f122d0098f1966de0f3a8cdaf8cd15f2.1485512626.git.patrick.steinhardt@elego.de","threadId":"44986","inReplyTo":"cover.1485512626.git.patrick.steinhardt@elego.de","subject":"[PATCH v4 2/5] urlmatch: enable normalization of URLs with globs","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-27T10:32:15Z","receivedAt":"2017-01-27T10:35:35Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `url_normalize` function is used to validate and normalize URLs. As\nsuch, it does not allow for some special characters to be part of the\nURLs that are to be normalized. As we want to allow using globs in some\nconfiguration keys making use of URLs, namely `http..`, but\nstill normalize them, we need to somehow enable some additional allowed\ncharacters.\n\nTo do this without having to change all callers of `url_normalize`,\nwhere most do not actually want globbing at all, we split off another\nfunction `url_normalize_1`. This function accepts an additional\nparameter `allow_globs`, which is subsequently called by `url_normalize`\nwith `allow_globs=0`.\n\nAs of now, this function is not used with globbing enabled. A caller\nwill be added in the following commit.\n\nSigned-off-by: Patrick Steinhardt \n---\n urlmatch.c | 14 ++++++++++++--\n 1 file changed, 12 insertions(+), 2 deletions(-)\n\ndiff --git a/urlmatch.c b/urlmatch.c\nindex 132d342bc..d350478c0 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -63,7 +63,7 @@ static int append_normalized_escapes(struct strbuf *buf,\n \treturn 1;\n }\n \n-char *url_normalize(const char *url, struct url_info *out_info)\n+static char *url_normalize_1(const char *url, struct url_info *out_info, char allow_globs)\n {\n \t/*\n \t * Normalize NUL-terminated url using the following rules:\n@@ -191,7 +191,12 @@ char *url_normalize(const char *url, struct url_info *out_info)\n \t\tstrbuf_release(&norm);\n \t\treturn NULL;\n \t}\n-\tspanned = strspn(url, URL_HOST_CHARS);\n+\n+\tif (allow_globs)\n+\t\tspanned = strspn(url, URL_HOST_CHARS \"*\");\n+\telse\n+\t\tspanned = strspn(url, URL_HOST_CHARS);\n+\n \tif (spanned < colon_ptr - url) {\n \t\t/* Host name has invalid characters */\n \t\tif (out_info) {\n@@ -380,6 +385,11 @@ char *url_normalize(const char *url, struct url_info *out_info)\n \treturn result;\n }\n \n+char *url_normalize(const char *url, struct url_info *out_info)\n+{\n+\treturn url_normalize_1(url, out_info, 0);\n+}\n+\n static size_t url_match_prefix(const char *url,\n \t\t\t const char *url_prefix,\n \t\t\t size_t url_prefix_len)\n-- \n2.11.0\n\n"},{"id":"310396","messageId":"cover.1485512626.git.patrick.steinhardt@elego.de","threadId":"44986","inReplyTo":"20170123130635.29577-1-patrick.steinhardt@elego.de","subject":"[PATCH v4 0/5] urlmatch: allow wildcard-based matches","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-27T10:32:13Z","receivedAt":"2017-01-27T10:35:38Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nso this is part four of my patch series. The previous version can\nbe found at [1]. The use case is to be able to configure an HTTP\nproxy for all subdomains of a domain where there are hundreds of\nsubdomains.\n\nChanges to the previous version:\n\n - applied Junio's proposed patch to replace `strtok_r` with a\n `memchr`-based loop\n - applied Junio's proposed rewrite of the commit message of\n patch 5\n - I realized that with my patches, \"ranking\" of URLs was broken.\n Previously, we've always taken the longest matching URL. As\n previously, only the user and path could actually differ, only\n these two components were used for the comparison. I've\n changed this now to also include the host part so that URLs\n with a longer host will take precedence. This resulted in a\n the patch 4.\n - New tests are included which examine if the precedence-rules\n are actually honored correctly. The tests are part of patches\n 4 and 5.\n\nYou can find the interdiff below.\n\nRegards\nPatrick\n\n[1]: http://public-inbox.org/git/20170125095648.4116-1-patrick.steinhardt@elego.de/T/#t\n\nPatrick Steinhardt (5):\n mailmap: add Patrick Steinhardt's work address\n urlmatch: enable normalization of URLs with globs\n urlmatch: split host and port fields in `struct url_info`\n urlmatch: include host and port in urlmatch length\n urlmatch: allow globbing for the URL host part\n\n .mailmap | 1 +\n Documentation/config.txt | 5 +-\n t/t1300-repo-config.sh | 105 ++++++++++++++++++++++++++++++++++++\n urlmatch.c | 138 +++++++++++++++++++++++++++++++++++------------\n urlmatch.h | 12 +++--\n 5 files changed, 220 insertions(+), 41 deletions(-)\n\n-- \n2.11.0\n\ndiff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh\nindex ec545e092..052f12021 100755\n--- a/t/t1300-repo-config.sh\n+++ b/t/t1300-repo-config.sh\n@@ -1177,7 +1177,72 @@ test_expect_success 'urlmatch' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success 'glob-based urlmatch' '\n+test_expect_success 'urlmatch favors more specific URLs' '\n+\tcat >.git/config <<-\\EOF &&\n+\t[http \"https://example.com/\"]\n+\t\tcookieFile = /tmp/root.txt\n+\t[http \"https://example.com/subdirectory\"]\n+\t\tcookieFile = /tmp/subdirectory.txt\n+\t[http \"https://user@example.com/\"]\n+\t\tcookieFile = /tmp/user.txt\n+\t[http \"https://averylonguser@example.com/\"]\n+\t\tcookieFile = /tmp/averylonguser.txt\n+\t[http \"https://preceding.example.com\"]\n+\t\tcookieFile = /tmp/preceding.txt\n+\t[http \"https://*.example.com\"]\n+\t\tcookieFile = /tmp/wildcard.txt\n+\t[http \"https://*.example.com/wildcardwithsubdomain\"]\n+\t\tcookieFile = /tmp/wildcardwithsubdomain.txt\n+\t[http \"https://trailing.example.com\"]\n+\t\tcookieFile = /tmp/trailing.txt\n+\t[http \"https://user@*.example.com/\"]\n+\t\tcookieFile = /tmp/wildcardwithuser.txt\n+\t[http \"https://sub.example.com/\"]\n+\t\tcookieFile = /tmp/sub.txt\n+\tEOF\n+\n+\techo http.cookiefile /tmp/root.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/subdirectory.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://example.com/subdirectory >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/subdirectory.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://example.com/subdirectory/nested >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/user.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://user@example.com/ >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/subdirectory.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://averylonguser@example.com/subdirectory >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/preceding.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://preceding.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/wildcard.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://wildcard.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/sub.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://sub.example.com/wildcardwithsubdomain >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/trailing.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://trailing.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/sub.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://user@sub.example.com >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'urlmatch with wildcard' '\n \tcat >.git/config <<-\\EOF &&\n \t[http]\n \t\tsslVerify\n@@ -1210,6 +1275,10 @@ test_expect_success 'glob-based urlmatch' '\n \t\techo http.sslverify false\n \t} >expect &&\n \tgit config --get-urlmatch HTTP https://good.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.sslverify >expect &&\n+\tgit config --get-urlmatch HTTP https://more.example.com.au >actual &&\n \ttest_cmp expect actual\n '\n \ndiff --git a/urlmatch.c b/urlmatch.c\nindex 53ff972a6..bb5267732 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -63,36 +63,47 @@ static int append_normalized_escapes(struct strbuf *buf,\n \treturn 1;\n }\n \n+static const char *end_of_token(const char *s, int c, size_t n)\n+{\n+\tconst char *next = memchr(s, c, n);\n+\tif (!next)\n+\t\tnext = s + n;\n+\treturn next;\n+}\n+\n static int match_host(const struct url_info *url_info,\n \t\t const struct url_info *pattern_info)\n {\n-\tchar *url = xmemdupz(url_info->url + url_info->host_off, url_info->host_len);\n-\tchar *pat = xmemdupz(pattern_info->url + pattern_info->host_off, pattern_info->host_len);\n-\tchar *url_tok, *pat_tok, *url_save, *pat_save;\n-\tint matching;\n+\tconst char *url = url_info->url + url_info->host_off;\n+\tconst char *pat = pattern_info->url + pattern_info->host_off;\n+\tint url_len = url_info->host_len;\n+\tint pat_len = pattern_info->host_len;\n \n-\turl_tok = strtok_r(url, \".\", &url_save);\n-\tpat_tok = strtok_r(pat, \".\", &pat_save);\n+\twhile (url_len && pat_len) {\n+\t\tconst char *url_next = end_of_token(url, '.', url_len);\n+\t\tconst char *pat_next = end_of_token(pat, '.', pat_len);\n \n-\tfor (; url_tok && pat_tok; url_tok = strtok_r(NULL, \".\", &url_save),\n-\t\t\t\t pat_tok = strtok_r(NULL, \".\", &pat_save)) {\n-\t\tif (!strcmp(pat_tok, \"*\"))\n-\t\t\tcontinue; /* a simple glob matches everything */\n+\t\tif (pat_next == pat + 1 && pat[0] == '*')\n+\t\t\t/* wildcard matches anything */\n+\t\t\t;\n+\t\telse if ((pat_next - pat) == (url_next - url) &&\n+\t\t\t !memcmp(url, pat, url_next - url))\n+\t\t\t/* the components are the same */\n+\t\t\t;\n+\t\telse\n+\t\t\treturn 0; /* found an unmatch */\n \n-\t\tif (strcmp(url_tok, pat_tok)) {\n-\t\t\t/* subdomains do not match */\n-\t\t\tmatching = 0;\n-\t\t\tbreak;\n-\t\t}\n+\t\tif (url_next < url + url_len)\n+\t\t\turl_next++;\n+\t\turl_len -= url_next - url;\n+\t\turl = url_next;\n+\t\tif (pat_next < pat + pat_len)\n+\t\t\tpat_next++;\n+\t\tpat_len -= pat_next - pat;\n+\t\tpat = pat_next;\n \t}\n \n-\t/* matching if both URL and pattern are at their ends */\n-\tmatching = (url_tok == NULL && pat_tok == NULL);\n-\n-\tfree(url);\n-\tfree(pat);\n-\n-\treturn matching;\n+\treturn (!url_len && !pat_len);\n }\n \n static char *url_normalize_1(const char *url, struct url_info *out_info, char allow_globs)\n@@ -458,7 +469,7 @@ static size_t url_match_prefix(const char *url,\n \n static int match_urls(const struct url_info *url,\n \t\t const struct url_info *url_prefix,\n-\t\t int *exactusermatch)\n+\t\t struct urlmatch_item *match)\n {\n \t/*\n \t * url_prefix matches url if the scheme, host and port of url_prefix\n@@ -477,8 +488,8 @@ static int match_urls(const struct url_info *url,\n \t * contained a user name or false if url_prefix did not have a\n \t * user name. If there is no match *exactusermatch is left untouched.\n \t */\n-\tint usermatched = 0;\n-\tint pathmatchlen;\n+\tchar usermatched = 0;\n+\tsize_t pathmatchlen;\n \n \tif (!url || !url_prefix || !url->url || !url_prefix->url)\n \t\treturn 0;\n@@ -513,22 +524,38 @@ static int match_urls(const struct url_info *url,\n \t\turl->url + url->path_off,\n \t\turl_prefix->url + url_prefix->path_off,\n \t\turl_prefix->url_len - url_prefix->path_off);\n+\tif (!pathmatchlen)\n+\t\treturn 0; /* paths do not match */\n \n-\tif (pathmatchlen && exactusermatch)\n-\t\t*exactusermatch = usermatched;\n-\treturn pathmatchlen;\n+\tif (match) {\n+\t\tmatch->hostmatch_len = url_prefix->host_len;\n+\t\tmatch->pathmatch_len = pathmatchlen;\n+\t\tmatch->user_matched = usermatched;\n+\t}\n+\n+\treturn 1;\n+}\n+\n+static int cmp_matches(const struct urlmatch_item *a,\n+\t\t const struct urlmatch_item *b)\n+{\n+\tif (a->hostmatch_len != b->hostmatch_len)\n+\t\treturn a->hostmatch_len < b->hostmatch_len ? -1 : 1;\n+\tif (a->pathmatch_len != b->pathmatch_len)\n+\t\treturn a->pathmatch_len < b->pathmatch_len ? -1 : 1;\n+\tif (a->user_matched != b->user_matched)\n+\t\treturn b->user_matched ? -1 : 1;\n+\treturn 0;\n }\n \n int urlmatch_config_entry(const char *var, const char *value, void *cb)\n {\n \tstruct string_list_item *item;\n \tstruct urlmatch_config *collect = cb;\n-\tstruct urlmatch_item *matched;\n+\tstruct urlmatch_item matched;\n \tstruct url_info *url = &collect->url;\n \tconst char *key, *dot;\n \tstruct strbuf synthkey = STRBUF_INIT;\n-\tsize_t matched_len = 0;\n-\tint user_matched = 0;\n \tint retval;\n \n \tif (!skip_prefix(var, collect->section, &key) || *(key++) != '.') {\n@@ -546,9 +573,9 @@ int urlmatch_config_entry(const char *var, const char *value, void *cb)\n \t\tfree(config_url);\n \t\tif (!norm_url)\n \t\t\treturn 0;\n-\t\tmatched_len = match_urls(url, &norm_info, &user_matched);\n+\t\tretval = match_urls(url, &norm_info, &matched);\n \t\tfree(norm_url);\n-\t\tif (!matched_len)\n+\t\tif (!retval)\n \t\t\treturn 0;\n \t\tkey = dot + 1;\n \t}\n@@ -558,24 +585,17 @@ int urlmatch_config_entry(const char *var, const char *value, void *cb)\n \n \titem = string_list_insert(&collect->vars, key);\n \tif (!item->util) {\n-\t\tmatched = xcalloc(1, sizeof(*matched));\n-\t\titem->util = matched;\n+\t\titem->util = xcalloc(1, sizeof(matched));\n \t} else {\n-\t\tmatched = item->util;\n-\t\t/*\n-\t\t * Is our match shorter? Is our match the same\n-\t\t * length, and without user while the current\n-\t\t * candidate is with user? Then we cannot use it.\n-\t\t */\n-\t\tif (matched_len < matched->matched_len ||\n-\t\t ((matched_len == matched->matched_len) &&\n-\t\t (!user_matched && matched->user_matched)))\n+\t\tif (cmp_matches(&matched, item->util) <= 0)\n+\t\t\t /*\n+\t\t\t * Our match is worse than the old one,\n+\t\t\t * we cannot use it.\n+\t\t\t */\n \t\t\treturn 0;\n-\t\t/* Otherwise, replace it with this one. */\n \t}\n \n-\tmatched->matched_len = matched_len;\n-\tmatched->user_matched = user_matched;\n+\tmemcpy(item->util, &matched, sizeof(matched));\n \tstrbuf_addstr(&synthkey, collect->section);\n \tstrbuf_addch(&synthkey, '.');\n \tstrbuf_addstr(&synthkey, key);\ndiff --git a/urlmatch.h b/urlmatch.h\nindex 0ea812b03..37ee5da85 100644\n--- a/urlmatch.h\n+++ b/urlmatch.h\n@@ -34,7 +34,8 @@ struct url_info {\n extern char *url_normalize(const char *, struct url_info *);\n \n struct urlmatch_item {\n-\tsize_t matched_len;\n+\tsize_t hostmatch_len;\n+\tsize_t pathmatch_len;\n \tchar user_matched;\n };\n \n"},{"id":"310397","messageId":"27c4d2844114c6bc8dc3b02d1a426411b7ab1d56.1485512626.git.patrick.steinhardt@elego.de","threadId":"44986","inReplyTo":"cover.1485512626.git.patrick.steinhardt@elego.de","subject":"[PATCH v4 4/5] urlmatch: include host and port in urlmatch length","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-27T10:32:17Z","receivedAt":"2017-01-27T10:35:40Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In order to be able to rank positive matches by `urlmatch`, we inspect\nthe path length and user part to decide whether a match is better than\nanother match. As all other parts are matched exactly between both URLs,\nthis is the right thing to do right now.\n\nIn the future, though, we want to introduce wild cards for the domain\npart. When doing this, though, it does not make sense anymore to only\ncompare the path lengths. Instead, we also want to compare the domain\nlengths to determine which of both URLs matches the host part more\nclosely.\n\nSigned-off-by: Patrick Steinhardt \n---\n t/t1300-repo-config.sh | 33 ++++++++++++++++++++++++++++\n urlmatch.c | 59 +++++++++++++++++++++++++++++---------------------\n urlmatch.h | 3 ++-\n 3 files changed, 69 insertions(+), 26 deletions(-)\n\ndiff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh\nindex 923bfc5a2..6c844d519 100755\n--- a/t/t1300-repo-config.sh\n+++ b/t/t1300-repo-config.sh\n@@ -1177,6 +1177,39 @@ test_expect_success 'urlmatch' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'urlmatch favors more specific URLs' '\n+\tcat >.git/config <<-\\EOF &&\n+\t[http \"https://example.com/\"]\n+\t\tcookieFile = /tmp/root.txt\n+\t[http \"https://example.com/subdirectory\"]\n+\t\tcookieFile = /tmp/subdirectory.txt\n+\t[http \"https://user@example.com/\"]\n+\t\tcookieFile = /tmp/user.txt\n+\t[http \"https://averylonguser@example.com/\"]\n+\t\tcookieFile = /tmp/averylonguser.txt\n+\tEOF\n+\n+\techo http.cookiefile /tmp/root.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/subdirectory.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://example.com/subdirectory >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/subdirectory.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://example.com/subdirectory/nested >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/user.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://user@example.com/ >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/subdirectory.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://averylonguser@example.com/subdirectory >actual &&\n+\ttest_cmp expect actual\n+'\n+\n # good section hygiene\n test_expect_failure 'unsetting the last key in a section removes header' '\n \tcat >.git/config <<-\\EOF &&\ndiff --git a/urlmatch.c b/urlmatch.c\nindex e328905eb..f35d00a6e 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -426,7 +426,7 @@ static size_t url_match_prefix(const char *url,\n \n static int match_urls(const struct url_info *url,\n \t\t const struct url_info *url_prefix,\n-\t\t int *exactusermatch)\n+\t\t struct urlmatch_item *match)\n {\n \t/*\n \t * url_prefix matches url if the scheme, host and port of url_prefix\n@@ -445,8 +445,8 @@ static int match_urls(const struct url_info *url,\n \t * contained a user name or false if url_prefix did not have a\n \t * user name. If there is no match *exactusermatch is left untouched.\n \t */\n-\tint usermatched = 0;\n-\tint pathmatchlen;\n+\tchar usermatched = 0;\n+\tsize_t pathmatchlen;\n \n \tif (!url || !url_prefix || !url->url || !url_prefix->url)\n \t\treturn 0;\n@@ -483,22 +483,38 @@ static int match_urls(const struct url_info *url,\n \t\turl->url + url->path_off,\n \t\turl_prefix->url + url_prefix->path_off,\n \t\turl_prefix->url_len - url_prefix->path_off);\n+\tif (!pathmatchlen)\n+\t\treturn 0; /* paths do not match */\n \n-\tif (pathmatchlen && exactusermatch)\n-\t\t*exactusermatch = usermatched;\n-\treturn pathmatchlen;\n+\tif (match) {\n+\t\tmatch->hostmatch_len = url_prefix->host_len;\n+\t\tmatch->pathmatch_len = pathmatchlen;\n+\t\tmatch->user_matched = usermatched;\n+\t}\n+\n+\treturn 1;\n+}\n+\n+static int cmp_matches(const struct urlmatch_item *a,\n+\t\t const struct urlmatch_item *b)\n+{\n+\tif (a->hostmatch_len != b->hostmatch_len)\n+\t\treturn a->hostmatch_len < b->hostmatch_len ? -1 : 1;\n+\tif (a->pathmatch_len != b->pathmatch_len)\n+\t\treturn a->pathmatch_len < b->pathmatch_len ? -1 : 1;\n+\tif (a->user_matched != b->user_matched)\n+\t\treturn b->user_matched ? -1 : 1;\n+\treturn 0;\n }\n \n int urlmatch_config_entry(const char *var, const char *value, void *cb)\n {\n \tstruct string_list_item *item;\n \tstruct urlmatch_config *collect = cb;\n-\tstruct urlmatch_item *matched;\n+\tstruct urlmatch_item matched;\n \tstruct url_info *url = &collect->url;\n \tconst char *key, *dot;\n \tstruct strbuf synthkey = STRBUF_INIT;\n-\tsize_t matched_len = 0;\n-\tint user_matched = 0;\n \tint retval;\n \n \tif (!skip_prefix(var, collect->section, &key) || *(key++) != '.') {\n@@ -516,9 +532,9 @@ int urlmatch_config_entry(const char *var, const char *value, void *cb)\n \t\tfree(config_url);\n \t\tif (!norm_url)\n \t\t\treturn 0;\n-\t\tmatched_len = match_urls(url, &norm_info, &user_matched);\n+\t\tretval = match_urls(url, &norm_info, &matched);\n \t\tfree(norm_url);\n-\t\tif (!matched_len)\n+\t\tif (!retval)\n \t\t\treturn 0;\n \t\tkey = dot + 1;\n \t}\n@@ -528,24 +544,17 @@ int urlmatch_config_entry(const char *var, const char *value, void *cb)\n \n \titem = string_list_insert(&collect->vars, key);\n \tif (!item->util) {\n-\t\tmatched = xcalloc(1, sizeof(*matched));\n-\t\titem->util = matched;\n+\t\titem->util = xcalloc(1, sizeof(matched));\n \t} else {\n-\t\tmatched = item->util;\n-\t\t/*\n-\t\t * Is our match shorter? Is our match the same\n-\t\t * length, and without user while the current\n-\t\t * candidate is with user? Then we cannot use it.\n-\t\t */\n-\t\tif (matched_len < matched->matched_len ||\n-\t\t ((matched_len == matched->matched_len) &&\n-\t\t (!user_matched && matched->user_matched)))\n+\t\tif (cmp_matches(&matched, item->util) <= 0)\n+\t\t\t /*\n+\t\t\t * Our match is worse than the old one,\n+\t\t\t * we cannot use it.\n+\t\t\t */\n \t\t\treturn 0;\n-\t\t/* Otherwise, replace it with this one. */\n \t}\n \n-\tmatched->matched_len = matched_len;\n-\tmatched->user_matched = user_matched;\n+\tmemcpy(item->util, &matched, sizeof(matched));\n \tstrbuf_addstr(&synthkey, collect->section);\n \tstrbuf_addch(&synthkey, '.');\n \tstrbuf_addstr(&synthkey, key);\ndiff --git a/urlmatch.h b/urlmatch.h\nindex 0ea812b03..37ee5da85 100644\n--- a/urlmatch.h\n+++ b/urlmatch.h\n@@ -34,7 +34,8 @@ struct url_info {\n extern char *url_normalize(const char *, struct url_info *);\n \n struct urlmatch_item {\n-\tsize_t matched_len;\n+\tsize_t hostmatch_len;\n+\tsize_t pathmatch_len;\n \tchar user_matched;\n };\n \n-- \n2.11.0\n\n"},{"id":"310398","messageId":"0f617eb0d00358d58476e413d58ee031e7475645.1485512626.git.patrick.steinhardt@elego.de","threadId":"44986","inReplyTo":"cover.1485512626.git.patrick.steinhardt@elego.de","subject":"[PATCH v4 3/5] urlmatch: split host and port fields in `struct url_info`","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-27T10:32:16Z","receivedAt":"2017-01-27T10:35:43Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `url_info` structure contains information about a normalized URL\nwith the URL's components being represented by different fields. The\nhost and port part though are to be accessed by the same `host` field,\nso that getting the host and/or port separately becomes more involved\nthan really necessary.\n\nTo make the port more readily accessible, split up the host and port\nfields. Namely, the `host_len` will not include the port length anymore\nand a new `port_off` field has been added which includes the offset to\nthe port, if available.\n\nThe only user of these fields is `url_normalize_1`. This change makes it\neasier later on to treat host and port differently when introducing\nglobs for domains.\n\nSigned-off-by: Patrick Steinhardt \n---\n urlmatch.c | 16 ++++++++++++----\n urlmatch.h | 9 +++++----\n 2 files changed, 17 insertions(+), 8 deletions(-)\n\ndiff --git a/urlmatch.c b/urlmatch.c\nindex d350478c0..e328905eb 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -104,7 +104,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al\n \tstruct strbuf norm;\n \tsize_t spanned;\n \tsize_t scheme_len, user_off=0, user_len=0, passwd_off=0, passwd_len=0;\n-\tsize_t host_off=0, host_len=0, port_len=0, path_off, path_len, result_len;\n+\tsize_t host_off=0, host_len=0, port_off=0, port_len=0, path_off, path_len, result_len;\n \tconst char *slash_ptr, *at_ptr, *colon_ptr, *path_start;\n \tchar *result;\n \n@@ -263,6 +263,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al\n \t\t\t\treturn NULL;\n \t\t\t}\n \t\t\tstrbuf_addch(&norm, ':');\n+\t\t\tport_off = norm.len;\n \t\t\tstrbuf_add(&norm, url, slash_ptr - url);\n \t\t\tport_len = slash_ptr - url;\n \t\t}\n@@ -270,7 +271,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al\n \t\turl = slash_ptr;\n \t}\n \tif (host_off)\n-\t\thost_len = norm.len - host_off;\n+\t\thost_len = norm.len - host_off - (port_len ? port_len + 1 : 0);\n \n \n \t/*\n@@ -378,6 +379,7 @@ static char *url_normalize_1(const char *url, struct url_info *out_info, char al\n \t\tout_info->passwd_len = passwd_len;\n \t\tout_info->host_off = host_off;\n \t\tout_info->host_len = host_len;\n+\t\tout_info->port_off = port_off;\n \t\tout_info->port_len = port_len;\n \t\tout_info->path_off = path_off;\n \t\tout_info->path_len = path_len;\n@@ -464,11 +466,17 @@ static int match_urls(const struct url_info *url,\n \t\tusermatched = 1;\n \t}\n \n-\t/* check the host and port */\n+\t/* check the host */\n \tif (url_prefix->host_len != url->host_len ||\n \t strncmp(url->url + url->host_off,\n \t\t url_prefix->url + url_prefix->host_off, url->host_len))\n-\t\treturn 0; /* host names and/or ports do not match */\n+\t\treturn 0; /* host names do not match */\n+\n+\t/* check the port */\n+\tif (url_prefix->port_len != url->port_len ||\n+\t strncmp(url->url + url->port_off,\n+\t\t url_prefix->url + url_prefix->port_off, url->port_len))\n+\t\treturn 0; /* ports do not match */\n \n \t/* check the path */\n \tpathmatchlen = url_match_prefix(\ndiff --git a/urlmatch.h b/urlmatch.h\nindex 528862adc..0ea812b03 100644\n--- a/urlmatch.h\n+++ b/urlmatch.h\n@@ -18,11 +18,12 @@ struct url_info {\n \tsize_t passwd_len;\t/* length of passwd; if passwd_off != 0 but\n \t\t\t\t passwd_len == 0, an empty passwd was given */\n \tsize_t host_off;\t/* offset into url to start of host name (0 => none) */\n-\tsize_t host_len;\t/* length of host name; this INCLUDES any ':portnum';\n+\tsize_t host_len;\t/* length of host name;\n \t\t\t\t * file urls may have host_len == 0 */\n-\tsize_t port_len;\t/* if a portnum is present (port_len != 0), it has\n-\t\t\t\t * this length (excluding the leading ':') at the\n-\t\t\t\t * end of the host name (always 0 for file urls) */\n+\tsize_t port_off;\t/* offset into url to start of port number (0 => none) */\n+\tsize_t port_len;\t/* if a portnum is present (port_off != 0), it has\n+\t\t\t\t * this length (excluding the leading ':') starting\n+\t\t\t\t * from port_off (always 0 for file urls) */\n \tsize_t path_off;\t/* offset into url to the start of the url path;\n \t\t\t\t * this will always point to a '/' character\n \t\t\t\t * after the url has been normalized */\n-- \n2.11.0\n\n"},{"id":"310399","messageId":"ac84183cc77f4d91e580ceb47c0c8bda5d6c3531.1485512626.git.patrick.steinhardt@elego.de","threadId":"44986","inReplyTo":"cover.1485512626.git.patrick.steinhardt@elego.de","subject":"[PATCH v4 5/5] urlmatch: allow globbing for the URL host part","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-27T10:32:18Z","receivedAt":"2017-01-27T10:35:45Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The URL matching function computes for two URLs whether they match not.\nThe match is performed by splitting up the URL into different parts and\nthen doing an exact comparison with the to-be-matched URL.\n\nThe main user of `urlmatch` is the configuration subsystem. It allows to\nset certain configurations based on the URL which is being connected to\nvia keys like `http..*`. A common use case for this is to set\nproxies for only some remotes which match the given URL. Unfortunately,\nhaving exact matches for all parts of the URL can become quite tedious\nin some setups. Imagine for example a corporate network where there are\ndozens or even hundreds of subdomains, which would have to be configured\nindividually.\n\nAllow users to write an asterisk '*' in place of any 'host' or\n'subdomain' label as part of the host name. For example,\n\"http.https://*.example.com.proxy\" sets \"http.proxy\" for all direct\nsubdomains of \"https://example.com\", e.g. \"https://foo.example.com\", but\nnot \"https://foo.bar.example.com\".\n\nSigned-off-by: Patrick Steinhardt \nHelped-by: Junio C Hamano \n---\n Documentation/config.txt | 5 +++-\n t/t1300-repo-config.sh | 72 ++++++++++++++++++++++++++++++++++++++++++++++++\n urlmatch.c | 49 +++++++++++++++++++++++++++++---\n 3 files changed, 121 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 506431267..078e9b490 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1914,7 +1914,10 @@ http..*::\n must match exactly between the config key and the URL.\n \n . Host/domain name (e.g., `example.com` in `https://example.com/`).\n- This field must match exactly between the config key and the URL.\n+ This field must match between the config key and the URL. It is\n+ possible to specify a `*` as part of the host name to match all subdomains\n+ at this level. `https://*.example.com/` for example would match\n+ `https://foo.example.com/`, but not `https://foo.bar.example.com/`.\n \n . Port number (e.g., `8080` in `http://example.com:8080/`).\n This field must match exactly between the config key and the URL.\ndiff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh\nindex 6c844d519..052f12021 100755\n--- a/t/t1300-repo-config.sh\n+++ b/t/t1300-repo-config.sh\n@@ -1187,6 +1187,18 @@ test_expect_success 'urlmatch favors more specific URLs' '\n \t\tcookieFile = /tmp/user.txt\n \t[http \"https://averylonguser@example.com/\"]\n \t\tcookieFile = /tmp/averylonguser.txt\n+\t[http \"https://preceding.example.com\"]\n+\t\tcookieFile = /tmp/preceding.txt\n+\t[http \"https://*.example.com\"]\n+\t\tcookieFile = /tmp/wildcard.txt\n+\t[http \"https://*.example.com/wildcardwithsubdomain\"]\n+\t\tcookieFile = /tmp/wildcardwithsubdomain.txt\n+\t[http \"https://trailing.example.com\"]\n+\t\tcookieFile = /tmp/trailing.txt\n+\t[http \"https://user@*.example.com/\"]\n+\t\tcookieFile = /tmp/wildcardwithuser.txt\n+\t[http \"https://sub.example.com/\"]\n+\t\tcookieFile = /tmp/sub.txt\n \tEOF\n \n \techo http.cookiefile /tmp/root.txt >expect &&\n@@ -1207,6 +1219,66 @@ test_expect_success 'urlmatch favors more specific URLs' '\n \n \techo http.cookiefile /tmp/subdirectory.txt >expect &&\n \tgit config --get-urlmatch HTTP https://averylonguser@example.com/subdirectory >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/preceding.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://preceding.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/wildcard.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://wildcard.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/sub.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://sub.example.com/wildcardwithsubdomain >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/trailing.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://trailing.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.cookiefile /tmp/sub.txt >expect &&\n+\tgit config --get-urlmatch HTTP https://user@sub.example.com >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'urlmatch with wildcard' '\n+\tcat >.git/config <<-\\EOF &&\n+\t[http]\n+\t\tsslVerify\n+\t[http \"https://*.example.com\"]\n+\t\tsslVerify = false\n+\t\tcookieFile = /tmp/cookie.txt\n+\tEOF\n+\n+\ttest_expect_code 1 git config --bool --get-urlmatch doesnt.exist https://good.example.com >actual &&\n+\ttest_must_be_empty actual &&\n+\n+\techo true >expect &&\n+\tgit config --bool --get-urlmatch http.SSLverify https://example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo true >expect &&\n+\tgit config --bool --get-urlmatch http.SSLverify https://good-example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo true >expect &&\n+\tgit config --bool --get-urlmatch http.sslverify https://deep.nested.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo false >expect &&\n+\tgit config --bool --get-urlmatch http.sslverify https://good.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\t{\n+\t\techo http.cookiefile /tmp/cookie.txt &&\n+\t\techo http.sslverify false\n+\t} >expect &&\n+\tgit config --get-urlmatch HTTP https://good.example.com >actual &&\n+\ttest_cmp expect actual &&\n+\n+\techo http.sslverify >expect &&\n+\tgit config --get-urlmatch HTTP https://more.example.com.au >actual &&\n \ttest_cmp expect actual\n '\n \ndiff --git a/urlmatch.c b/urlmatch.c\nindex f35d00a6e..bb5267732 100644\n--- a/urlmatch.c\n+++ b/urlmatch.c\n@@ -63,6 +63,49 @@ static int append_normalized_escapes(struct strbuf *buf,\n \treturn 1;\n }\n \n+static const char *end_of_token(const char *s, int c, size_t n)\n+{\n+\tconst char *next = memchr(s, c, n);\n+\tif (!next)\n+\t\tnext = s + n;\n+\treturn next;\n+}\n+\n+static int match_host(const struct url_info *url_info,\n+\t\t const struct url_info *pattern_info)\n+{\n+\tconst char *url = url_info->url + url_info->host_off;\n+\tconst char *pat = pattern_info->url + pattern_info->host_off;\n+\tint url_len = url_info->host_len;\n+\tint pat_len = pattern_info->host_len;\n+\n+\twhile (url_len && pat_len) {\n+\t\tconst char *url_next = end_of_token(url, '.', url_len);\n+\t\tconst char *pat_next = end_of_token(pat, '.', pat_len);\n+\n+\t\tif (pat_next == pat + 1 && pat[0] == '*')\n+\t\t\t/* wildcard matches anything */\n+\t\t\t;\n+\t\telse if ((pat_next - pat) == (url_next - url) &&\n+\t\t\t !memcmp(url, pat, url_next - url))\n+\t\t\t/* the components are the same */\n+\t\t\t;\n+\t\telse\n+\t\t\treturn 0; /* found an unmatch */\n+\n+\t\tif (url_next < url + url_len)\n+\t\t\turl_next++;\n+\t\turl_len -= url_next - url;\n+\t\turl = url_next;\n+\t\tif (pat_next < pat + pat_len)\n+\t\t\tpat_next++;\n+\t\tpat_len -= pat_next - pat;\n+\t\tpat = pat_next;\n+\t}\n+\n+\treturn (!url_len && !pat_len);\n+}\n+\n static char *url_normalize_1(const char *url, struct url_info *out_info, char allow_globs)\n {\n \t/*\n@@ -467,9 +510,7 @@ static int match_urls(const struct url_info *url,\n \t}\n \n \t/* check the host */\n-\tif (url_prefix->host_len != url->host_len ||\n-\t strncmp(url->url + url->host_off,\n-\t\t url_prefix->url + url_prefix->host_off, url->host_len))\n+\tif (!match_host(url, url_prefix))\n \t\treturn 0; /* host names do not match */\n \n \t/* check the port */\n@@ -528,7 +569,7 @@ int urlmatch_config_entry(const char *var, const char *value, void *cb)\n \t\tstruct url_info norm_info;\n \n \t\tconfig_url = xmemdupz(key, dot - key);\n-\t\tnorm_url = url_normalize(config_url, &norm_info);\n+\t\tnorm_url = url_normalize_1(config_url, &norm_info, 1);\n \t\tfree(config_url);\n \t\tif (!norm_url)\n \t\t\treturn 0;\n-- \n2.11.0\n\n"},{"id":"310565","messageId":"xmqqy3xstdh9.fsf@gitster.mtv.corp.google.com","threadId":"44986","inReplyTo":"cover.1485512626.git.patrick.steinhardt@elego.de","subject":"Re: [PATCH v4 0/5] urlmatch: allow wildcard-based matches","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-01-30T22:00:34Z","receivedAt":"2017-01-30T22:00:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> - I realized that with my patches, \"ranking\" of URLs was broken.\n> Previously, we've always taken the longest matching URL. As\n> previously, only the user and path could actually differ, only\n> these two components were used for the comparison. I've\n> changed this now to also include the host part so that URLs\n> with a longer host will take precedence. This resulted in a\n> the patch 4.\n\nGood thinking. I was wondering about this, too.\n\nThanks. Will read it through and replace.\n"},{"id":"310571","messageId":"xmqqfuk0tb3j.fsf@gitster.mtv.corp.google.com","threadId":"44986","inReplyTo":"xmqqy3xstdh9.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v4 0/5] urlmatch: allow wildcard-based matches","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-01-30T22:52:00Z","receivedAt":"2017-01-30T22:52:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Patrick Steinhardt writes:\n>\n>> - I realized that with my patches, \"ranking\" of URLs was broken.\n>> Previously, we've always taken the longest matching URL. As\n>> previously, only the user and path could actually differ, only\n>> these two components were used for the comparison. I've\n>> changed this now to also include the host part so that URLs\n>> with a longer host will take precedence. This resulted in a\n>> the patch 4.\n>\n> Good thinking. I was wondering about this, too.\n>\n> Thanks. Will read it through and replace.\n\nUgh. When applied on top of 4e59582ff7 (\"Seventh batch for 2.12\",\n2017-01-23), Git fails its self-test in t5551 #31 (I do not run with\nEXPENSIVE so some tests liks #30 are skipped, if it matters).\n\n"},{"id":"310591","messageId":"20170131082644.GA560@pks-xps","threadId":"44986","inReplyTo":"xmqqfuk0tb3j.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v4 0/5] urlmatch: allow wildcard-based matches","fromName":"Patrick Steinhardt","fromEmail":"patrick.steinhardt@elego.de","sentAt":"2017-01-31T08:26:44Z","receivedAt":"2017-01-31T08:34:41Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Jan 30, 2017 at 02:52:00PM -0800, Junio C Hamano wrote:\n> Junio C Hamano writes:\n> \n> > Patrick Steinhardt writes:\n> >\n> >> - I realized that with my patches, \"ranking\" of URLs was broken.\n> >> Previously, we've always taken the longest matching URL. As\n> >> previously, only the user and path could actually differ, only\n> >> these two components were used for the comparison. I've\n> >> changed this now to also include the host part so that URLs\n> >> with a longer host will take precedence. This resulted in a\n> >> the patch 4.\n> >\n> > Good thinking. I was wondering about this, too.\n> >\n> > Thanks. Will read it through and replace.\n> \n> Ugh. When applied on top of 4e59582ff7 (\"Seventh batch for 2.12\",\n> 2017-01-23), Git fails its self-test in t5551 #31 (I do not run with\n> EXPENSIVE so some tests liks #30 are skipped, if it matters).\n> \n\nThanks for letting me know. I didn't have any errors on my other\nmachine, but was actually able to reproduce test failures on this\none.\n\nEmbarassingly, I forgot to zero-initialize the `struct\nurlmatch_item`, leading to undefined behavior when searching for\nconfiguration keys without a dot inside. This is the case for\nnearly all keys, except for example the ones with a URL inside.\n\nWill re-send a fixed version.\n"}]}