{"thread":{"id":"61693","subject":"[GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","startedAt":"2024-06-28T12:56:59Z","lastAt":"2024-08-21T16:08:35Z","messageCount":23,"participants":["Ghanshyam Thakkar","Karthik Nayak","Patrick Steinhardt","Junio C Hamano","Kaartic Sivaraam","Christian Couder"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"497811","messageId":"20240628125632.45603-1-shyamthakkar001@gmail.com","threadId":"61693","inReplyTo":null,"subject":"[GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","fromName":"Ghanshyam Thakkar","fromEmail":"shyamthakkar001@gmail.com","sentAt":"2024-06-28T12:56:24Z","receivedAt":"2024-06-28T12:56:59Z","isPatch":true,"sender":{"key":"shyamthakkar001@gmail.com","avatar":"https://avatars.githubusercontent.com/u/72698233?v=4"},"body":"helper/test-urlmatch-normalization along with\nt0110-urlmatch-normalization test the `url_normalize()` function from\n'urlmatch.h'. Migrate them to the unit testing framework for better\nperformance. And also add different test_msg()s for better debugging.\n\nIn the migration, last two of the checks from `t_url_general_escape()`\nwere slightly changed compared to the shellscript. This involves changing\n\n'\\'' -> '\n'\\!' -> !\n\nin the urls of those checks. This is because in C strings, we don't\nneed to escape \"'\" and \"!\". Other than these two, all the urls were\npasted verbatim from the shellscript.\n\nAnother change is the removal of MINGW prerequisite from one of the\ntest. It was there because[1] on Windows, the command line is a Unicode\nstring, it is not possible to pass arbitrary bytes to a program. But\nin unit tests we don't have this limitation.\n\n[1]: https://lore.kernel.org/git/53CAC8EF.6020707@gmail.com/\n\nMentored-by: Christian Couder \nMentored-by: Kaartic Sivaraam \nSigned-off-by: Ghanshyam Thakkar \n---\n Makefile | 2 +-\n t/helper/test-tool.c | 1 -\n t/helper/test-tool.h | 1 -\n t/helper/test-urlmatch-normalization.c | 56 ----\n t/t0110-urlmatch-normalization.sh | 182 -----------\n t/unit-tests/t-urlmatch-normalization.c | 284 ++++++++++++++++++\n .../t-urlmatch-normalization}/README | 0\n .../t-urlmatch-normalization}/url-1 | Bin\n .../t-urlmatch-normalization}/url-10 | Bin\n .../t-urlmatch-normalization}/url-11 | Bin\n .../t-urlmatch-normalization}/url-2 | Bin\n .../t-urlmatch-normalization}/url-3 | Bin\n .../t-urlmatch-normalization}/url-4 | Bin\n .../t-urlmatch-normalization}/url-5 | Bin\n .../t-urlmatch-normalization}/url-6 | Bin\n .../t-urlmatch-normalization}/url-7 | Bin\n .../t-urlmatch-normalization}/url-8 | Bin\n .../t-urlmatch-normalization}/url-9 | Bin\n 18 files changed, 285 insertions(+), 241 deletions(-)\n delete mode 100644 t/helper/test-urlmatch-normalization.c\n delete mode 100755 t/t0110-urlmatch-normalization.sh\n create mode 100644 t/unit-tests/t-urlmatch-normalization.c\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/README (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-1 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-10 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-11 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-2 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-3 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-4 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-5 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-6 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-7 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-8 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-9 (100%)\n\ndiff --git a/Makefile b/Makefile\nindex 83bd9d13af..0fc0ee8c3e 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -844,7 +844,6 @@ TEST_BUILTINS_OBJS += test-submodule.o\n TEST_BUILTINS_OBJS += test-subprocess.o\n TEST_BUILTINS_OBJS += test-trace2.o\n TEST_BUILTINS_OBJS += test-truncate.o\n-TEST_BUILTINS_OBJS += test-urlmatch-normalization.o\n TEST_BUILTINS_OBJS += test-userdiff.o\n TEST_BUILTINS_OBJS += test-wildmatch.o\n TEST_BUILTINS_OBJS += test-windows-named-pipe.o\n@@ -1343,6 +1342,7 @@ UNIT_TEST_PROGRAMS += t-strbuf\n UNIT_TEST_PROGRAMS += t-strcmp-offset\n UNIT_TEST_PROGRAMS += t-strvec\n UNIT_TEST_PROGRAMS += t-trailer\n+UNIT_TEST_PROGRAMS += t-urlmatch-normalization\n UNIT_TEST_PROGS = $(patsubst %,$(UNIT_TEST_BIN)/%$X,$(UNIT_TEST_PROGRAMS))\n UNIT_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))\n UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex 93436a82ae..feed419cdd 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -84,7 +84,6 @@ static struct test_cmd cmds[] = {\n \t{ \"trace2\", cmd__trace2 },\n \t{ \"truncate\", cmd__truncate },\n \t{ \"userdiff\", cmd__userdiff },\n-\t{ \"urlmatch-normalization\", cmd__urlmatch_normalization },\n \t{ \"xml-encode\", cmd__xml_encode },\n \t{ \"wildmatch\", cmd__wildmatch },\n #ifdef GIT_WINDOWS_NATIVE\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex d9033d14e1..0c80529604 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -77,7 +77,6 @@ int cmd__subprocess(int argc, const char **argv);\n int cmd__trace2(int argc, const char **argv);\n int cmd__truncate(int argc, const char **argv);\n int cmd__userdiff(int argc, const char **argv);\n-int cmd__urlmatch_normalization(int argc, const char **argv);\n int cmd__xml_encode(int argc, const char **argv);\n int cmd__wildmatch(int argc, const char **argv);\n #ifdef GIT_WINDOWS_NATIVE\ndiff --git a/t/helper/test-urlmatch-normalization.c b/t/helper/test-urlmatch-normalization.c\ndeleted file mode 100644\nindex 86edd454f5..0000000000\n--- a/t/helper/test-urlmatch-normalization.c\n+++ /dev/null\n@@ -1,56 +0,0 @@\n-#include \"test-tool.h\"\n-#include \"git-compat-util.h\"\n-#include \"urlmatch.h\"\n-\n-int cmd__urlmatch_normalization(int argc, const char **argv)\n-{\n-\tconst char usage[] = \"test-tool urlmatch-normalization [-p | -l] | \";\n-\tchar *url1 = NULL, *url2 = NULL;\n-\tint opt_p = 0, opt_l = 0;\n-\tint ret = 0;\n-\n-\t/*\n-\t * For one url, succeed if url_normalize succeeds on it, fail otherwise.\n-\t * For two urls, succeed only if url_normalize succeeds on both and\n-\t * the results compare equal with strcmp. If -p is given (one url only)\n-\t * and url_normalize succeeds, print the result followed by \"\\n\". If\n-\t * -l is given (one url only) and url_normalize succeeds, print the\n-\t * returned length in decimal followed by \"\\n\".\n-\t */\n-\n-\tif (argc > 1 && !strcmp(argv[1], \"-p\")) {\n-\t\topt_p = 1;\n-\t\targc--;\n-\t\targv++;\n-\t} else if (argc > 1 && !strcmp(argv[1], \"-l\")) {\n-\t\topt_l = 1;\n-\t\targc--;\n-\t\targv++;\n-\t}\n-\n-\tif (argc < 2 || argc > 3)\n-\t\tdie(\"%s\", usage);\n-\n-\tif (argc == 2) {\n-\t\tstruct url_info info;\n-\t\turl1 = url_normalize(argv[1], &info);\n-\t\tif (!url1)\n-\t\t\treturn 1;\n-\t\tif (opt_p)\n-\t\t\tprintf(\"%s\\n\", url1);\n-\t\tif (opt_l)\n-\t\t\tprintf(\"%u\\n\", (unsigned)info.url_len);\n-\t\tgoto cleanup;\n-\t}\n-\n-\tif (opt_p || opt_l)\n-\t\tdie(\"%s\", usage);\n-\n-\turl1 = url_normalize(argv[1], NULL);\n-\turl2 = url_normalize(argv[2], NULL);\n-\tret = (url1 && url2 && !strcmp(url1, url2)) ? 0 : 1;\n-cleanup:\n-\tfree(url1);\n-\tfree(url2);\n-\treturn ret;\n-}\ndiff --git a/t/t0110-urlmatch-normalization.sh b/t/t0110-urlmatch-normalization.sh\ndeleted file mode 100755\nindex 12d817fbd3..0000000000\n--- a/t/t0110-urlmatch-normalization.sh\n+++ /dev/null\n@@ -1,182 +0,0 @@\n-#!/bin/sh\n-\n-test_description='urlmatch URL normalization'\n-\n-TEST_PASSES_SANITIZE_LEAK=true\n-. ./test-lib.sh\n-\n-# The base name of the test url files\n-tu=\"$TEST_DIRECTORY/t0110/url\"\n-\n-# Note that only file: URLs should be allowed without a host\n-\n-test_expect_success 'url scheme' '\n-\t! test-tool urlmatch-normalization \"\" &&\n-\t! test-tool urlmatch-normalization \"_\" &&\n-\t! test-tool urlmatch-normalization \"scheme\" &&\n-\t! test-tool urlmatch-normalization \"scheme:\" &&\n-\t! test-tool urlmatch-normalization \"scheme:/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://\" &&\n-\t! test-tool urlmatch-normalization \"file\" &&\n-\t! test-tool urlmatch-normalization \"file:\" &&\n-\t! test-tool urlmatch-normalization \"file:/\" &&\n-\ttest-tool urlmatch-normalization \"file://\" &&\n-\t! test-tool urlmatch-normalization \"://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"x_test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"-test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"0test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"+test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \".test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"schem%6e://\" &&\n-\ttest-tool urlmatch-normalization \"x-Test+v1.0://acme.co\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"AbCdeF://x.Y\")\" = \"abcdef://x.y/\"\n-'\n-\n-test_expect_success 'url authority' '\n-\t! test-tool urlmatch-normalization \"scheme://user:pass@\" &&\n-\t! test-tool urlmatch-normalization \"scheme://?\" &&\n-\t! test-tool urlmatch-normalization \"scheme://#\" &&\n-\t! test-tool urlmatch-normalization \"scheme:///\" &&\n-\t! test-tool urlmatch-normalization \"scheme://:\" &&\n-\t! test-tool urlmatch-normalization \"scheme://:555\" &&\n-\ttest-tool urlmatch-normalization \"file://user:pass@\" &&\n-\ttest-tool urlmatch-normalization \"file://?\" &&\n-\ttest-tool urlmatch-normalization \"file://#\" &&\n-\ttest-tool urlmatch-normalization \"file:///\" &&\n-\ttest-tool urlmatch-normalization \"file://:\" &&\n-\t! test-tool urlmatch-normalization \"file://:555\" &&\n-\ttest-tool urlmatch-normalization \"scheme://user:pass@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://%00@host\" &&\n-\t! test-tool urlmatch-normalization \"scheme://%%@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host_\" &&\n-\ttest-tool urlmatch-normalization \"scheme://user:pass@host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://@host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host?x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host#x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host/@\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host?@x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host#@x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[::1]\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[::1]/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://hos%41/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[invalid....:/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://invalid....:]/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://invalid....:[/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://invalid....:[\"\n-'\n-\n-test_expect_success 'url port checks' '\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:0\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:0000000\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:0000001?\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:065535#\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:65535\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:65536\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:99999\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:100000\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:100001\" &&\n-\ttest-tool urlmatch-normalization \"http://q@some.host:80\" &&\n-\ttest-tool urlmatch-normalization \"https://q@some.host:443\" &&\n-\ttest-tool urlmatch-normalization \"http://q@some.host:80/\" &&\n-\ttest-tool urlmatch-normalization \"https://q@some.host:443?\" &&\n-\t! test-tool urlmatch-normalization \"http://q@:8008\" &&\n-\t! test-tool urlmatch-normalization \"http://:8080\" &&\n-\t! test-tool urlmatch-normalization \"http://:\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n-\ttest-tool urlmatch-normalization \"xyz://[::1]:456/\" &&\n-\ttest-tool urlmatch-normalization \"xyz://[::1]:/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:000/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:0%300/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:0x80/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:4294967297/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:030f/\"\n-'\n-\n-test_expect_success 'url port normalization' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:0800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:00000800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:065535\")\" = \"http://x:65535/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:1\")\" = \"http://x:1/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:80\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:080\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:000000080\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:443\")\" = \"https://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:0443\")\" = \"https://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:000000443\")\" = \"https://x/\"\n-'\n-\n-test_expect_success 'url general escapes' '\n-\t! test-tool urlmatch-normalization \"http://x.y?%fg\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/%7e%41^%3a\")\" = \"x://w/~A%5E%3A\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/:/?#[]@\")\" = \"x://w/:/?#[]@\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/$&()*+,;=\")\" = \"x://w/$&()*+,;=\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/'\\''\")\" = \"x://w/'\\''\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W?'\\!'\")\" = \"x://w/?'\\!'\"\n-'\n-\n-test_expect_success !MINGW 'url high-bit escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-1\")\")\" = \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-2\")\")\" = \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-3\")\")\" = \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-4\")\")\" = \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-5\")\")\" = \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-6\")\")\" = \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-7\")\")\" = \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-8\")\")\" = \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-9\")\")\" = \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-10\")\")\" = \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\"\n-'\n-\n-test_expect_success 'url utf-8 escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-11\")\")\" = \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\"\n-'\n-\n-test_expect_success 'url username/password escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://%41%62(^):%70+d@foo\")\" = \"x://Ab(%5E):p+d@foo/\"\n-'\n-\n-test_expect_success 'url normalized lengths' '\n-\ttest \"$(test-tool urlmatch-normalization -l \"Http://%4d%65:%4d^%70@The.Host\")\" = 25 &&\n-\ttest \"$(test-tool urlmatch-normalization -l \"http://%41:%42@x.y/%61/\")\" = 17 &&\n-\ttest \"$(test-tool urlmatch-normalization -l \"http://@x.y/^\")\" = 15\n-'\n-\n-test_expect_success 'url . and .. segments' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/.\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/./\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/.\")\" = \"x://y/a\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./\")\" = \"x://y/a/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/.?\")\" = \"x://y/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/./?\")\" = \"x://y/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/.?\")\" = \"x://y/a?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./?\")\" = \"x://y/a/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/.././../c\")\" = \"x://y/c\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/../.././c/\")\" = \"x://y/c/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/.././../c/././.././.\")\" = \"x://y/\" &&\n-\t! test-tool urlmatch-normalization \"x://y/a/./b/.././../c/././.././..\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./?/././..\")\" = \"x://y/a/?/././..\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/%2e/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/%2E/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/%2e./\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/b/.%2E/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/c/%2e%2E/\")\" = \"x://y/\"\n-'\n-\n-# http://@foo specifies an empty user name but does not specify a password\n-# http://foo specifies neither a user name nor a password\n-# So they should not be equivalent\n-test_expect_success 'url equivalents' '\n-\ttest-tool urlmatch-normalization \"httP://x\" \"Http://X/\" &&\n-\ttest-tool urlmatch-normalization \"Http://%4d%65:%4d^%70@The.Host\" \"hTTP://Me:%4D^p@the.HOST:80/\" &&\n-\t! test-tool urlmatch-normalization \"https://@x.y/^\" \"httpS://x.y:443/^\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^\" \"httpS://@x.y:0443/^\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^/../abc\" \"httpS://@x.y:0443/abc\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^/..\" \"httpS://@x.y:0443/\"\n-'\n-\n-test_done\ndiff --git a/t/unit-tests/t-urlmatch-normalization.c b/t/unit-tests/t-urlmatch-normalization.c\nnew file mode 100644\nindex 0000000000..4f225802b0\n--- /dev/null\n+++ b/t/unit-tests/t-urlmatch-normalization.c\n@@ -0,0 +1,284 @@\n+#include \"test-lib.h\"\n+#include \"urlmatch.h\"\n+#include \"strbuf.h\"\n+\n+static void check_url_normalizable(const char *url, int normalizable)\n+{\n+\tchar *url_norm = url_normalize(url, NULL);\n+\n+\tif (!check_int(normalizable, ==, url_norm ? 1 : 0))\n+\t\ttest_msg(\"input url: %s\", url);\n+\tfree(url_norm);\n+}\n+\n+static void check_normalized_url(const char *url, const char *expect)\n+{\n+\tchar *url_norm = url_normalize(url, NULL);\n+\n+\tif (!check_str(url_norm, expect))\n+\t\ttest_msg(\"input url: %s\", url);\n+\tfree(url_norm);\n+}\n+\n+static void compare_normalized_urls(const char *url1, const char *url2,\n+\t\t\t\t size_t equal)\n+{\n+\tchar *url1_norm = url_normalize(url1, NULL);\n+\tchar *url2_norm = url_normalize(url2, NULL);\n+\n+\tif (equal) {\n+\t\tif (!check_str(url1_norm, url2_norm))\n+\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n+\t\t\t\t url2);\n+\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0))\n+\t\ttest_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n+\t\t\t \" input url1: %s\\n input url2: %s\",\n+\t\t\t url1_norm, url2_norm, url1, url2);\n+\tfree(url1_norm);\n+\tfree(url2_norm);\n+}\n+\n+static void check_normalized_url_from_file(const char *file, const char *expect)\n+{\n+\tstruct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n+\n+\tstrbuf_getcwd(&path);\n+\tstrbuf_strip_suffix(&path, \"/unit-tests/bin\"); /* because 'unit-tests-test-tool' is run from 'bin' directory */\n+\tstrbuf_addf(&path, \"/unit-tests/t-urlmatch-normalization/%s\", file);\n+\n+\tif (!check_int(strbuf_read_file(&content, path.buf, 0), >, 0)) {\n+\t\ttest_msg(\"failed to read from file '%s': %s\", file, strerror(errno));\n+\t} else {\n+\t\tchar *url_norm;\n+\n+\t\tstrbuf_trim_trailing_newline(&content);\n+\t\turl_norm = url_normalize(content.buf, NULL);\n+\t\tif (!check_str(url_norm, expect))\n+\t\t\ttest_msg(\"input file: %s\", file);\n+\t\tfree(url_norm);\n+\t}\n+\n+\tstrbuf_release(&content);\n+\tstrbuf_release(&path);\n+}\n+\n+static void check_normalized_url_length(const char *url, size_t len)\n+{\n+\tstruct url_info info;\n+\tchar *url_norm = url_normalize(url, &info);\n+\n+\tif (!check_int(info.url_len, ==, len))\n+\t\ttest_msg(\" input url: %s\\n normalized url: %s\", url,\n+\t\t\t url_norm);\n+\tfree(url_norm);\n+}\n+\n+/* Note that only file: URLs should be allowed without a host */\n+static void t_url_scheme(void)\n+{\n+\tcheck_url_normalizable(\"\", 0);\n+\tcheck_url_normalizable(\"_\", 0);\n+\tcheck_url_normalizable(\"scheme\", 0);\n+\tcheck_url_normalizable(\"scheme:\", 0);\n+\tcheck_url_normalizable(\"scheme:/\", 0);\n+\tcheck_url_normalizable(\"scheme://\", 0);\n+\tcheck_url_normalizable(\"file\", 0);\n+\tcheck_url_normalizable(\"file:\", 0);\n+\tcheck_url_normalizable(\"file:/\", 0);\n+\tcheck_url_normalizable(\"file://\", 1);\n+\tcheck_url_normalizable(\"://acme.co\", 0);\n+\tcheck_url_normalizable(\"x_test://acme.co\", 0);\n+\tcheck_url_normalizable(\"-test://acme.co\", 0);\n+\tcheck_url_normalizable(\"0test://acme.co\", 0);\n+\tcheck_url_normalizable(\"+test://acme.co\", 0);\n+\tcheck_url_normalizable(\".test://acme.co\", 0);\n+\tcheck_url_normalizable(\"schem%6e://\", 0);\n+\tcheck_url_normalizable(\"x-Test+v1.0://acme.co\", 1);\n+\tcheck_normalized_url(\"AbCdeF://x.Y\", \"abcdef://x.y/\");\n+}\n+\n+static void t_url_authority(void)\n+{\n+\tcheck_url_normalizable(\"scheme://user:pass@\", 0);\n+\tcheck_url_normalizable(\"scheme://?\", 0);\n+\tcheck_url_normalizable(\"scheme://#\", 0);\n+\tcheck_url_normalizable(\"scheme:///\", 0);\n+\tcheck_url_normalizable(\"scheme://:\", 0);\n+\tcheck_url_normalizable(\"scheme://:555\", 0);\n+\tcheck_url_normalizable(\"file://user:pass@\", 1);\n+\tcheck_url_normalizable(\"file://?\", 1);\n+\tcheck_url_normalizable(\"file://#\", 1);\n+\tcheck_url_normalizable(\"file:///\", 1);\n+\tcheck_url_normalizable(\"file://:\", 1);\n+\tcheck_url_normalizable(\"file://:555\", 0);\n+\tcheck_url_normalizable(\"scheme://user:pass@host\", 1);\n+\tcheck_url_normalizable(\"scheme://@host\", 1);\n+\tcheck_url_normalizable(\"scheme://%00@host\", 1);\n+\tcheck_url_normalizable(\"scheme://%%@host\", 0);\n+\tcheck_url_normalizable(\"scheme://host_\", 1);\n+\tcheck_url_normalizable(\"scheme://user:pass@host/\", 1);\n+\tcheck_url_normalizable(\"scheme://@host/\", 1);\n+\tcheck_url_normalizable(\"scheme://host/\", 1);\n+\tcheck_url_normalizable(\"scheme://host?x\", 1);\n+\tcheck_url_normalizable(\"scheme://host#x\", 1);\n+\tcheck_url_normalizable(\"scheme://host/@\", 1);\n+\tcheck_url_normalizable(\"scheme://host?@x\", 1);\n+\tcheck_url_normalizable(\"scheme://host#@x\", 1);\n+\tcheck_url_normalizable(\"scheme://[::1]\", 1);\n+\tcheck_url_normalizable(\"scheme://[::1]/\", 1);\n+\tcheck_url_normalizable(\"scheme://hos%41/\", 0);\n+\tcheck_url_normalizable(\"scheme://[invalid....:/\", 1);\n+\tcheck_url_normalizable(\"scheme://invalid....:]/\", 1);\n+\tcheck_url_normalizable(\"scheme://invalid....:[/\", 0);\n+\tcheck_url_normalizable(\"scheme://invalid....:[\", 0);\n+}\n+\n+static void t_url_port(void)\n+{\n+\tcheck_url_normalizable(\"xyz://q@some.host:\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0000000\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0000001?\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:065535#\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:65535\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:65536\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:99999\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:100000\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:100001\", 0);\n+\tcheck_url_normalizable(\"http://q@some.host:80\", 1);\n+\tcheck_url_normalizable(\"https://q@some.host:443\", 1);\n+\tcheck_url_normalizable(\"http://q@some.host:80/\", 1);\n+\tcheck_url_normalizable(\"https://q@some.host:443?\", 1);\n+\tcheck_url_normalizable(\"http://q@:8008\", 0);\n+\tcheck_url_normalizable(\"http://:8080\", 0);\n+\tcheck_url_normalizable(\"http://:\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:000/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:0%300/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:0x80/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:4294967297/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:030f/\", 0);\n+}\n+\n+static void t_url_port_normalization(void)\n+{\n+\tcheck_normalized_url(\"http://x:800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:0800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:00000800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:065535\", \"http://x:65535/\");\n+\tcheck_normalized_url(\"http://x:1\", \"http://x:1/\");\n+\tcheck_normalized_url(\"http://x:80\", \"http://x/\");\n+\tcheck_normalized_url(\"http://x:080\", \"http://x/\");\n+\tcheck_normalized_url(\"http://x:000000080\", \"http://x/\");\n+\tcheck_normalized_url(\"https://x:443\", \"https://x/\");\n+\tcheck_normalized_url(\"https://x:0443\", \"https://x/\");\n+\tcheck_normalized_url(\"https://x:000000443\", \"https://x/\");\n+}\n+\n+static void t_url_general_escape(void)\n+{\n+\tcheck_url_normalizable(\"http://x.y?%fg\", 0);\n+\tcheck_normalized_url(\"X://W/%7e%41^%3a\", \"x://w/~A%5E%3A\");\n+\tcheck_normalized_url(\"X://W/:/?#[]@\", \"x://w/:/?#[]@\");\n+\tcheck_normalized_url(\"X://W/$&()*+,;=\", \"x://w/$&()*+,;=\");\n+\tcheck_normalized_url(\"X://W/'\", \"x://w/'\");\n+\tcheck_normalized_url(\"X://W?!\", \"x://w/?!\");\n+}\n+\n+static void t_url_high_bit(void)\n+{\n+\tcheck_normalized_url_from_file(\"url-1\",\n+\t\t\t \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\");\n+\tcheck_normalized_url_from_file(\"url-2\",\n+\t\t\t \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\");\n+\tcheck_normalized_url_from_file(\"url-3\",\n+\t\t\t \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\");\n+\tcheck_normalized_url_from_file(\"url-4\",\n+\t\t\t \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\");\n+\tcheck_normalized_url_from_file(\"url-5\",\n+\t\t\t \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\");\n+\tcheck_normalized_url_from_file(\"url-6\",\n+\t\t\t \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\");\n+\tcheck_normalized_url_from_file(\"url-7\",\n+\t\t\t \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\");\n+\tcheck_normalized_url_from_file(\"url-8\",\n+\t\t\t \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\");\n+\tcheck_normalized_url_from_file(\"url-9\",\n+\t\t\t \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\");\n+\tcheck_normalized_url_from_file(\"url-10\",\n+\t\t\t \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\");\n+}\n+\n+static void t_url_utf8_escape(void)\n+{\n+\tcheck_normalized_url_from_file(\"url-11\",\n+\t\t\t \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\");\n+}\n+\n+static void t_url_username_pass(void)\n+{\n+\tcheck_normalized_url(\"x://%41%62(^):%70+d@foo\", \"x://Ab(%5E):p+d@foo/\");\n+}\n+\n+static void t_url_length(void)\n+{\n+\tcheck_normalized_url_length(\"Http://%4d%65:%4d^%70@The.Host\", 25);\n+\tcheck_normalized_url_length(\"http://%41:%42@x.y/%61/\", 17);\n+\tcheck_normalized_url_length(\"http://@x.y/^\", 15);\n+}\n+\n+static void t_url_dots(void)\n+{\n+\tcheck_normalized_url(\"x://y/.\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/./\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/a/.\", \"x://y/a\");\n+\tcheck_normalized_url(\"x://y/a/./\", \"x://y/a/\");\n+\tcheck_normalized_url(\"x://y/.?\", \"x://y/?\");\n+\tcheck_normalized_url(\"x://y/./?\", \"x://y/?\");\n+\tcheck_normalized_url(\"x://y/a/.?\", \"x://y/a?\");\n+\tcheck_normalized_url(\"x://y/a/./?\", \"x://y/a/?\");\n+\tcheck_normalized_url(\"x://y/a/./b/.././../c\", \"x://y/c\");\n+\tcheck_normalized_url(\"x://y/a/./b/../.././c/\", \"x://y/c/\");\n+\tcheck_normalized_url(\"x://y/a/./b/.././../c/././.././.\", \"x://y/\");\n+\tcheck_url_normalizable(\"x://y/a/./b/.././../c/././.././..\", 0);\n+\tcheck_normalized_url(\"x://y/a/./?/././..\", \"x://y/a/?/././..\");\n+\tcheck_normalized_url(\"x://y/%2e/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/%2E/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/a/%2e./\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/b/.%2E/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/c/%2e%2E/\", \"x://y/\");\n+}\n+\n+/*\n+ * http://@foo specifies an empty user name but does not specify a password\n+ * http://foo specifies neither a user name nor a password\n+ * So they should not be equivalent\n+ */\n+static void t_url_equivalents(void)\n+{\n+\tcompare_normalized_urls(\"httP://x\", \"Http://X/\", 1);\n+\tcompare_normalized_urls(\"Http://%4d%65:%4d^%70@The.Host\", \"hTTP://Me:%4D^p@the.HOST:80/\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^\", \"httpS://x.y:443/^\", 0);\n+\tcompare_normalized_urls(\"https://@x.y/^\", \"httpS://@x.y:0443/^\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^/../abc\", \"httpS://@x.y:0443/abc\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^/..\", \"httpS://@x.y:0443/\", 1);\n+}\n+\n+int cmd_main(int argc UNUSED, const char **argv UNUSED)\n+{\n+\tTEST(t_url_scheme(), \"url scheme\");\n+\tTEST(t_url_authority(), \"url authority\");\n+\tTEST(t_url_port(), \"url port checks\");\n+\tTEST(t_url_port_normalization(), \"url port normalization\");\n+\tTEST(t_url_general_escape(), \"url general escapes\");\n+\tTEST(t_url_high_bit(), \"url high-bit escapes\");\n+\tTEST(t_url_utf8_escape(), \"url utf8 escapes\");\n+\tTEST(t_url_username_pass(), \"url username/password escapes\");\n+\tTEST(t_url_length(), \"url normalized lengths\");\n+\tTEST(t_url_dots(), \"url . and .. segments\");\n+\tTEST(t_url_equivalents(), \"url equivalents\");\n+\treturn test_done();\n+}\ndiff --git a/t/t0110/README b/t/unit-tests/t-urlmatch-normalization/README\nsimilarity index 100%\nrename from t/t0110/README\nrename to t/unit-tests/t-urlmatch-normalization/README\ndiff --git a/t/t0110/url-1 b/t/unit-tests/t-urlmatch-normalization/url-1\nsimilarity index 100%\nrename from t/t0110/url-1\nrename to t/unit-tests/t-urlmatch-normalization/url-1\ndiff --git a/t/t0110/url-10 b/t/unit-tests/t-urlmatch-normalization/url-10\nsimilarity index 100%\nrename from t/t0110/url-10\nrename to t/unit-tests/t-urlmatch-normalization/url-10\ndiff --git a/t/t0110/url-11 b/t/unit-tests/t-urlmatch-normalization/url-11\nsimilarity index 100%\nrename from t/t0110/url-11\nrename to t/unit-tests/t-urlmatch-normalization/url-11\ndiff --git a/t/t0110/url-2 b/t/unit-tests/t-urlmatch-normalization/url-2\nsimilarity index 100%\nrename from t/t0110/url-2\nrename to t/unit-tests/t-urlmatch-normalization/url-2\ndiff --git a/t/t0110/url-3 b/t/unit-tests/t-urlmatch-normalization/url-3\nsimilarity index 100%\nrename from t/t0110/url-3\nrename to t/unit-tests/t-urlmatch-normalization/url-3\ndiff --git a/t/t0110/url-4 b/t/unit-tests/t-urlmatch-normalization/url-4\nsimilarity index 100%\nrename from t/t0110/url-4\nrename to t/unit-tests/t-urlmatch-normalization/url-4\ndiff --git a/t/t0110/url-5 b/t/unit-tests/t-urlmatch-normalization/url-5\nsimilarity index 100%\nrename from t/t0110/url-5\nrename to t/unit-tests/t-urlmatch-normalization/url-5\ndiff --git a/t/t0110/url-6 b/t/unit-tests/t-urlmatch-normalization/url-6\nsimilarity index 100%\nrename from t/t0110/url-6\nrename to t/unit-tests/t-urlmatch-normalization/url-6\ndiff --git a/t/t0110/url-7 b/t/unit-tests/t-urlmatch-normalization/url-7\nsimilarity index 100%\nrename from t/t0110/url-7\nrename to t/unit-tests/t-urlmatch-normalization/url-7\ndiff --git a/t/t0110/url-8 b/t/unit-tests/t-urlmatch-normalization/url-8\nsimilarity index 100%\nrename from t/t0110/url-8\nrename to t/unit-tests/t-urlmatch-normalization/url-8\ndiff --git a/t/t0110/url-9 b/t/unit-tests/t-urlmatch-normalization/url-9\nsimilarity index 100%\nrename from t/t0110/url-9\nrename to t/unit-tests/t-urlmatch-normalization/url-9\n-- \n2.45.2\n\n"},{"id":"498334","messageId":"D2KLALHQTTYS.3GAR71UQ7QBA0@gmail.com","threadId":"61693","inReplyTo":"20240628125632.45603-1-shyamthakkar001@gmail.com","subject":"Re: [GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","fromName":"Ghanshyam Thakkar","fromEmail":"shyamthakkar001@gmail.com","sentAt":"2024-07-09T00:42:19Z","receivedAt":"2024-07-09T00:42:25Z","isPatch":true,"sender":{"key":"shyamthakkar001@gmail.com","avatar":"https://avatars.githubusercontent.com/u/72698233?v=4"},"body":"Ghanshyam Thakkar wrote:\n> helper/test-urlmatch-normalization along with\n> t0110-urlmatch-normalization test the `url_normalize()` function from\n> 'urlmatch.h'. Migrate them to the unit testing framework for better\n> performance. And also add different test_msg()s for better debugging.\n>\n> In the migration, last two of the checks from `t_url_general_escape()`\n> were slightly changed compared to the shellscript. This involves\n> changing\n>\n> '\\'' -> '\n> '\\!' -> !\n>\n> in the urls of those checks. This is because in C strings, we don't\n> need to escape \"'\" and \"!\". Other than these two, all the urls were\n> pasted verbatim from the shellscript.\n>\n> Another change is the removal of MINGW prerequisite from one of the\n> test. It was there because[1] on Windows, the command line is a Unicode\n> string, it is not possible to pass arbitrary bytes to a program. But\n> in unit tests we don't have this limitation.\n>\n> [1]: https://lore.kernel.org/git/53CAC8EF.6020707@gmail.com/\n>\n> Mentored-by: Christian Couder \n> Mentored-by: Kaartic Sivaraam \n> Signed-off-by: Ghanshyam Thakkar \n> ---\n\nFriendly reminder for reviews/acks. :)\n\nThanks.\n\n> Makefile | 2 +-\n> t/helper/test-tool.c | 1 -\n> t/helper/test-tool.h | 1 -\n> t/helper/test-urlmatch-normalization.c | 56 ----\n> t/t0110-urlmatch-normalization.sh | 182 -----------\n> t/unit-tests/t-urlmatch-normalization.c | 284 ++++++++++++++++++\n> .../t-urlmatch-normalization}/README | 0\n> .../t-urlmatch-normalization}/url-1 | Bin\n> .../t-urlmatch-normalization}/url-10 | Bin\n> .../t-urlmatch-normalization}/url-11 | Bin\n> .../t-urlmatch-normalization}/url-2 | Bin\n> .../t-urlmatch-normalization}/url-3 | Bin\n> .../t-urlmatch-normalization}/url-4 | Bin\n> .../t-urlmatch-normalization}/url-5 | Bin\n> .../t-urlmatch-normalization}/url-6 | Bin\n> .../t-urlmatch-normalization}/url-7 | Bin\n> .../t-urlmatch-normalization}/url-8 | Bin\n> .../t-urlmatch-normalization}/url-9 | Bin\n> 18 files changed, 285 insertions(+), 241 deletions(-)\n> delete mode 100644 t/helper/test-urlmatch-normalization.c\n> delete mode 100755 t/t0110-urlmatch-normalization.sh\n> create mode 100644 t/unit-tests/t-urlmatch-normalization.c\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/README (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-1 (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-10 (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-11 (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-2 (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-3 (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-4 (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-5 (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-6 (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-7 (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-8 (100%)\n> rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-9 (100%)\n>\n> diff --git a/Makefile b/Makefile\n> index 83bd9d13af..0fc0ee8c3e 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -844,7 +844,6 @@ TEST_BUILTINS_OBJS += test-submodule.o\n> TEST_BUILTINS_OBJS += test-subprocess.o\n> TEST_BUILTINS_OBJS += test-trace2.o\n> TEST_BUILTINS_OBJS += test-truncate.o\n> -TEST_BUILTINS_OBJS += test-urlmatch-normalization.o\n> TEST_BUILTINS_OBJS += test-userdiff.o\n> TEST_BUILTINS_OBJS += test-wildmatch.o\n> TEST_BUILTINS_OBJS += test-windows-named-pipe.o\n> @@ -1343,6 +1342,7 @@ UNIT_TEST_PROGRAMS += t-strbuf\n> UNIT_TEST_PROGRAMS += t-strcmp-offset\n> UNIT_TEST_PROGRAMS += t-strvec\n> UNIT_TEST_PROGRAMS += t-trailer\n> +UNIT_TEST_PROGRAMS += t-urlmatch-normalization\n> UNIT_TEST_PROGS = $(patsubst\n> %,$(UNIT_TEST_BIN)/%$X,$(UNIT_TEST_PROGRAMS))\n> UNIT_TEST_OBJS = $(patsubst\n> %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))\n> UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o\n> diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\n> index 93436a82ae..feed419cdd 100644\n> --- a/t/helper/test-tool.c\n> +++ b/t/helper/test-tool.c\n> @@ -84,7 +84,6 @@ static struct test_cmd cmds[] = {\n> { \"trace2\", cmd__trace2 },\n> { \"truncate\", cmd__truncate },\n> { \"userdiff\", cmd__userdiff },\n> - { \"urlmatch-normalization\", cmd__urlmatch_normalization },\n> { \"xml-encode\", cmd__xml_encode },\n> { \"wildmatch\", cmd__wildmatch },\n> #ifdef GIT_WINDOWS_NATIVE\n> diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\n> index d9033d14e1..0c80529604 100644\n> --- a/t/helper/test-tool.h\n> +++ b/t/helper/test-tool.h\n> @@ -77,7 +77,6 @@ int cmd__subprocess(int argc, const char **argv);\n> int cmd__trace2(int argc, const char **argv);\n> int cmd__truncate(int argc, const char **argv);\n> int cmd__userdiff(int argc, const char **argv);\n> -int cmd__urlmatch_normalization(int argc, const char **argv);\n> int cmd__xml_encode(int argc, const char **argv);\n> int cmd__wildmatch(int argc, const char **argv);\n> #ifdef GIT_WINDOWS_NATIVE\n> diff --git a/t/helper/test-urlmatch-normalization.c\n> b/t/helper/test-urlmatch-normalization.c\n> deleted file mode 100644\n> index 86edd454f5..0000000000\n> --- a/t/helper/test-urlmatch-normalization.c\n> +++ /dev/null\n> @@ -1,56 +0,0 @@\n> -#include \"test-tool.h\"\n> -#include \"git-compat-util.h\"\n> -#include \"urlmatch.h\"\n> -\n> -int cmd__urlmatch_normalization(int argc, const char **argv)\n> -{\n> - const char usage[] = \"test-tool urlmatch-normalization [-p | -l]\n> | \";\n> - char *url1 = NULL, *url2 = NULL;\n> - int opt_p = 0, opt_l = 0;\n> - int ret = 0;\n> -\n> - /*\n> - * For one url, succeed if url_normalize succeeds on it, fail\n> otherwise.\n> - * For two urls, succeed only if url_normalize succeeds on both and\n> - * the results compare equal with strcmp. If -p is given (one url only)\n> - * and url_normalize succeeds, print the result followed by \"\\n\". If\n> - * -l is given (one url only) and url_normalize succeeds, print the\n> - * returned length in decimal followed by \"\\n\".\n> - */\n> -\n> - if (argc > 1 && !strcmp(argv[1], \"-p\")) {\n> - opt_p = 1;\n> - argc--;\n> - argv++;\n> - } else if (argc > 1 && !strcmp(argv[1], \"-l\")) {\n> - opt_l = 1;\n> - argc--;\n> - argv++;\n> - }\n> -\n> - if (argc < 2 || argc > 3)\n> - die(\"%s\", usage);\n> -\n> - if (argc == 2) {\n> - struct url_info info;\n> - url1 = url_normalize(argv[1], &info);\n> - if (!url1)\n> - return 1;\n> - if (opt_p)\n> - printf(\"%s\\n\", url1);\n> - if (opt_l)\n> - printf(\"%u\\n\", (unsigned)info.url_len);\n> - goto cleanup;\n> - }\n> -\n> - if (opt_p || opt_l)\n> - die(\"%s\", usage);\n> -\n> - url1 = url_normalize(argv[1], NULL);\n> - url2 = url_normalize(argv[2], NULL);\n> - ret = (url1 && url2 && !strcmp(url1, url2)) ? 0 : 1;\n> -cleanup:\n> - free(url1);\n> - free(url2);\n> - return ret;\n> -}\n> diff --git a/t/t0110-urlmatch-normalization.sh\n> b/t/t0110-urlmatch-normalization.sh\n> deleted file mode 100755\n> index 12d817fbd3..0000000000\n> --- a/t/t0110-urlmatch-normalization.sh\n> +++ /dev/null\n> @@ -1,182 +0,0 @@\n> -#!/bin/sh\n> -\n> -test_description='urlmatch URL normalization'\n> -\n> -TEST_PASSES_SANITIZE_LEAK=true\n> -. ./test-lib.sh\n> -\n> -# The base name of the test url files\n> -tu=\"$TEST_DIRECTORY/t0110/url\"\n> -\n> -# Note that only file: URLs should be allowed without a host\n> -\n> -test_expect_success 'url scheme' '\n> - ! test-tool urlmatch-normalization \"\" &&\n> - ! test-tool urlmatch-normalization \"_\" &&\n> - ! test-tool urlmatch-normalization \"scheme\" &&\n> - ! test-tool urlmatch-normalization \"scheme:\" &&\n> - ! test-tool urlmatch-normalization \"scheme:/\" &&\n> - ! test-tool urlmatch-normalization \"scheme://\" &&\n> - ! test-tool urlmatch-normalization \"file\" &&\n> - ! test-tool urlmatch-normalization \"file:\" &&\n> - ! test-tool urlmatch-normalization \"file:/\" &&\n> - test-tool urlmatch-normalization \"file://\" &&\n> - ! test-tool urlmatch-normalization \"://acme.co\" &&\n> - ! test-tool urlmatch-normalization \"x_test://acme.co\" &&\n> - ! test-tool urlmatch-normalization \"-test://acme.co\" &&\n> - ! test-tool urlmatch-normalization \"0test://acme.co\" &&\n> - ! test-tool urlmatch-normalization \"+test://acme.co\" &&\n> - ! test-tool urlmatch-normalization \".test://acme.co\" &&\n> - ! test-tool urlmatch-normalization \"schem%6e://\" &&\n> - test-tool urlmatch-normalization \"x-Test+v1.0://acme.co\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"AbCdeF://x.Y\")\" =\n> \"abcdef://x.y/\"\n> -'\n> -\n> -test_expect_success 'url authority' '\n> - ! test-tool urlmatch-normalization \"scheme://user:pass@\" &&\n> - ! test-tool urlmatch-normalization \"scheme://?\" &&\n> - ! test-tool urlmatch-normalization \"scheme://#\" &&\n> - ! test-tool urlmatch-normalization \"scheme:///\" &&\n> - ! test-tool urlmatch-normalization \"scheme://:\" &&\n> - ! test-tool urlmatch-normalization \"scheme://:555\" &&\n> - test-tool urlmatch-normalization \"file://user:pass@\" &&\n> - test-tool urlmatch-normalization \"file://?\" &&\n> - test-tool urlmatch-normalization \"file://#\" &&\n> - test-tool urlmatch-normalization \"file:///\" &&\n> - test-tool urlmatch-normalization \"file://:\" &&\n> - ! test-tool urlmatch-normalization \"file://:555\" &&\n> - test-tool urlmatch-normalization \"scheme://user:pass@host\" &&\n> - test-tool urlmatch-normalization \"scheme://@host\" &&\n> - test-tool urlmatch-normalization \"scheme://%00@host\" &&\n> - ! test-tool urlmatch-normalization \"scheme://%%@host\" &&\n> - test-tool urlmatch-normalization \"scheme://host_\" &&\n> - test-tool urlmatch-normalization \"scheme://user:pass@host/\" &&\n> - test-tool urlmatch-normalization \"scheme://@host/\" &&\n> - test-tool urlmatch-normalization \"scheme://host/\" &&\n> - test-tool urlmatch-normalization \"scheme://host?x\" &&\n> - test-tool urlmatch-normalization \"scheme://host#x\" &&\n> - test-tool urlmatch-normalization \"scheme://host/@\" &&\n> - test-tool urlmatch-normalization \"scheme://host?@x\" &&\n> - test-tool urlmatch-normalization \"scheme://host#@x\" &&\n> - test-tool urlmatch-normalization \"scheme://[::1]\" &&\n> - test-tool urlmatch-normalization \"scheme://[::1]/\" &&\n> - ! test-tool urlmatch-normalization \"scheme://hos%41/\" &&\n> - test-tool urlmatch-normalization \"scheme://[invalid....:/\" &&\n> - test-tool urlmatch-normalization \"scheme://invalid....:]/\" &&\n> - ! test-tool urlmatch-normalization \"scheme://invalid....:[/\" &&\n> - ! test-tool urlmatch-normalization \"scheme://invalid....:[\"\n> -'\n> -\n> -test_expect_success 'url port checks' '\n> - test-tool urlmatch-normalization \"xyz://q@some.host:\" &&\n> - test-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:0\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:0000000\" &&\n> - test-tool urlmatch-normalization \"xyz://q@some.host:0000001?\" &&\n> - test-tool urlmatch-normalization \"xyz://q@some.host:065535#\" &&\n> - test-tool urlmatch-normalization \"xyz://q@some.host:65535\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:65536\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:99999\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:100000\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:100001\" &&\n> - test-tool urlmatch-normalization \"http://q@some.host:80\" &&\n> - test-tool urlmatch-normalization \"https://q@some.host:443\" &&\n> - test-tool urlmatch-normalization \"http://q@some.host:80/\" &&\n> - test-tool urlmatch-normalization \"https://q@some.host:443?\" &&\n> - ! test-tool urlmatch-normalization \"http://q@:8008\" &&\n> - ! test-tool urlmatch-normalization \"http://:8080\" &&\n> - ! test-tool urlmatch-normalization \"http://:\" &&\n> - test-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n> - test-tool urlmatch-normalization \"xyz://[::1]:456/\" &&\n> - test-tool urlmatch-normalization \"xyz://[::1]:/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://[::1]:000/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://[::1]:0%300/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://[::1]:0x80/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://[::1]:4294967297/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://[::1]:030f/\"\n> -'\n> -\n> -test_expect_success 'url port normalization' '\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:800\")\" =\n> \"http://x:800/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:0800\")\" =\n> \"http://x:800/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:00000800\")\" =\n> \"http://x:800/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:065535\")\" =\n> \"http://x:65535/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:1\")\" =\n> \"http://x:1/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:80\")\" =\n> \"http://x/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:080\")\" =\n> \"http://x/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:000000080\")\" =\n> \"http://x/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"https://x:443\")\" =\n> \"https://x/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"https://x:0443\")\" =\n> \"https://x/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"https://x:000000443\")\" =\n> \"https://x/\"\n> -'\n> -\n> -test_expect_success 'url general escapes' '\n> - ! test-tool urlmatch-normalization \"http://x.y?%fg\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"X://W/%7e%41^%3a\")\" =\n> \"x://w/~A%5E%3A\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"X://W/:/?#[]@\")\" =\n> \"x://w/:/?#[]@\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"X://W/$&()*+,;=\")\" =\n> \"x://w/$&()*+,;=\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"X://W/'\\''\")\" =\n> \"x://w/'\\''\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"X://W?'\\!'\")\" =\n> \"x://w/?'\\!'\"\n> -'\n> -\n> -test_expect_success !MINGW 'url high-bit escapes' '\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-1\")\")\" =\n> \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-2\")\")\" =\n> \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-3\")\")\" =\n> \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-4\")\")\" =\n> \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-5\")\")\" =\n> \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-6\")\")\" =\n> \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-7\")\")\" =\n> \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-8\")\")\" =\n> \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-9\")\")\" =\n> \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-10\")\")\" =\n> \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\"\n> -'\n> -\n> -test_expect_success 'url utf-8 escapes' '\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-11\")\")\" =\n> \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\"\n> -'\n> -\n> -test_expect_success 'url username/password escapes' '\n> - test \"$(test-tool urlmatch-normalization -p\n> \"x://%41%62(^):%70+d@foo\")\" = \"x://Ab(%5E):p+d@foo/\"\n> -'\n> -\n> -test_expect_success 'url normalized lengths' '\n> - test \"$(test-tool urlmatch-normalization -l\n> \"Http://%4d%65:%4d^%70@The.Host\")\" = 25 &&\n> - test \"$(test-tool urlmatch-normalization -l\n> \"http://%41:%42@x.y/%61/\")\" = 17 &&\n> - test \"$(test-tool urlmatch-normalization -l \"http://@x.y/^\")\" = 15\n> -'\n> -\n> -test_expect_success 'url . and .. segments' '\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/.\")\" = \"x://y/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/./\")\" = \"x://y/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/.\")\" = \"x://y/a\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/./\")\" =\n> \"x://y/a/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/.?\")\" = \"x://y/?\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/./?\")\" = \"x://y/?\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/.?\")\" =\n> \"x://y/a?\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/./?\")\" =\n> \"x://y/a/?\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/.././../c\")\"\n> = \"x://y/c\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/../.././c/\")\"\n> = \"x://y/c/\" &&\n> - test \"$(test-tool urlmatch-normalization -p\n> \"x://y/a/./b/.././../c/././.././.\")\" = \"x://y/\" &&\n> - ! test-tool urlmatch-normalization \"x://y/a/./b/.././../c/././.././..\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/./?/././..\")\" =\n> \"x://y/a/?/././..\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/%2e/\")\" = \"x://y/\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/%2E/\")\" = \"x://y/\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/%2e./\")\" =\n> \"x://y/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/b/.%2E/\")\" =\n> \"x://y/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/c/%2e%2E/\")\" =\n> \"x://y/\"\n> -'\n> -\n> -# http://@foo specifies an empty user name but does not specify a\n> password\n> -# http://foo specifies neither a user name nor a password\n> -# So they should not be equivalent\n> -test_expect_success 'url equivalents' '\n> - test-tool urlmatch-normalization \"httP://x\" \"Http://X/\" &&\n> - test-tool urlmatch-normalization \"Http://%4d%65:%4d^%70@The.Host\"\n> \"hTTP://Me:%4D^p@the.HOST:80/\" &&\n> - ! test-tool urlmatch-normalization \"https://@x.y/^\"\n> \"httpS://x.y:443/^\" &&\n> - test-tool urlmatch-normalization \"https://@x.y/^\"\n> \"httpS://@x.y:0443/^\" &&\n> - test-tool urlmatch-normalization \"https://@x.y/^/../abc\"\n> \"httpS://@x.y:0443/abc\" &&\n> - test-tool urlmatch-normalization \"https://@x.y/^/..\"\n> \"httpS://@x.y:0443/\"\n> -'\n> -\n> -test_done\n> diff --git a/t/unit-tests/t-urlmatch-normalization.c\n> b/t/unit-tests/t-urlmatch-normalization.c\n> new file mode 100644\n> index 0000000000..4f225802b0\n> --- /dev/null\n> +++ b/t/unit-tests/t-urlmatch-normalization.c\n> @@ -0,0 +1,284 @@\n> +#include \"test-lib.h\"\n> +#include \"urlmatch.h\"\n> +#include \"strbuf.h\"\n> +\n> +static void check_url_normalizable(const char *url, int normalizable)\n> +{\n> + char *url_norm = url_normalize(url, NULL);\n> +\n> + if (!check_int(normalizable, ==, url_norm ? 1 : 0))\n> + test_msg(\"input url: %s\", url);\n> + free(url_norm);\n> +}\n> +\n> +static void check_normalized_url(const char *url, const char *expect)\n> +{\n> + char *url_norm = url_normalize(url, NULL);\n> +\n> + if (!check_str(url_norm, expect))\n> + test_msg(\"input url: %s\", url);\n> + free(url_norm);\n> +}\n> +\n> +static void compare_normalized_urls(const char *url1, const char *url2,\n> + size_t equal)\n> +{\n> + char *url1_norm = url_normalize(url1, NULL);\n> + char *url2_norm = url_normalize(url2, NULL);\n> +\n> + if (equal) {\n> + if (!check_str(url1_norm, url2_norm))\n> + test_msg(\"input url1: %s\\n input url2: %s\", url1,\n> + url2);\n> + } else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0))\n> + test_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n> + \" input url1: %s\\n input url2: %s\",\n> + url1_norm, url2_norm, url1, url2);\n> + free(url1_norm);\n> + free(url2_norm);\n> +}\n> +\n> +static void check_normalized_url_from_file(const char *file, const char\n> *expect)\n> +{\n> + struct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n> +\n> + strbuf_getcwd(&path);\n> + strbuf_strip_suffix(&path, \"/unit-tests/bin\"); /* because\n> 'unit-tests-test-tool' is run from 'bin' directory */\n> + strbuf_addf(&path, \"/unit-tests/t-urlmatch-normalization/%s\", file);\n> +\n> + if (!check_int(strbuf_read_file(&content, path.buf, 0), >, 0)) {\n> + test_msg(\"failed to read from file '%s': %s\", file, strerror(errno));\n> + } else {\n> + char *url_norm;\n> +\n> + strbuf_trim_trailing_newline(&content);\n> + url_norm = url_normalize(content.buf, NULL);\n> + if (!check_str(url_norm, expect))\n> + test_msg(\"input file: %s\", file);\n> + free(url_norm);\n> + }\n> +\n> + strbuf_release(&content);\n> + strbuf_release(&path);\n> +}\n> +\n> +static void check_normalized_url_length(const char *url, size_t len)\n> +{\n> + struct url_info info;\n> + char *url_norm = url_normalize(url, &info);\n> +\n> + if (!check_int(info.url_len, ==, len))\n> + test_msg(\" input url: %s\\n normalized url: %s\", url,\n> + url_norm);\n> + free(url_norm);\n> +}\n> +\n> +/* Note that only file: URLs should be allowed without a host */\n> +static void t_url_scheme(void)\n> +{\n> + check_url_normalizable(\"\", 0);\n> + check_url_normalizable(\"_\", 0);\n> + check_url_normalizable(\"scheme\", 0);\n> + check_url_normalizable(\"scheme:\", 0);\n> + check_url_normalizable(\"scheme:/\", 0);\n> + check_url_normalizable(\"scheme://\", 0);\n> + check_url_normalizable(\"file\", 0);\n> + check_url_normalizable(\"file:\", 0);\n> + check_url_normalizable(\"file:/\", 0);\n> + check_url_normalizable(\"file://\", 1);\n> + check_url_normalizable(\"://acme.co\", 0);\n> + check_url_normalizable(\"x_test://acme.co\", 0);\n> + check_url_normalizable(\"-test://acme.co\", 0);\n> + check_url_normalizable(\"0test://acme.co\", 0);\n> + check_url_normalizable(\"+test://acme.co\", 0);\n> + check_url_normalizable(\".test://acme.co\", 0);\n> + check_url_normalizable(\"schem%6e://\", 0);\n> + check_url_normalizable(\"x-Test+v1.0://acme.co\", 1);\n> + check_normalized_url(\"AbCdeF://x.Y\", \"abcdef://x.y/\");\n> +}\n> +\n> +static void t_url_authority(void)\n> +{\n> + check_url_normalizable(\"scheme://user:pass@\", 0);\n> + check_url_normalizable(\"scheme://?\", 0);\n> + check_url_normalizable(\"scheme://#\", 0);\n> + check_url_normalizable(\"scheme:///\", 0);\n> + check_url_normalizable(\"scheme://:\", 0);\n> + check_url_normalizable(\"scheme://:555\", 0);\n> + check_url_normalizable(\"file://user:pass@\", 1);\n> + check_url_normalizable(\"file://?\", 1);\n> + check_url_normalizable(\"file://#\", 1);\n> + check_url_normalizable(\"file:///\", 1);\n> + check_url_normalizable(\"file://:\", 1);\n> + check_url_normalizable(\"file://:555\", 0);\n> + check_url_normalizable(\"scheme://user:pass@host\", 1);\n> + check_url_normalizable(\"scheme://@host\", 1);\n> + check_url_normalizable(\"scheme://%00@host\", 1);\n> + check_url_normalizable(\"scheme://%%@host\", 0);\n> + check_url_normalizable(\"scheme://host_\", 1);\n> + check_url_normalizable(\"scheme://user:pass@host/\", 1);\n> + check_url_normalizable(\"scheme://@host/\", 1);\n> + check_url_normalizable(\"scheme://host/\", 1);\n> + check_url_normalizable(\"scheme://host?x\", 1);\n> + check_url_normalizable(\"scheme://host#x\", 1);\n> + check_url_normalizable(\"scheme://host/@\", 1);\n> + check_url_normalizable(\"scheme://host?@x\", 1);\n> + check_url_normalizable(\"scheme://host#@x\", 1);\n> + check_url_normalizable(\"scheme://[::1]\", 1);\n> + check_url_normalizable(\"scheme://[::1]/\", 1);\n> + check_url_normalizable(\"scheme://hos%41/\", 0);\n> + check_url_normalizable(\"scheme://[invalid....:/\", 1);\n> + check_url_normalizable(\"scheme://invalid....:]/\", 1);\n> + check_url_normalizable(\"scheme://invalid....:[/\", 0);\n> + check_url_normalizable(\"scheme://invalid....:[\", 0);\n> +}\n> +\n> +static void t_url_port(void)\n> +{\n> + check_url_normalizable(\"xyz://q@some.host:\", 1);\n> + check_url_normalizable(\"xyz://q@some.host:456/\", 1);\n> + check_url_normalizable(\"xyz://q@some.host:0\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:0000000\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:0000001?\", 1);\n> + check_url_normalizable(\"xyz://q@some.host:065535#\", 1);\n> + check_url_normalizable(\"xyz://q@some.host:65535\", 1);\n> + check_url_normalizable(\"xyz://q@some.host:65536\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:99999\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:100000\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:100001\", 0);\n> + check_url_normalizable(\"http://q@some.host:80\", 1);\n> + check_url_normalizable(\"https://q@some.host:443\", 1);\n> + check_url_normalizable(\"http://q@some.host:80/\", 1);\n> + check_url_normalizable(\"https://q@some.host:443?\", 1);\n> + check_url_normalizable(\"http://q@:8008\", 0);\n> + check_url_normalizable(\"http://:8080\", 0);\n> + check_url_normalizable(\"http://:\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:456/\", 1);\n> + check_url_normalizable(\"xyz://[::1]:456/\", 1);\n> + check_url_normalizable(\"xyz://[::1]:/\", 1);\n> + check_url_normalizable(\"xyz://[::1]:000/\", 0);\n> + check_url_normalizable(\"xyz://[::1]:0%300/\", 0);\n> + check_url_normalizable(\"xyz://[::1]:0x80/\", 0);\n> + check_url_normalizable(\"xyz://[::1]:4294967297/\", 0);\n> + check_url_normalizable(\"xyz://[::1]:030f/\", 0);\n> +}\n> +\n> +static void t_url_port_normalization(void)\n> +{\n> + check_normalized_url(\"http://x:800\", \"http://x:800/\");\n> + check_normalized_url(\"http://x:0800\", \"http://x:800/\");\n> + check_normalized_url(\"http://x:00000800\", \"http://x:800/\");\n> + check_normalized_url(\"http://x:065535\", \"http://x:65535/\");\n> + check_normalized_url(\"http://x:1\", \"http://x:1/\");\n> + check_normalized_url(\"http://x:80\", \"http://x/\");\n> + check_normalized_url(\"http://x:080\", \"http://x/\");\n> + check_normalized_url(\"http://x:000000080\", \"http://x/\");\n> + check_normalized_url(\"https://x:443\", \"https://x/\");\n> + check_normalized_url(\"https://x:0443\", \"https://x/\");\n> + check_normalized_url(\"https://x:000000443\", \"https://x/\");\n> +}\n> +\n> +static void t_url_general_escape(void)\n> +{\n> + check_url_normalizable(\"http://x.y?%fg\", 0);\n> + check_normalized_url(\"X://W/%7e%41^%3a\", \"x://w/~A%5E%3A\");\n> + check_normalized_url(\"X://W/:/?#[]@\", \"x://w/:/?#[]@\");\n> + check_normalized_url(\"X://W/$&()*+,;=\", \"x://w/$&()*+,;=\");\n> + check_normalized_url(\"X://W/'\", \"x://w/'\");\n> + check_normalized_url(\"X://W?!\", \"x://w/?!\");\n> +}\n> +\n> +static void t_url_high_bit(void)\n> +{\n> + check_normalized_url_from_file(\"url-1\",\n> + \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\");\n> + check_normalized_url_from_file(\"url-2\",\n> + \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\");\n> + check_normalized_url_from_file(\"url-3\",\n> + \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\");\n> + check_normalized_url_from_file(\"url-4\",\n> + \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\");\n> + check_normalized_url_from_file(\"url-5\",\n> + \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\");\n> + check_normalized_url_from_file(\"url-6\",\n> + \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\");\n> + check_normalized_url_from_file(\"url-7\",\n> + \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\");\n> + check_normalized_url_from_file(\"url-8\",\n> + \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\");\n> + check_normalized_url_from_file(\"url-9\",\n> + \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\");\n> + check_normalized_url_from_file(\"url-10\",\n> + \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\");\n> +}\n> +\n> +static void t_url_utf8_escape(void)\n> +{\n> + check_normalized_url_from_file(\"url-11\",\n> + \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\");\n> +}\n> +\n> +static void t_url_username_pass(void)\n> +{\n> + check_normalized_url(\"x://%41%62(^):%70+d@foo\",\n> \"x://Ab(%5E):p+d@foo/\");\n> +}\n> +\n> +static void t_url_length(void)\n> +{\n> + check_normalized_url_length(\"Http://%4d%65:%4d^%70@The.Host\", 25);\n> + check_normalized_url_length(\"http://%41:%42@x.y/%61/\", 17);\n> + check_normalized_url_length(\"http://@x.y/^\", 15);\n> +}\n> +\n> +static void t_url_dots(void)\n> +{\n> + check_normalized_url(\"x://y/.\", \"x://y/\");\n> + check_normalized_url(\"x://y/./\", \"x://y/\");\n> + check_normalized_url(\"x://y/a/.\", \"x://y/a\");\n> + check_normalized_url(\"x://y/a/./\", \"x://y/a/\");\n> + check_normalized_url(\"x://y/.?\", \"x://y/?\");\n> + check_normalized_url(\"x://y/./?\", \"x://y/?\");\n> + check_normalized_url(\"x://y/a/.?\", \"x://y/a?\");\n> + check_normalized_url(\"x://y/a/./?\", \"x://y/a/?\");\n> + check_normalized_url(\"x://y/a/./b/.././../c\", \"x://y/c\");\n> + check_normalized_url(\"x://y/a/./b/../.././c/\", \"x://y/c/\");\n> + check_normalized_url(\"x://y/a/./b/.././../c/././.././.\", \"x://y/\");\n> + check_url_normalizable(\"x://y/a/./b/.././../c/././.././..\", 0);\n> + check_normalized_url(\"x://y/a/./?/././..\", \"x://y/a/?/././..\");\n> + check_normalized_url(\"x://y/%2e/\", \"x://y/\");\n> + check_normalized_url(\"x://y/%2E/\", \"x://y/\");\n> + check_normalized_url(\"x://y/a/%2e./\", \"x://y/\");\n> + check_normalized_url(\"x://y/b/.%2E/\", \"x://y/\");\n> + check_normalized_url(\"x://y/c/%2e%2E/\", \"x://y/\");\n> +}\n> +\n> +/*\n> + * http://@foo specifies an empty user name but does not specify a\n> password\n> + * http://foo specifies neither a user name nor a password\n> + * So they should not be equivalent\n> + */\n> +static void t_url_equivalents(void)\n> +{\n> + compare_normalized_urls(\"httP://x\", \"Http://X/\", 1);\n> + compare_normalized_urls(\"Http://%4d%65:%4d^%70@The.Host\",\n> \"hTTP://Me:%4D^p@the.HOST:80/\", 1);\n> + compare_normalized_urls(\"https://@x.y/^\", \"httpS://x.y:443/^\", 0);\n> + compare_normalized_urls(\"https://@x.y/^\", \"httpS://@x.y:0443/^\", 1);\n> + compare_normalized_urls(\"https://@x.y/^/../abc\",\n> \"httpS://@x.y:0443/abc\", 1);\n> + compare_normalized_urls(\"https://@x.y/^/..\", \"httpS://@x.y:0443/\", 1);\n> +}\n> +\n> +int cmd_main(int argc UNUSED, const char **argv UNUSED)\n> +{\n> + TEST(t_url_scheme(), \"url scheme\");\n> + TEST(t_url_authority(), \"url authority\");\n> + TEST(t_url_port(), \"url port checks\");\n> + TEST(t_url_port_normalization(), \"url port normalization\");\n> + TEST(t_url_general_escape(), \"url general escapes\");\n> + TEST(t_url_high_bit(), \"url high-bit escapes\");\n> + TEST(t_url_utf8_escape(), \"url utf8 escapes\");\n> + TEST(t_url_username_pass(), \"url username/password escapes\");\n> + TEST(t_url_length(), \"url normalized lengths\");\n> + TEST(t_url_dots(), \"url . and .. segments\");\n> + TEST(t_url_equivalents(), \"url equivalents\");\n> + return test_done();\n> +}\n> diff --git a/t/t0110/README\n> b/t/unit-tests/t-urlmatch-normalization/README\n> similarity index 100%\n> rename from t/t0110/README\n> rename to t/unit-tests/t-urlmatch-normalization/README\n> diff --git a/t/t0110/url-1 b/t/unit-tests/t-urlmatch-normalization/url-1\n> similarity index 100%\n> rename from t/t0110/url-1\n> rename to t/unit-tests/t-urlmatch-normalization/url-1\n> diff --git a/t/t0110/url-10\n> b/t/unit-tests/t-urlmatch-normalization/url-10\n> similarity index 100%\n> rename from t/t0110/url-10\n> rename to t/unit-tests/t-urlmatch-normalization/url-10\n> diff --git a/t/t0110/url-11\n> b/t/unit-tests/t-urlmatch-normalization/url-11\n> similarity index 100%\n> rename from t/t0110/url-11\n> rename to t/unit-tests/t-urlmatch-normalization/url-11\n> diff --git a/t/t0110/url-2 b/t/unit-tests/t-urlmatch-normalization/url-2\n> similarity index 100%\n> rename from t/t0110/url-2\n> rename to t/unit-tests/t-urlmatch-normalization/url-2\n> diff --git a/t/t0110/url-3 b/t/unit-tests/t-urlmatch-normalization/url-3\n> similarity index 100%\n> rename from t/t0110/url-3\n> rename to t/unit-tests/t-urlmatch-normalization/url-3\n> diff --git a/t/t0110/url-4 b/t/unit-tests/t-urlmatch-normalization/url-4\n> similarity index 100%\n> rename from t/t0110/url-4\n> rename to t/unit-tests/t-urlmatch-normalization/url-4\n> diff --git a/t/t0110/url-5 b/t/unit-tests/t-urlmatch-normalization/url-5\n> similarity index 100%\n> rename from t/t0110/url-5\n> rename to t/unit-tests/t-urlmatch-normalization/url-5\n> diff --git a/t/t0110/url-6 b/t/unit-tests/t-urlmatch-normalization/url-6\n> similarity index 100%\n> rename from t/t0110/url-6\n> rename to t/unit-tests/t-urlmatch-normalization/url-6\n> diff --git a/t/t0110/url-7 b/t/unit-tests/t-urlmatch-normalization/url-7\n> similarity index 100%\n> rename from t/t0110/url-7\n> rename to t/unit-tests/t-urlmatch-normalization/url-7\n> diff --git a/t/t0110/url-8 b/t/unit-tests/t-urlmatch-normalization/url-8\n> similarity index 100%\n> rename from t/t0110/url-8\n> rename to t/unit-tests/t-urlmatch-normalization/url-8\n> diff --git a/t/t0110/url-9 b/t/unit-tests/t-urlmatch-normalization/url-9\n> similarity index 100%\n> rename from t/t0110/url-9\n> rename to t/unit-tests/t-urlmatch-normalization/url-9\n> --\n> 2.45.2\n\n"},{"id":"499053","messageId":"CAOLa=ZQ8fuCksk+CGP2thMqKnT_KME4goLwEt6hKYvt+Gbn5tw@mail.gmail.com","threadId":"61693","inReplyTo":"20240628125632.45603-1-shyamthakkar001@gmail.com","subject":"Re: [GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-07-22T12:53:04Z","receivedAt":"2024-07-22T12:53:05Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Ghanshyam Thakkar writes:\n\n[snip]\n\n> +static void compare_normalized_urls(const char *url1, const char *url2,\n> +\t\t\t\t size_t equal)\n[ 5 more citation lines. Click/Enter to show. ]\n> +{\n> +\tchar *url1_norm = url_normalize(url1, NULL);\n> +\tchar *url2_norm = url_normalize(url2, NULL);\n> +\n> +\tif (equal) {\n> +\t\tif (!check_str(url1_norm, url2_norm))\n> +\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n> +\t\t\t\t url2);\n\ncheck_str() checks and prints the values if they don't match, so here\nsince the normalized urls will be printed by check_str(), we print the\ninput urls. Makes sense.\n\n> +\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0))\n> +\t\ttest_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n> +\t\t\t \" input url1: %s\\n input url2: %s\",\n> +\t\t\t url1_norm, url2_norm, url1, url2);\n\nHere we use strcmp and hence, it won't print the normalized urls, so we\nalso print them. This is because we want to make sure they are not\nequal.\n\nI don't understand why there is inconsistent spacing in this message\nthough.\n\nApart from this small question, the patch looks great!\n\nThanks\n"},{"id":"499054","messageId":"D2W30F19QN7J.146CV16OMTO6M@gmail.com","threadId":"61693","inReplyTo":"CAOLa=ZQ8fuCksk+CGP2thMqKnT_KME4goLwEt6hKYvt+Gbn5tw@mail.gmail.com","subject":"Re: [GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","fromName":"Ghanshyam Thakkar","fromEmail":"shyamthakkar001@gmail.com","sentAt":"2024-07-22T12:54:41Z","receivedAt":"2024-07-22T12:54:47Z","isPatch":true,"sender":{"key":"shyamthakkar001@gmail.com","avatar":"https://avatars.githubusercontent.com/u/72698233?v=4"},"body":"Karthik Nayak wrote:\n> Ghanshyam Thakkar writes:\n>\n> [snip]\n>\n> > +static void compare_normalized_urls(const char *url1, const char *url2,\n> > +\t\t\t\t size_t equal)\n> [ 5 more citation lines. Click/Enter to show. ]\n> > +{\n> > +\tchar *url1_norm = url_normalize(url1, NULL);\n> > +\tchar *url2_norm = url_normalize(url2, NULL);\n> > +\n> > +\tif (equal) {\n> > +\t\tif (!check_str(url1_norm, url2_norm))\n> > +\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n> > +\t\t\t\t url2);\n>\n> check_str() checks and prints the values if they don't match, so here\n> since the normalized urls will be printed by check_str(), we print the\n> input urls. Makes sense.\n>\n> > +\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0))\n> > +\t\ttest_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n> > +\t\t\t \" input url1: %s\\n input url2: %s\",\n> > +\t\t\t url1_norm, url2_norm, url1, url2);\n>\n> Here we use strcmp and hence, it won't print the normalized urls, so we\n> also print them. This is because we want to make sure they are not\n> equal.\n>\n> I don't understand why there is inconsistent spacing in this message\n> though.\n\nThat is for alignment purposes, so the ':' matches vertically between\nthem. I.e.\n\n# url1_norm: https://@x.y/%5E\n url2_norm: https://x.y/%5E\n input url1: https://@x.y/^\n input url2: httpS://x.y:443/^\n \nThanks.\n\n> Apart from this small question, the patch looks great!\n>\n> Thanks\n\n"},{"id":"499136","messageId":"CAOLa=ZS_rNEL66aJJgjdq8p-1dLBm22xPJWYH-iDNLO30E6Dew@mail.gmail.com","threadId":"61693","inReplyTo":"D2W30F19QN7J.146CV16OMTO6M@gmail.com","subject":"Re: [GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-07-23T08:26:28Z","receivedAt":"2024-07-23T08:26:29Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"\"Ghanshyam Thakkar\" writes:\n\n> Karthik Nayak wrote:\n>> Ghanshyam Thakkar writes:\n>>\n>> [snip]\n>>\n>> > +static void compare_normalized_urls(const char *url1, const char *url2,\n>> > +\t\t\t\t size_t equal)\n>> [ 5 more citation lines. Click/Enter to show. ]\n>> > +{\n>> > +\tchar *url1_norm = url_normalize(url1, NULL);\n>> > +\tchar *url2_norm = url_normalize(url2, NULL);\n>> > +\n>> > +\tif (equal) {\n>> > +\t\tif (!check_str(url1_norm, url2_norm))\n>> > +\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n>> > +\t\t\t\t url2);\n>>\n>> check_str() checks and prints the values if they don't match, so here\n>> since the normalized urls will be printed by check_str(), we print the\n>> input urls. Makes sense.\n>>\n>> > +\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0))\n>> > +\t\ttest_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n>> > +\t\t\t \" input url1: %s\\n input url2: %s\",\n>> > +\t\t\t url1_norm, url2_norm, url1, url2);\n>>\n>> Here we use strcmp and hence, it won't print the normalized urls, so we\n>> also print them. This is because we want to make sure they are not\n>> equal.\n>>\n>> I don't understand why there is inconsistent spacing in this message\n>> though.\n>\n> That is for alignment purposes, so the ':' matches vertically between\n> them. I.e.\n>\n> # url1_norm: https://@x.y/%5E\n> url2_norm: https://x.y/%5E\n> input url1: https://@x.y/^\n> input url2: httpS://x.y:443/^\n>\n> Thanks.\n>\n\nYeah makes sense, I was expecting it to be left aligned, but this looks\ngood! Thanks.\n"},{"id":"499157","messageId":"Zp-3e6VV5bl8dWvR@tanuki","threadId":"61693","inReplyTo":"20240628125632.45603-1-shyamthakkar001@gmail.com","subject":"Re: [GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-07-23T14:00:27Z","receivedAt":"2024-07-23T14:00:35Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Jun 28, 2024 at 06:26:24PM +0530, Ghanshyam Thakkar wrote:\n> +static void compare_normalized_urls(const char *url1, const char *url2,\n> +\t\t\t\t size_t equal)\n> +{\n> +\tchar *url1_norm = url_normalize(url1, NULL);\n> +\tchar *url2_norm = url_normalize(url2, NULL);\n> +\n> +\tif (equal) {\n> +\t\tif (!check_str(url1_norm, url2_norm))\n> +\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n> +\t\t\t\t url2);\n> +\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0))\n> +\t\ttest_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n> +\t\t\t \" input url1: %s\\n input url2: %s\",\n> +\t\t\t url1_norm, url2_norm, url1, url2);\n\nNit: this is missing braces around the `else if` branch. If one of the\nconditional bodies has braces, then all should have according to our\nstyle guide.\n\n> +\tfree(url1_norm);\n> +\tfree(url2_norm);\n> +}\n> +\n> +static void check_normalized_url_from_file(const char *file, const char *expect)\n> +{\n> +\tstruct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n> +\n> +\tstrbuf_getcwd(&path);\n> +\tstrbuf_strip_suffix(&path, \"/unit-tests/bin\"); /* because 'unit-tests-test-tool' is run from 'bin' directory */\n\nCurious: is this a new requirement or do other tests have the same\nrequirement? I was under the impression that I could execude the\nresulting unit test binaries from whatever directory I wanted to, but\ndidn't verify.\n\nIn any case, the line should probably be wrapped as it is overly long.\n\nOther than that this looks good to me. I've gave a cursory read to the\ntestcases themselves and they do look like a faithful conversion to me.\n\nThanks!\n\nPatrick\n"},{"id":"499213","messageId":"D2XCB5UYIB5B.1L3U95DQSWJ2@gmail.com","threadId":"61693","inReplyTo":"Zp-3e6VV5bl8dWvR@tanuki","subject":"Re: [GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","fromName":"Ghanshyam Thakkar","fromEmail":"shyamthakkar001@gmail.com","sentAt":"2024-07-24T00:24:33Z","receivedAt":"2024-07-24T00:24:39Z","isPatch":true,"sender":{"key":"shyamthakkar001@gmail.com","avatar":"https://avatars.githubusercontent.com/u/72698233?v=4"},"body":"Patrick Steinhardt wrote:\n> On Fri, Jun 28, 2024 at 06:26:24PM +0530, Ghanshyam Thakkar wrote:\n> > +static void compare_normalized_urls(const char *url1, const char *url2,\n> > +\t\t\t\t size_t equal)\n> > +{\n> > +\tchar *url1_norm = url_normalize(url1, NULL);\n> > +\tchar *url2_norm = url_normalize(url2, NULL);\n> > +\n> > +\tif (equal) {\n> > +\t\tif (!check_str(url1_norm, url2_norm))\n> > +\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n> > +\t\t\t\t url2);\n> > +\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0))\n> > +\t\ttest_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n> > +\t\t\t \" input url1: %s\\n input url2: %s\",\n> > +\t\t\t url1_norm, url2_norm, url1, url2);\n>\n> Nit: this is missing braces around the `else if` branch. If one of the\n> conditional bodies has braces, then all should have according to our\n> style guide.\n\nWill update.\n\n>\n> > +\tfree(url1_norm);\n> > +\tfree(url2_norm);\n> > +}\n> > +\n> > +static void check_normalized_url_from_file(const char *file, const char *expect)\n> > +{\n> > +\tstruct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n> > +\n> > +\tstrbuf_getcwd(&path);\n> > +\tstrbuf_strip_suffix(&path, \"/unit-tests/bin\"); /* because 'unit-tests-test-tool' is run from 'bin' directory */\n>\n> Curious: is this a new requirement or do other tests have the same\n> requirement? I was under the impression that I could execude the\n> resulting unit test binaries from whatever directory I wanted to, but\n> didn't verify.\n\nI am not aware of any requirements, but if we want to interact with\nother files like in this case (and where we potentially have to\ninteract with a test repository), we'd need to have some requirement\nto construct the path to these data files (and the test repository),\nsimilar to end-to-end tests where they can be run in only t/\ndirectory. Do you think calling `setup_git_directory()` and then using\n`the_repository->worktree` to get the root of the worktree of Git source\nand then construct the path relative to that, would be useful? That way\nwe can atleast call the binaries from anywhere within the tree.\n\n(P.S. I know we want to avoid using `the_repository`, but I don't know\nany other way yet.)\n\n>\n> In any case, the line should probably be wrapped as it is overly long.\n\nWill update.\n\nThank you.\n\n>\n> Other than that this looks good to me. I've gave a cursory read to the\n> testcases themselves and they do look like a faithful conversion to me.\n>\n> Thanks!\n>\n> Patrick\n\n"},{"id":"499221","messageId":"ZqCO6JTQqdxD73Wq@tanuki","threadId":"61693","inReplyTo":"D2XCB5UYIB5B.1L3U95DQSWJ2@gmail.com","subject":"Re: [GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-07-24T05:19:36Z","receivedAt":"2024-07-24T05:19:41Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Jul 24, 2024 at 05:54:33AM +0530, Ghanshyam Thakkar wrote:\n> Patrick Steinhardt wrote:\n> > On Fri, Jun 28, 2024 at 06:26:24PM +0530, Ghanshyam Thakkar wrote:\n> > > +\tfree(url1_norm);\n> > > +\tfree(url2_norm);\n> > > +}\n> > > +\n> > > +static void check_normalized_url_from_file(const char *file, const char *expect)\n> > > +{\n> > > +\tstruct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n> > > +\n> > > +\tstrbuf_getcwd(&path);\n> > > +\tstrbuf_strip_suffix(&path, \"/unit-tests/bin\"); /* because 'unit-tests-test-tool' is run from 'bin' directory */\n> >\n> > Curious: is this a new requirement or do other tests have the same\n> > requirement? I was under the impression that I could execude the\n> > resulting unit test binaries from whatever directory I wanted to, but\n> > didn't verify.\n> \n> I am not aware of any requirements, but if we want to interact with\n> other files like in this case (and where we potentially have to\n> interact with a test repository), we'd need to have some requirement\n> to construct the path to these data files (and the test repository),\n> similar to end-to-end tests where they can be run in only t/\n> directory. Do you think calling `setup_git_directory()` and then using\n> `the_repository->worktree` to get the root of the worktree of Git source\n> and then construct the path relative to that, would be useful? That way\n> we can atleast call the binaries from anywhere within the tree.\n\nInstead of using the working directory, you can also use the `__FILE__`\npreprocessor macro to access the files relative to the directory of the\noriginal source file. That at least makes it possible to execute the\nresult from all directories, but still obviously ties us to the location\nof the source directory.\n\nWhether that's ultimately much better.. dunno. But I guess this should\nat least be discussed in the commit message.\n\n> (P.S. I know we want to avoid using `the_repository`, but I don't know\n> any other way yet.)\n\nYou can use e.g. \"t/helper/test-repository.c\" as an example, where we\nuse `repo_init()` to initialize a local repository variable.\n\nPatrick\n"},{"id":"499225","messageId":"D2XKUY6MQJOR.2B3YHEXQPOQQL@gmail.com","threadId":"61693","inReplyTo":"ZqCO6JTQqdxD73Wq@tanuki","subject":"Re: [GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","fromName":"Ghanshyam Thakkar","fromEmail":"shyamthakkar001@gmail.com","sentAt":"2024-07-24T07:06:32Z","receivedAt":"2024-07-24T07:06:38Z","isPatch":true,"sender":{"key":"shyamthakkar001@gmail.com","avatar":"https://avatars.githubusercontent.com/u/72698233?v=4"},"body":"Patrick Steinhardt wrote:\n> On Wed, Jul 24, 2024 at 05:54:33AM +0530, Ghanshyam Thakkar wrote:\n> > Patrick Steinhardt wrote:\n> > > On Fri, Jun 28, 2024 at 06:26:24PM +0530, Ghanshyam Thakkar wrote:\n> > > > +\tfree(url1_norm);\n> > > > +\tfree(url2_norm);\n> > > > +}\n> > > > +\n> > > > +static void check_normalized_url_from_file(const char *file, const char *expect)\n> > > > +{\n> > > > +\tstruct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n> > > > +\n> > > > +\tstrbuf_getcwd(&path);\n> > > > +\tstrbuf_strip_suffix(&path, \"/unit-tests/bin\"); /* because 'unit-tests-test-tool' is run from 'bin' directory */\n> > >\n> > > Curious: is this a new requirement or do other tests have the same\n> > > requirement? I was under the impression that I could execude the\n> > > resulting unit test binaries from whatever directory I wanted to, but\n> > > didn't verify.\n> > \n> > I am not aware of any requirements, but if we want to interact with\n> > other files like in this case (and where we potentially have to\n> > interact with a test repository), we'd need to have some requirement\n> > to construct the path to these data files (and the test repository),\n> > similar to end-to-end tests where they can be run in only t/\n> > directory. Do you think calling `setup_git_directory()` and then using\n> > `the_repository->worktree` to get the root of the worktree of Git source\n> > and then construct the path relative to that, would be useful? That way\n> > we can atleast call the binaries from anywhere within the tree.\n>\n> Instead of using the working directory, you can also use the `__FILE__`\n> preprocessor macro to access the files relative to the directory of the\n> original source file. That at least makes it possible to execute the\n> result from all directories, but still obviously ties us to the location\n> of the source directory.\n\nBut doesn't '__FILE__' give relative path instead of absolute? A quick\ntest_msg() tells me that '__FILE__' gives the path\n't/unit-tests/t-urlmatch-normalization.c' for me. So, I don't know\nhow we would be able to execute from _all_ directories. Although, I\nthink the restriction of running from only 't/' would be fine as\nend-to-end tests have similar restrictions.\n\n> Whether that's ultimately much better.. dunno. But I guess this should\n> at least be discussed in the commit message.\n\nWill update.\n\n>\n> > (P.S. I know we want to avoid using `the_repository`, but I don't know\n> > any other way yet.)\n>\n> You can use e.g. \"t/helper/test-repository.c\" as an example, where we\n> use `repo_init()` to initialize a local repository variable.\n\nBut it requires us to know the path to the repo and worktree in\nadvance, which kinda defeats the purpose of using 'repository->worktree'.\nsetup_git_directory() sets up 'the_repository' from any subdirectory\nof the worktree, so we can get the root without us having to know\nwhich sub-directory (of the worktree) we are in.\n\nThanks.\n"},{"id":"499227","messageId":"ZqCxAZ2O7SkX_eNB@tanuki","threadId":"61693","inReplyTo":"D2XKUY6MQJOR.2B3YHEXQPOQQL@gmail.com","subject":"Re: [GSoC][PATCH] t: migrate helper/test-urlmatch-normalization to unit tests","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-07-24T07:45:05Z","receivedAt":"2024-07-24T07:45:10Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Jul 24, 2024 at 12:36:32PM +0530, Ghanshyam Thakkar wrote:\n> Patrick Steinhardt wrote:\n> > On Wed, Jul 24, 2024 at 05:54:33AM +0530, Ghanshyam Thakkar wrote:\n> > > Patrick Steinhardt wrote:\n> > > > On Fri, Jun 28, 2024 at 06:26:24PM +0530, Ghanshyam Thakkar wrote:\n> > > > > +\tfree(url1_norm);\n> > > > > +\tfree(url2_norm);\n> > > > > +}\n> > > > > +\n> > > > > +static void check_normalized_url_from_file(const char *file, const char *expect)\n> > > > > +{\n> > > > > +\tstruct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n> > > > > +\n> > > > > +\tstrbuf_getcwd(&path);\n> > > > > +\tstrbuf_strip_suffix(&path, \"/unit-tests/bin\"); /* because 'unit-tests-test-tool' is run from 'bin' directory */\n> > > >\n> > > > Curious: is this a new requirement or do other tests have the same\n> > > > requirement? I was under the impression that I could execude the\n> > > > resulting unit test binaries from whatever directory I wanted to, but\n> > > > didn't verify.\n> > > \n> > > I am not aware of any requirements, but if we want to interact with\n> > > other files like in this case (and where we potentially have to\n> > > interact with a test repository), we'd need to have some requirement\n> > > to construct the path to these data files (and the test repository),\n> > > similar to end-to-end tests where they can be run in only t/\n> > > directory. Do you think calling `setup_git_directory()` and then using\n> > > `the_repository->worktree` to get the root of the worktree of Git source\n> > > and then construct the path relative to that, would be useful? That way\n> > > we can atleast call the binaries from anywhere within the tree.\n> >\n> > Instead of using the working directory, you can also use the `__FILE__`\n> > preprocessor macro to access the files relative to the directory of the\n> > original source file. That at least makes it possible to execute the\n> > result from all directories, but still obviously ties us to the location\n> > of the source directory.\n> \n> But doesn't '__FILE__' give relative path instead of absolute? A quick\n> test_msg() tells me that '__FILE__' gives the path\n> 't/unit-tests/t-urlmatch-normalization.c' for me. So, I don't know\n> how we would be able to execute from _all_ directories. Although, I\n> think the restriction of running from only 't/' would be fine as\n> end-to-end tests have similar restrictions.\n\nAh, you're right of course. Scratch that then.\n\nPatrick\n"},{"id":"500810","messageId":"20240813172432.55487-1-shyamthakkar001@gmail.com","threadId":"61693","inReplyTo":"20240628125632.45603-1-shyamthakkar001@gmail.com","subject":"[GSoC][PATCH v2] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Ghanshyam Thakkar","fromEmail":"shyamthakkar001@gmail.com","sentAt":"2024-08-13T17:24:21Z","receivedAt":"2024-08-13T17:25:24Z","isPatch":true,"sender":{"key":"shyamthakkar001@gmail.com","avatar":"https://avatars.githubusercontent.com/u/72698233?v=4"},"body":"helper/test-urlmatch-normalization along with\nt0110-urlmatch-normalization test the `url_normalize()` function from\n'urlmatch.h'. Migrate them to the unit testing framework for better\nperformance. And also add different test_msg()s for better debugging.\n\nIn the migration, last two of the checks from `t_url_general_escape()`\nwere slightly changed compared to the shellscript. This involves changing\n\n'\\'' -> '\n'\\!' -> !\n\nin the urls of those checks. This is because in C strings, we don't\nneed to escape \"'\" and \"!\". Other than these two, all the urls were\npasted verbatim from the shellscript.\n\nAnother change is the removal of MINGW prerequisite from one of the\ntest. It was there because[1] on Windows, the command line is a Unicode\nstring, it is not possible to pass arbitrary bytes to a program. But\nin unit tests we don't have this limitation.\n\nWith the addition of this unit test, we impose a new restriction of\nrunning the unit tests from either 't/' or 't/unit-tests/bin/'\ndirectories. This is to construct the path to files which contain some\ninput urls under the 't/t-urlmatch-normalization' directory. This\nrestriction is similar to one we have for end-to-end tests, where they\ncan be ran from only 't/'. Addition of 't/unit-tests/bin/' is to allow\nfor running individual tests which is not currently possible via any\n'make' targets and also 'unit-tests-test-tool' target is also ran from\nthe 't/unit-tests/bin' directory.\n\n[1]: https://lore.kernel.org/git/53CAC8EF.6020707@gmail.com/\n\nMentored-by: Christian Couder \nMentored-by: Kaartic Sivaraam \nSigned-off-by: Ghanshyam Thakkar \n---\n Makefile | 2 +-\n t/helper/test-tool.c | 1 -\n t/helper/test-tool.h | 1 -\n t/helper/test-urlmatch-normalization.c | 56 ----\n t/t0110-urlmatch-normalization.sh | 182 -----------\n t/unit-tests/t-urlmatch-normalization.c | 294 ++++++++++++++++++\n .../t-urlmatch-normalization}/README | 0\n .../t-urlmatch-normalization}/url-1 | Bin\n .../t-urlmatch-normalization}/url-10 | Bin\n .../t-urlmatch-normalization}/url-11 | Bin\n .../t-urlmatch-normalization}/url-2 | Bin\n .../t-urlmatch-normalization}/url-3 | Bin\n .../t-urlmatch-normalization}/url-4 | Bin\n .../t-urlmatch-normalization}/url-5 | Bin\n .../t-urlmatch-normalization}/url-6 | Bin\n .../t-urlmatch-normalization}/url-7 | Bin\n .../t-urlmatch-normalization}/url-8 | Bin\n .../t-urlmatch-normalization}/url-9 | Bin\n 18 files changed, 295 insertions(+), 241 deletions(-)\n delete mode 100644 t/helper/test-urlmatch-normalization.c\n delete mode 100755 t/t0110-urlmatch-normalization.sh\n create mode 100644 t/unit-tests/t-urlmatch-normalization.c\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/README (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-1 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-10 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-11 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-2 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-3 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-4 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-5 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-6 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-7 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-8 (100%)\n rename t/{t0110 => unit-tests/t-urlmatch-normalization}/url-9 (100%)\n\ndiff --git a/Makefile b/Makefile\nindex 3863e60b66..d7bc19e823 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -843,7 +843,6 @@ TEST_BUILTINS_OBJS += test-submodule.o\n TEST_BUILTINS_OBJS += test-subprocess.o\n TEST_BUILTINS_OBJS += test-trace2.o\n TEST_BUILTINS_OBJS += test-truncate.o\n-TEST_BUILTINS_OBJS += test-urlmatch-normalization.o\n TEST_BUILTINS_OBJS += test-userdiff.o\n TEST_BUILTINS_OBJS += test-wildmatch.o\n TEST_BUILTINS_OBJS += test-windows-named-pipe.o\n@@ -1346,6 +1345,7 @@ UNIT_TEST_PROGRAMS += t-strbuf\n UNIT_TEST_PROGRAMS += t-strcmp-offset\n UNIT_TEST_PROGRAMS += t-strvec\n UNIT_TEST_PROGRAMS += t-trailer\n+UNIT_TEST_PROGRAMS += t-urlmatch-normalization\n UNIT_TEST_PROGS = $(patsubst %,$(UNIT_TEST_BIN)/%$X,$(UNIT_TEST_PROGRAMS))\n UNIT_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))\n UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex da3e69128a..f8a67df7de 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -83,7 +83,6 @@ static struct test_cmd cmds[] = {\n \t{ \"trace2\", cmd__trace2 },\n \t{ \"truncate\", cmd__truncate },\n \t{ \"userdiff\", cmd__userdiff },\n-\t{ \"urlmatch-normalization\", cmd__urlmatch_normalization },\n \t{ \"xml-encode\", cmd__xml_encode },\n \t{ \"wildmatch\", cmd__wildmatch },\n #ifdef GIT_WINDOWS_NATIVE\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex 642a34578c..e74bc0ffd4 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -76,7 +76,6 @@ int cmd__subprocess(int argc, const char **argv);\n int cmd__trace2(int argc, const char **argv);\n int cmd__truncate(int argc, const char **argv);\n int cmd__userdiff(int argc, const char **argv);\n-int cmd__urlmatch_normalization(int argc, const char **argv);\n int cmd__xml_encode(int argc, const char **argv);\n int cmd__wildmatch(int argc, const char **argv);\n #ifdef GIT_WINDOWS_NATIVE\ndiff --git a/t/helper/test-urlmatch-normalization.c b/t/helper/test-urlmatch-normalization.c\ndeleted file mode 100644\nindex 86edd454f5..0000000000\n--- a/t/helper/test-urlmatch-normalization.c\n+++ /dev/null\n@@ -1,56 +0,0 @@\n-#include \"test-tool.h\"\n-#include \"git-compat-util.h\"\n-#include \"urlmatch.h\"\n-\n-int cmd__urlmatch_normalization(int argc, const char **argv)\n-{\n-\tconst char usage[] = \"test-tool urlmatch-normalization [-p | -l] | \";\n-\tchar *url1 = NULL, *url2 = NULL;\n-\tint opt_p = 0, opt_l = 0;\n-\tint ret = 0;\n-\n-\t/*\n-\t * For one url, succeed if url_normalize succeeds on it, fail otherwise.\n-\t * For two urls, succeed only if url_normalize succeeds on both and\n-\t * the results compare equal with strcmp. If -p is given (one url only)\n-\t * and url_normalize succeeds, print the result followed by \"\\n\". If\n-\t * -l is given (one url only) and url_normalize succeeds, print the\n-\t * returned length in decimal followed by \"\\n\".\n-\t */\n-\n-\tif (argc > 1 && !strcmp(argv[1], \"-p\")) {\n-\t\topt_p = 1;\n-\t\targc--;\n-\t\targv++;\n-\t} else if (argc > 1 && !strcmp(argv[1], \"-l\")) {\n-\t\topt_l = 1;\n-\t\targc--;\n-\t\targv++;\n-\t}\n-\n-\tif (argc < 2 || argc > 3)\n-\t\tdie(\"%s\", usage);\n-\n-\tif (argc == 2) {\n-\t\tstruct url_info info;\n-\t\turl1 = url_normalize(argv[1], &info);\n-\t\tif (!url1)\n-\t\t\treturn 1;\n-\t\tif (opt_p)\n-\t\t\tprintf(\"%s\\n\", url1);\n-\t\tif (opt_l)\n-\t\t\tprintf(\"%u\\n\", (unsigned)info.url_len);\n-\t\tgoto cleanup;\n-\t}\n-\n-\tif (opt_p || opt_l)\n-\t\tdie(\"%s\", usage);\n-\n-\turl1 = url_normalize(argv[1], NULL);\n-\turl2 = url_normalize(argv[2], NULL);\n-\tret = (url1 && url2 && !strcmp(url1, url2)) ? 0 : 1;\n-cleanup:\n-\tfree(url1);\n-\tfree(url2);\n-\treturn ret;\n-}\ndiff --git a/t/t0110-urlmatch-normalization.sh b/t/t0110-urlmatch-normalization.sh\ndeleted file mode 100755\nindex 12d817fbd3..0000000000\n--- a/t/t0110-urlmatch-normalization.sh\n+++ /dev/null\n@@ -1,182 +0,0 @@\n-#!/bin/sh\n-\n-test_description='urlmatch URL normalization'\n-\n-TEST_PASSES_SANITIZE_LEAK=true\n-. ./test-lib.sh\n-\n-# The base name of the test url files\n-tu=\"$TEST_DIRECTORY/t0110/url\"\n-\n-# Note that only file: URLs should be allowed without a host\n-\n-test_expect_success 'url scheme' '\n-\t! test-tool urlmatch-normalization \"\" &&\n-\t! test-tool urlmatch-normalization \"_\" &&\n-\t! test-tool urlmatch-normalization \"scheme\" &&\n-\t! test-tool urlmatch-normalization \"scheme:\" &&\n-\t! test-tool urlmatch-normalization \"scheme:/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://\" &&\n-\t! test-tool urlmatch-normalization \"file\" &&\n-\t! test-tool urlmatch-normalization \"file:\" &&\n-\t! test-tool urlmatch-normalization \"file:/\" &&\n-\ttest-tool urlmatch-normalization \"file://\" &&\n-\t! test-tool urlmatch-normalization \"://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"x_test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"-test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"0test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"+test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \".test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"schem%6e://\" &&\n-\ttest-tool urlmatch-normalization \"x-Test+v1.0://acme.co\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"AbCdeF://x.Y\")\" = \"abcdef://x.y/\"\n-'\n-\n-test_expect_success 'url authority' '\n-\t! test-tool urlmatch-normalization \"scheme://user:pass@\" &&\n-\t! test-tool urlmatch-normalization \"scheme://?\" &&\n-\t! test-tool urlmatch-normalization \"scheme://#\" &&\n-\t! test-tool urlmatch-normalization \"scheme:///\" &&\n-\t! test-tool urlmatch-normalization \"scheme://:\" &&\n-\t! test-tool urlmatch-normalization \"scheme://:555\" &&\n-\ttest-tool urlmatch-normalization \"file://user:pass@\" &&\n-\ttest-tool urlmatch-normalization \"file://?\" &&\n-\ttest-tool urlmatch-normalization \"file://#\" &&\n-\ttest-tool urlmatch-normalization \"file:///\" &&\n-\ttest-tool urlmatch-normalization \"file://:\" &&\n-\t! test-tool urlmatch-normalization \"file://:555\" &&\n-\ttest-tool urlmatch-normalization \"scheme://user:pass@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://%00@host\" &&\n-\t! test-tool urlmatch-normalization \"scheme://%%@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host_\" &&\n-\ttest-tool urlmatch-normalization \"scheme://user:pass@host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://@host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host?x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host#x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host/@\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host?@x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host#@x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[::1]\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[::1]/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://hos%41/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[invalid....:/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://invalid....:]/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://invalid....:[/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://invalid....:[\"\n-'\n-\n-test_expect_success 'url port checks' '\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:0\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:0000000\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:0000001?\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:065535#\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:65535\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:65536\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:99999\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:100000\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:100001\" &&\n-\ttest-tool urlmatch-normalization \"http://q@some.host:80\" &&\n-\ttest-tool urlmatch-normalization \"https://q@some.host:443\" &&\n-\ttest-tool urlmatch-normalization \"http://q@some.host:80/\" &&\n-\ttest-tool urlmatch-normalization \"https://q@some.host:443?\" &&\n-\t! test-tool urlmatch-normalization \"http://q@:8008\" &&\n-\t! test-tool urlmatch-normalization \"http://:8080\" &&\n-\t! test-tool urlmatch-normalization \"http://:\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n-\ttest-tool urlmatch-normalization \"xyz://[::1]:456/\" &&\n-\ttest-tool urlmatch-normalization \"xyz://[::1]:/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:000/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:0%300/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:0x80/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:4294967297/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:030f/\"\n-'\n-\n-test_expect_success 'url port normalization' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:0800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:00000800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:065535\")\" = \"http://x:65535/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:1\")\" = \"http://x:1/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:80\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:080\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:000000080\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:443\")\" = \"https://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:0443\")\" = \"https://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:000000443\")\" = \"https://x/\"\n-'\n-\n-test_expect_success 'url general escapes' '\n-\t! test-tool urlmatch-normalization \"http://x.y?%fg\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/%7e%41^%3a\")\" = \"x://w/~A%5E%3A\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/:/?#[]@\")\" = \"x://w/:/?#[]@\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/$&()*+,;=\")\" = \"x://w/$&()*+,;=\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/'\\''\")\" = \"x://w/'\\''\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W?'\\!'\")\" = \"x://w/?'\\!'\"\n-'\n-\n-test_expect_success !MINGW 'url high-bit escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-1\")\")\" = \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-2\")\")\" = \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-3\")\")\" = \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-4\")\")\" = \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-5\")\")\" = \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-6\")\")\" = \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-7\")\")\" = \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-8\")\")\" = \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-9\")\")\" = \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-10\")\")\" = \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\"\n-'\n-\n-test_expect_success 'url utf-8 escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-11\")\")\" = \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\"\n-'\n-\n-test_expect_success 'url username/password escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://%41%62(^):%70+d@foo\")\" = \"x://Ab(%5E):p+d@foo/\"\n-'\n-\n-test_expect_success 'url normalized lengths' '\n-\ttest \"$(test-tool urlmatch-normalization -l \"Http://%4d%65:%4d^%70@The.Host\")\" = 25 &&\n-\ttest \"$(test-tool urlmatch-normalization -l \"http://%41:%42@x.y/%61/\")\" = 17 &&\n-\ttest \"$(test-tool urlmatch-normalization -l \"http://@x.y/^\")\" = 15\n-'\n-\n-test_expect_success 'url . and .. segments' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/.\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/./\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/.\")\" = \"x://y/a\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./\")\" = \"x://y/a/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/.?\")\" = \"x://y/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/./?\")\" = \"x://y/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/.?\")\" = \"x://y/a?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./?\")\" = \"x://y/a/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/.././../c\")\" = \"x://y/c\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/../.././c/\")\" = \"x://y/c/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/.././../c/././.././.\")\" = \"x://y/\" &&\n-\t! test-tool urlmatch-normalization \"x://y/a/./b/.././../c/././.././..\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./?/././..\")\" = \"x://y/a/?/././..\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/%2e/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/%2E/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/%2e./\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/b/.%2E/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/c/%2e%2E/\")\" = \"x://y/\"\n-'\n-\n-# http://@foo specifies an empty user name but does not specify a password\n-# http://foo specifies neither a user name nor a password\n-# So they should not be equivalent\n-test_expect_success 'url equivalents' '\n-\ttest-tool urlmatch-normalization \"httP://x\" \"Http://X/\" &&\n-\ttest-tool urlmatch-normalization \"Http://%4d%65:%4d^%70@The.Host\" \"hTTP://Me:%4D^p@the.HOST:80/\" &&\n-\t! test-tool urlmatch-normalization \"https://@x.y/^\" \"httpS://x.y:443/^\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^\" \"httpS://@x.y:0443/^\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^/../abc\" \"httpS://@x.y:0443/abc\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^/..\" \"httpS://@x.y:0443/\"\n-'\n-\n-test_done\ndiff --git a/t/unit-tests/t-urlmatch-normalization.c b/t/unit-tests/t-urlmatch-normalization.c\nnew file mode 100644\nindex 0000000000..e0dd50dc11\n--- /dev/null\n+++ b/t/unit-tests/t-urlmatch-normalization.c\n@@ -0,0 +1,294 @@\n+#include \"test-lib.h\"\n+#include \"urlmatch.h\"\n+#include \"strbuf.h\"\n+\n+static void check_url_normalizable(const char *url, int normalizable)\n+{\n+\tchar *url_norm = url_normalize(url, NULL);\n+\n+\tif (!check_int(normalizable, ==, url_norm ? 1 : 0))\n+\t\ttest_msg(\"input url: %s\", url);\n+\tfree(url_norm);\n+}\n+\n+static void check_normalized_url(const char *url, const char *expect)\n+{\n+\tchar *url_norm = url_normalize(url, NULL);\n+\n+\tif (!check_str(url_norm, expect))\n+\t\ttest_msg(\"input url: %s\", url);\n+\tfree(url_norm);\n+}\n+\n+static void compare_normalized_urls(const char *url1, const char *url2,\n+\t\t\t\t size_t equal)\n+{\n+\tchar *url1_norm = url_normalize(url1, NULL);\n+\tchar *url2_norm = url_normalize(url2, NULL);\n+\n+\tif (equal) {\n+\t\tif (!check_str(url1_norm, url2_norm))\n+\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n+\t\t\t\t url2);\n+\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0)) {\n+\t\ttest_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n+\t\t\t \" input url1: %s\\n input url2: %s\",\n+\t\t\t url1_norm, url2_norm, url1, url2);\n+\t}\n+\tfree(url1_norm);\n+\tfree(url2_norm);\n+}\n+\n+static void check_normalized_url_from_file(const char *file, const char *expect)\n+{\n+\tstruct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n+\tchar *cwd_basename;\n+\n+\tif (!check_int(strbuf_getcwd(&path), ==, 0))\n+\t\treturn;\n+\n+\tcwd_basename = basename(path.buf);\n+\tif (!check(!strcmp(cwd_basename, \"t\") || !strcmp(cwd_basename, \"bin\"))) {\n+\t\ttest_msg(\"BUG: unit-tests should be run from either 't/' or 't/unit-tests/bin' directory\");\n+\t\treturn;\n+\t}\n+\n+\tstrbuf_strip_suffix(&path, \"/unit-tests/bin\");\n+\tstrbuf_addf(&path, \"/unit-tests/t-urlmatch-normalization/%s\", file);\n+\n+\tif (!check_int(strbuf_read_file(&content, path.buf, 0), >, 0)) {\n+\t\ttest_msg(\"failed to read from file '%s': %s\", file, strerror(errno));\n+\t} else {\n+\t\tchar *url_norm;\n+\n+\t\tstrbuf_trim_trailing_newline(&content);\n+\t\turl_norm = url_normalize(content.buf, NULL);\n+\t\tif (!check_str(url_norm, expect))\n+\t\t\ttest_msg(\"input file: %s\", file);\n+\t\tfree(url_norm);\n+\t}\n+\n+\tstrbuf_release(&content);\n+\tstrbuf_release(&path);\n+}\n+\n+static void check_normalized_url_length(const char *url, size_t len)\n+{\n+\tstruct url_info info;\n+\tchar *url_norm = url_normalize(url, &info);\n+\n+\tif (!check_int(info.url_len, ==, len))\n+\t\ttest_msg(\" input url: %s\\n normalized url: %s\", url,\n+\t\t\t url_norm);\n+\tfree(url_norm);\n+}\n+\n+/* Note that only file: URLs should be allowed without a host */\n+static void t_url_scheme(void)\n+{\n+\tcheck_url_normalizable(\"\", 0);\n+\tcheck_url_normalizable(\"_\", 0);\n+\tcheck_url_normalizable(\"scheme\", 0);\n+\tcheck_url_normalizable(\"scheme:\", 0);\n+\tcheck_url_normalizable(\"scheme:/\", 0);\n+\tcheck_url_normalizable(\"scheme://\", 0);\n+\tcheck_url_normalizable(\"file\", 0);\n+\tcheck_url_normalizable(\"file:\", 0);\n+\tcheck_url_normalizable(\"file:/\", 0);\n+\tcheck_url_normalizable(\"file://\", 1);\n+\tcheck_url_normalizable(\"://acme.co\", 0);\n+\tcheck_url_normalizable(\"x_test://acme.co\", 0);\n+\tcheck_url_normalizable(\"-test://acme.co\", 0);\n+\tcheck_url_normalizable(\"0test://acme.co\", 0);\n+\tcheck_url_normalizable(\"+test://acme.co\", 0);\n+\tcheck_url_normalizable(\".test://acme.co\", 0);\n+\tcheck_url_normalizable(\"schem%6e://\", 0);\n+\tcheck_url_normalizable(\"x-Test+v1.0://acme.co\", 1);\n+\tcheck_normalized_url(\"AbCdeF://x.Y\", \"abcdef://x.y/\");\n+}\n+\n+static void t_url_authority(void)\n+{\n+\tcheck_url_normalizable(\"scheme://user:pass@\", 0);\n+\tcheck_url_normalizable(\"scheme://?\", 0);\n+\tcheck_url_normalizable(\"scheme://#\", 0);\n+\tcheck_url_normalizable(\"scheme:///\", 0);\n+\tcheck_url_normalizable(\"scheme://:\", 0);\n+\tcheck_url_normalizable(\"scheme://:555\", 0);\n+\tcheck_url_normalizable(\"file://user:pass@\", 1);\n+\tcheck_url_normalizable(\"file://?\", 1);\n+\tcheck_url_normalizable(\"file://#\", 1);\n+\tcheck_url_normalizable(\"file:///\", 1);\n+\tcheck_url_normalizable(\"file://:\", 1);\n+\tcheck_url_normalizable(\"file://:555\", 0);\n+\tcheck_url_normalizable(\"scheme://user:pass@host\", 1);\n+\tcheck_url_normalizable(\"scheme://@host\", 1);\n+\tcheck_url_normalizable(\"scheme://%00@host\", 1);\n+\tcheck_url_normalizable(\"scheme://%%@host\", 0);\n+\tcheck_url_normalizable(\"scheme://host_\", 1);\n+\tcheck_url_normalizable(\"scheme://user:pass@host/\", 1);\n+\tcheck_url_normalizable(\"scheme://@host/\", 1);\n+\tcheck_url_normalizable(\"scheme://host/\", 1);\n+\tcheck_url_normalizable(\"scheme://host?x\", 1);\n+\tcheck_url_normalizable(\"scheme://host#x\", 1);\n+\tcheck_url_normalizable(\"scheme://host/@\", 1);\n+\tcheck_url_normalizable(\"scheme://host?@x\", 1);\n+\tcheck_url_normalizable(\"scheme://host#@x\", 1);\n+\tcheck_url_normalizable(\"scheme://[::1]\", 1);\n+\tcheck_url_normalizable(\"scheme://[::1]/\", 1);\n+\tcheck_url_normalizable(\"scheme://hos%41/\", 0);\n+\tcheck_url_normalizable(\"scheme://[invalid....:/\", 1);\n+\tcheck_url_normalizable(\"scheme://invalid....:]/\", 1);\n+\tcheck_url_normalizable(\"scheme://invalid....:[/\", 0);\n+\tcheck_url_normalizable(\"scheme://invalid....:[\", 0);\n+}\n+\n+static void t_url_port(void)\n+{\n+\tcheck_url_normalizable(\"xyz://q@some.host:\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0000000\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0000001?\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:065535#\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:65535\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:65536\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:99999\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:100000\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:100001\", 0);\n+\tcheck_url_normalizable(\"http://q@some.host:80\", 1);\n+\tcheck_url_normalizable(\"https://q@some.host:443\", 1);\n+\tcheck_url_normalizable(\"http://q@some.host:80/\", 1);\n+\tcheck_url_normalizable(\"https://q@some.host:443?\", 1);\n+\tcheck_url_normalizable(\"http://q@:8008\", 0);\n+\tcheck_url_normalizable(\"http://:8080\", 0);\n+\tcheck_url_normalizable(\"http://:\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:000/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:0%300/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:0x80/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:4294967297/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:030f/\", 0);\n+}\n+\n+static void t_url_port_normalization(void)\n+{\n+\tcheck_normalized_url(\"http://x:800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:0800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:00000800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:065535\", \"http://x:65535/\");\n+\tcheck_normalized_url(\"http://x:1\", \"http://x:1/\");\n+\tcheck_normalized_url(\"http://x:80\", \"http://x/\");\n+\tcheck_normalized_url(\"http://x:080\", \"http://x/\");\n+\tcheck_normalized_url(\"http://x:000000080\", \"http://x/\");\n+\tcheck_normalized_url(\"https://x:443\", \"https://x/\");\n+\tcheck_normalized_url(\"https://x:0443\", \"https://x/\");\n+\tcheck_normalized_url(\"https://x:000000443\", \"https://x/\");\n+}\n+\n+static void t_url_general_escape(void)\n+{\n+\tcheck_url_normalizable(\"http://x.y?%fg\", 0);\n+\tcheck_normalized_url(\"X://W/%7e%41^%3a\", \"x://w/~A%5E%3A\");\n+\tcheck_normalized_url(\"X://W/:/?#[]@\", \"x://w/:/?#[]@\");\n+\tcheck_normalized_url(\"X://W/$&()*+,;=\", \"x://w/$&()*+,;=\");\n+\tcheck_normalized_url(\"X://W/'\", \"x://w/'\");\n+\tcheck_normalized_url(\"X://W?!\", \"x://w/?!\");\n+}\n+\n+static void t_url_high_bit(void)\n+{\n+\tcheck_normalized_url_from_file(\"url-1\",\n+\t\t\t \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\");\n+\tcheck_normalized_url_from_file(\"url-2\",\n+\t\t\t \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\");\n+\tcheck_normalized_url_from_file(\"url-3\",\n+\t\t\t \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\");\n+\tcheck_normalized_url_from_file(\"url-4\",\n+\t\t\t \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\");\n+\tcheck_normalized_url_from_file(\"url-5\",\n+\t\t\t \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\");\n+\tcheck_normalized_url_from_file(\"url-6\",\n+\t\t\t \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\");\n+\tcheck_normalized_url_from_file(\"url-7\",\n+\t\t\t \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\");\n+\tcheck_normalized_url_from_file(\"url-8\",\n+\t\t\t \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\");\n+\tcheck_normalized_url_from_file(\"url-9\",\n+\t\t\t \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\");\n+\tcheck_normalized_url_from_file(\"url-10\",\n+\t\t\t \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\");\n+}\n+\n+static void t_url_utf8_escape(void)\n+{\n+\tcheck_normalized_url_from_file(\"url-11\",\n+\t\t\t \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\");\n+}\n+\n+static void t_url_username_pass(void)\n+{\n+\tcheck_normalized_url(\"x://%41%62(^):%70+d@foo\", \"x://Ab(%5E):p+d@foo/\");\n+}\n+\n+static void t_url_length(void)\n+{\n+\tcheck_normalized_url_length(\"Http://%4d%65:%4d^%70@The.Host\", 25);\n+\tcheck_normalized_url_length(\"http://%41:%42@x.y/%61/\", 17);\n+\tcheck_normalized_url_length(\"http://@x.y/^\", 15);\n+}\n+\n+static void t_url_dots(void)\n+{\n+\tcheck_normalized_url(\"x://y/.\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/./\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/a/.\", \"x://y/a\");\n+\tcheck_normalized_url(\"x://y/a/./\", \"x://y/a/\");\n+\tcheck_normalized_url(\"x://y/.?\", \"x://y/?\");\n+\tcheck_normalized_url(\"x://y/./?\", \"x://y/?\");\n+\tcheck_normalized_url(\"x://y/a/.?\", \"x://y/a?\");\n+\tcheck_normalized_url(\"x://y/a/./?\", \"x://y/a/?\");\n+\tcheck_normalized_url(\"x://y/a/./b/.././../c\", \"x://y/c\");\n+\tcheck_normalized_url(\"x://y/a/./b/../.././c/\", \"x://y/c/\");\n+\tcheck_normalized_url(\"x://y/a/./b/.././../c/././.././.\", \"x://y/\");\n+\tcheck_url_normalizable(\"x://y/a/./b/.././../c/././.././..\", 0);\n+\tcheck_normalized_url(\"x://y/a/./?/././..\", \"x://y/a/?/././..\");\n+\tcheck_normalized_url(\"x://y/%2e/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/%2E/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/a/%2e./\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/b/.%2E/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/c/%2e%2E/\", \"x://y/\");\n+}\n+\n+/*\n+ * http://@foo specifies an empty user name but does not specify a password\n+ * http://foo specifies neither a user name nor a password\n+ * So they should not be equivalent\n+ */\n+static void t_url_equivalents(void)\n+{\n+\tcompare_normalized_urls(\"httP://x\", \"Http://X/\", 1);\n+\tcompare_normalized_urls(\"Http://%4d%65:%4d^%70@The.Host\", \"hTTP://Me:%4D^p@the.HOST:80/\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^\", \"httpS://x.y:443/^\", 0);\n+\tcompare_normalized_urls(\"https://@x.y/^\", \"httpS://@x.y:0443/^\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^/../abc\", \"httpS://@x.y:0443/abc\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^/..\", \"httpS://@x.y:0443/\", 1);\n+}\n+\n+int cmd_main(int argc UNUSED, const char **argv UNUSED)\n+{\n+\tTEST(t_url_scheme(), \"url scheme\");\n+\tTEST(t_url_authority(), \"url authority\");\n+\tTEST(t_url_port(), \"url port checks\");\n+\tTEST(t_url_port_normalization(), \"url port normalization\");\n+\tTEST(t_url_general_escape(), \"url general escapes\");\n+\tTEST(t_url_high_bit(), \"url high-bit escapes\");\n+\tTEST(t_url_utf8_escape(), \"url utf8 escapes\");\n+\tTEST(t_url_username_pass(), \"url username/password escapes\");\n+\tTEST(t_url_length(), \"url normalized lengths\");\n+\tTEST(t_url_dots(), \"url . and .. segments\");\n+\tTEST(t_url_equivalents(), \"url equivalents\");\n+\treturn test_done();\n+}\ndiff --git a/t/t0110/README b/t/unit-tests/t-urlmatch-normalization/README\nsimilarity index 100%\nrename from t/t0110/README\nrename to t/unit-tests/t-urlmatch-normalization/README\ndiff --git a/t/t0110/url-1 b/t/unit-tests/t-urlmatch-normalization/url-1\nsimilarity index 100%\nrename from t/t0110/url-1\nrename to t/unit-tests/t-urlmatch-normalization/url-1\ndiff --git a/t/t0110/url-10 b/t/unit-tests/t-urlmatch-normalization/url-10\nsimilarity index 100%\nrename from t/t0110/url-10\nrename to t/unit-tests/t-urlmatch-normalization/url-10\ndiff --git a/t/t0110/url-11 b/t/unit-tests/t-urlmatch-normalization/url-11\nsimilarity index 100%\nrename from t/t0110/url-11\nrename to t/unit-tests/t-urlmatch-normalization/url-11\ndiff --git a/t/t0110/url-2 b/t/unit-tests/t-urlmatch-normalization/url-2\nsimilarity index 100%\nrename from t/t0110/url-2\nrename to t/unit-tests/t-urlmatch-normalization/url-2\ndiff --git a/t/t0110/url-3 b/t/unit-tests/t-urlmatch-normalization/url-3\nsimilarity index 100%\nrename from t/t0110/url-3\nrename to t/unit-tests/t-urlmatch-normalization/url-3\ndiff --git a/t/t0110/url-4 b/t/unit-tests/t-urlmatch-normalization/url-4\nsimilarity index 100%\nrename from t/t0110/url-4\nrename to t/unit-tests/t-urlmatch-normalization/url-4\ndiff --git a/t/t0110/url-5 b/t/unit-tests/t-urlmatch-normalization/url-5\nsimilarity index 100%\nrename from t/t0110/url-5\nrename to t/unit-tests/t-urlmatch-normalization/url-5\ndiff --git a/t/t0110/url-6 b/t/unit-tests/t-urlmatch-normalization/url-6\nsimilarity index 100%\nrename from t/t0110/url-6\nrename to t/unit-tests/t-urlmatch-normalization/url-6\ndiff --git a/t/t0110/url-7 b/t/unit-tests/t-urlmatch-normalization/url-7\nsimilarity index 100%\nrename from t/t0110/url-7\nrename to t/unit-tests/t-urlmatch-normalization/url-7\ndiff --git a/t/t0110/url-8 b/t/unit-tests/t-urlmatch-normalization/url-8\nsimilarity index 100%\nrename from t/t0110/url-8\nrename to t/unit-tests/t-urlmatch-normalization/url-8\ndiff --git a/t/t0110/url-9 b/t/unit-tests/t-urlmatch-normalization/url-9\nsimilarity index 100%\nrename from t/t0110/url-9\nrename to t/unit-tests/t-urlmatch-normalization/url-9\n\nRange-diff against v1:\n1: 6a44d676cd ! 1: 3f4e4be1a6 t: migrate helper/test-urlmatch-normalization to unit tests\n @@ Commit message\n string, it is not possible to pass arbitrary bytes to a program. But\n in unit tests we don't have this limitation.\n \n + With the addition of this unit test, we impose a new restriction of\n + running the unit tests from either 't/' or 't/unit-tests/bin/'\n + directories. This is to construct the path to files which contain some\n + input urls under the 't/t-urlmatch-normalization' directory. This\n + restriction is similar to one we have for end-to-end tests, where they\n + can be ran from only 't/'. Addition of 't/unit-tests/bin/' is to allow\n + for running individual tests which is not currently possible via any\n + 'make' targets and also 'unit-tests-test-tool' target is also ran from\n + the 't/unit-tests/bin' directory.\n +\n [1]: https://lore.kernel.org/git/53CAC8EF.6020707@gmail.com/\n \n Mentored-by: Christian Couder \n @@ t/unit-tests/t-urlmatch-normalization.c (new)\n +\t\tif (!check_str(url1_norm, url2_norm))\n +\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n +\t\t\t\t url2);\n -+\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0))\n ++\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0)) {\n +\t\ttest_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n +\t\t\t \" input url1: %s\\n input url2: %s\",\n +\t\t\t url1_norm, url2_norm, url1, url2);\n ++\t}\n +\tfree(url1_norm);\n +\tfree(url2_norm);\n +}\n @@ t/unit-tests/t-urlmatch-normalization.c (new)\n +static void check_normalized_url_from_file(const char *file, const char *expect)\n +{\n +\tstruct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n ++\tchar *cwd_basename;\n ++\n ++\tif (!check_int(strbuf_getcwd(&path), ==, 0))\n ++\t\treturn;\n ++\n ++\tcwd_basename = basename(path.buf);\n ++\tif (!check(!strcmp(cwd_basename, \"t\") || !strcmp(cwd_basename, \"bin\"))) {\n ++\t\ttest_msg(\"BUG: unit-tests should be run from either 't/' or 't/unit-tests/bin' directory\");\n ++\t\treturn;\n ++\t}\n +\n -+\tstrbuf_getcwd(&path);\n -+\tstrbuf_strip_suffix(&path, \"/unit-tests/bin\"); /* because 'unit-tests-test-tool' is run from 'bin' directory */\n ++\tstrbuf_strip_suffix(&path, \"/unit-tests/bin\");\n +\tstrbuf_addf(&path, \"/unit-tests/t-urlmatch-normalization/%s\", file);\n +\n +\tif (!check_int(strbuf_read_file(&content, path.buf, 0), >, 0)) {\n-- \n2.46.0\n\n"},{"id":"500816","messageId":"xmqqh6bo448j.fsf@gitster.g","threadId":"61693","inReplyTo":"20240813172432.55487-1-shyamthakkar001@gmail.com","subject":"Re: [GSoC][PATCH v2] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-13T19:22:52Z","receivedAt":"2024-08-13T19:22:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ghanshyam Thakkar writes:\n\n> With the addition of this unit test, we impose a new restriction of\n> running the unit tests from either 't/' or 't/unit-tests/bin/'\n> directories. This is to construct the path to files which contain some\n> input urls under the 't/t-urlmatch-normalization' directory. This\n> restriction is similar to one we have for end-to-end tests, where they\n> can be ran from only 't/'.\n>\n> Addition of 't/unit-tests/bin/' is to allow\n> for running individual tests which is not currently possible via any\n> 'make' targets and also 'unit-tests-test-tool' target is also ran from\n> the 't/unit-tests/bin' directory.\n\nSorry, but I do not quite follow. The above makes it sound as if\nthe 'bin' subdirectory is something that never existed before this\npatch and this patch introduces the use of that directory, but that\nis hardly the case. What does that \"Addition of\" really refer to?\n\nDo you mean \"we cannot run the tests from arbitrary places, and we\nallow them to be run from t/, just like the normal tests\" followed\nby \"in addition, we also allow them to be run from t/unit-tests/bin\ndirectory because ...\"?\n\nI wonder if we should get of t/t-urlmatch-normalization/ directory\nand instead hold these test data in the form of string constants in\nthe program. After all, you have the expected normalization result\nhardcoded in the binary (e.g. t_url_high_bit() asks the checker\nfunction to read from \"url-1\" file and then compare the result of\nnormalization with a hardcoded string constant), so having the test\ndata in separate files only risks the input and the output easily\ndrift apart.\n\nAs a side effect, it would make it easily possible to run the tests\nanywhere, because you no longer depend on these url-$n input files.\nIt of course depends on how burdensome the limitation that we can\nrun the tests only from a fixed place really is, but it generally is\nnot a good idea to have these random sequence of bytes in small\nfiles that nobody looks at in a repository in the first place.\n\nThanks.\n"},{"id":"500833","messageId":"b369797c-37c0-4f17-b857-a2ff8680cb4f@gmail.com","threadId":"61693","inReplyTo":"xmqqh6bo448j.fsf@gitster.g","subject":"Re: [GSoC][PATCH v2] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Kaartic Sivaraam","fromEmail":"kaartic.sivaraam@gmail.com","sentAt":"2024-08-14T01:35:47Z","receivedAt":"2024-08-14T01:35:54Z","isPatch":true,"sender":{"key":"kaartic.sivaraam@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12448084?v=4"},"body":"On 14/08/24 00:52, Junio C Hamano wrote:\n> Ghanshyam Thakkar writes:\n>\n> I wonder if we should get of t/t-urlmatch-normalization/ directory\n> and instead hold these test data in the form of string constants in\n> the program. After all, you have the expected normalization result\n> hardcoded in the binary (e.g. t_url_high_bit() asks the checker\n> function to read from \"url-1\" file and then compare the result of\n> normalization with a hardcoded string constant), so having the test\n> data in separate files only risks the input and the output easily\n> drift apart.\n> \n> As a side effect, it would make it easily possible to run the tests\n> anywhere, because you no longer depend on these url-$n input files.\n> It of course depends on how burdensome the limitation that we can\n> run the tests only from a fixed place really is, but it generally is\n> not a good idea to have these random sequence of bytes in small\n> files that nobody looks at in a repository in the first place.\n> \n\nI think the reason these inputs are present in the files is solely\nbecause they are random sequence of characters which contain unicode\nand even some control characters. This makes it tricky to hold the\ninput string in the source itself.\n\nI'm not sure there would be a straight-forward way to have these inputs\nin the C source file. There may be some way to represent them in an \nalternate form but I suppose that would sacrifice the readability of \nthese inputs which I believe is also a significant factor for test cases.\n\nFeel free to enlighten us if we're possibly missing some straight \nforward way of having these input URLs in the source files.\n\n--\nSivaraam\n"},{"id":"500840","messageId":"xmqqo75v3dkf.fsf@gitster.g","threadId":"61693","inReplyTo":"b369797c-37c0-4f17-b857-a2ff8680cb4f@gmail.com","subject":"Re: [GSoC][PATCH v2] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-14T04:58:56Z","receivedAt":"2024-08-14T04:59:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kaartic Sivaraam writes:\n\n> I'm not sure there would be a straight-forward way to have these inputs\n> in the C source file. There may be some way to represent them in an\n> alternate form but I suppose that would sacrifice the readability of\n> these inputs which I believe is also a significant factor for test\n> cases.\n\nJust as a byte string, all bytes \\012 octally expressed?\n"},{"id":"500842","messageId":"4c14e3bf-0f0d-4600-8e17-5b5d15ad7447@gmail.com","threadId":"61693","inReplyTo":"20240813172432.55487-1-shyamthakkar001@gmail.com","subject":"Re: [GSoC][PATCH v2] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Kaartic Sivaraam","fromEmail":"kaartic.sivaraam@gmail.com","sentAt":"2024-08-14T05:17:50Z","receivedAt":"2024-08-14T05:17:56Z","isPatch":true,"sender":{"key":"kaartic.sivaraam@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12448084?v=4"},"body":"Hi Ghansyam,\n\nI just wanted to share two comments based on what I observed from the \nrecent changes.\n\nOn 13/08/24 22:54, Ghanshyam Thakkar wrote:\n >\n> index 0000000000..e0dd50dc11\n> --- /dev/null\n> +++ b/t/unit-tests/t-urlmatch-normalization.c\n> @@ -0,0 +1,294 @@\n> +\n> +static void check_normalized_url_from_file(const char *file, const char *expect)\n> +{\n> +\tstruct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n> +\tchar *cwd_basename;\n> +\n> +\tif (!check_int(strbuf_getcwd(&path), ==, 0))\n> +\t\treturn;\n> +\n> +\tcwd_basename = basename(path.buf);\n> +\tif (!check(!strcmp(cwd_basename, \"t\") || !strcmp(cwd_basename, \"bin\"))) {\n\nI think comparing blindly for \"bin\" would not be an ideal thing to do as \nthis would let other locations which have the \"bin\" basename to be slip \nthrough. For instance, running the test from \"perl/blib/bin\" would \nresult in this check letting it through. I suppose we could need a bit \nmore specific check that ensures that the test is indeed running from \n\"t/unit-tests/bin\".\n\n> ... snip ...\n >\n> +static void t_url_high_bit(void)\n> +{\n> +\tcheck_normalized_url_from_file(\"url-1\",\n> +\t\t\t \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\");\n> +\tcheck_normalized_url_from_file(\"url-2\",\n> +\t\t\t \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\");\n\nWhen we run the unit-test binary from other directories, the error \nmessage is thrown appropriately. But it seems to be printed for every \ntest case which seems a bit spammy. I suppose it might be helpful to do \nit once before actually running the test cases and skip the cases when \nwe realize the binary is being run from a different directory.\n\n--\nSivaraam\n"},{"id":"500911","messageId":"20240814142057.94671-1-shyamthakkar001@gmail.com","threadId":"61693","inReplyTo":"20240813172432.55487-1-shyamthakkar001@gmail.com","subject":"[GSoC][PATCH v3] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Ghanshyam Thakkar","fromEmail":"shyamthakkar001@gmail.com","sentAt":"2024-08-14T14:20:46Z","receivedAt":"2024-08-14T14:21:25Z","isPatch":true,"sender":{"key":"shyamthakkar001@gmail.com","avatar":"https://avatars.githubusercontent.com/u/72698233?v=4"},"body":"helper/test-urlmatch-normalization along with\nt0110-urlmatch-normalization test the `url_normalize()` function from\n'urlmatch.h'. Migrate them to the unit testing framework for better\nperformance. And also add different test_msg()s for better debugging.\n\nIn the migration, last two of the checks from `t_url_general_escape()`\nwere slightly changed compared to the shellscript. This involves changing\n\n'\\'' -> '\n'\\!' -> !\n\nin the urls of those checks. This is because in C strings, we don't\nneed to escape \"'\" and \"!\". Other than these two, all the urls were\npasted verbatim from the shellscript.\n\nAnother change is the removal of MINGW prerequisite from one of the\ntest. It was there because[1] on Windows, the command line is a\nUnicode string, it is not possible to pass arbitrary bytes to a\nprogram. But in unit tests we don't have this limitation.\n\nAnd since we can construct strings with arbitrary bytes in C, let's\nalso remove the test files which contain URLs with arbitrary bytes in\nthe 't/t0110' directory and instead embed those URLs in the unit test\ncode itself.\n\n[1]: https://lore.kernel.org/git/53CAC8EF.6020707@gmail.com/\n\nMentored-by: Christian Couder \nMentored-by: Kaartic Sivaraam \nSigned-off-by: Ghanshyam Thakkar \n---\nThis version addresses Junio's review and removes the restriction\nof running the unit tests in the 't/' and 't/unit-tests/bin'\nintroduced in v2 by embedding the URLs in the code itself.\n\n Makefile | 2 +-\n t/helper/test-tool.c | 1 -\n t/helper/test-tool.h | 1 -\n t/helper/test-urlmatch-normalization.c | 56 -----\n t/t0110-urlmatch-normalization.sh | 182 ----------------\n t/t0110/README | 9 -\n t/t0110/url-1 | Bin 20 -> 0 bytes\n t/t0110/url-10 | Bin 23 -> 0 bytes\n t/t0110/url-11 | Bin 25 -> 0 bytes\n t/t0110/url-2 | Bin 20 -> 0 bytes\n t/t0110/url-3 | Bin 23 -> 0 bytes\n t/t0110/url-4 | Bin 23 -> 0 bytes\n t/t0110/url-5 | Bin 23 -> 0 bytes\n t/t0110/url-6 | Bin 23 -> 0 bytes\n t/t0110/url-7 | Bin 23 -> 0 bytes\n t/t0110/url-8 | Bin 23 -> 0 bytes\n t/t0110/url-9 | Bin 23 -> 0 bytes\n t/unit-tests/t-urlmatch-normalization.c | 271 ++++++++++++++++++++++++\n 18 files changed, 272 insertions(+), 250 deletions(-)\n delete mode 100644 t/helper/test-urlmatch-normalization.c\n delete mode 100755 t/t0110-urlmatch-normalization.sh\n delete mode 100644 t/t0110/README\n delete mode 100644 t/t0110/url-1\n delete mode 100644 t/t0110/url-10\n delete mode 100644 t/t0110/url-11\n delete mode 100644 t/t0110/url-2\n delete mode 100644 t/t0110/url-3\n delete mode 100644 t/t0110/url-4\n delete mode 100644 t/t0110/url-5\n delete mode 100644 t/t0110/url-6\n delete mode 100644 t/t0110/url-7\n delete mode 100644 t/t0110/url-8\n delete mode 100644 t/t0110/url-9\n create mode 100644 t/unit-tests/t-urlmatch-normalization.c\n\ndiff --git a/Makefile b/Makefile\nindex 3863e60b66..d7bc19e823 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -843,7 +843,6 @@ TEST_BUILTINS_OBJS += test-submodule.o\n TEST_BUILTINS_OBJS += test-subprocess.o\n TEST_BUILTINS_OBJS += test-trace2.o\n TEST_BUILTINS_OBJS += test-truncate.o\n-TEST_BUILTINS_OBJS += test-urlmatch-normalization.o\n TEST_BUILTINS_OBJS += test-userdiff.o\n TEST_BUILTINS_OBJS += test-wildmatch.o\n TEST_BUILTINS_OBJS += test-windows-named-pipe.o\n@@ -1346,6 +1345,7 @@ UNIT_TEST_PROGRAMS += t-strbuf\n UNIT_TEST_PROGRAMS += t-strcmp-offset\n UNIT_TEST_PROGRAMS += t-strvec\n UNIT_TEST_PROGRAMS += t-trailer\n+UNIT_TEST_PROGRAMS += t-urlmatch-normalization\n UNIT_TEST_PROGS = $(patsubst %,$(UNIT_TEST_BIN)/%$X,$(UNIT_TEST_PROGRAMS))\n UNIT_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))\n UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex da3e69128a..f8a67df7de 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -83,7 +83,6 @@ static struct test_cmd cmds[] = {\n \t{ \"trace2\", cmd__trace2 },\n \t{ \"truncate\", cmd__truncate },\n \t{ \"userdiff\", cmd__userdiff },\n-\t{ \"urlmatch-normalization\", cmd__urlmatch_normalization },\n \t{ \"xml-encode\", cmd__xml_encode },\n \t{ \"wildmatch\", cmd__wildmatch },\n #ifdef GIT_WINDOWS_NATIVE\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex 642a34578c..e74bc0ffd4 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -76,7 +76,6 @@ int cmd__subprocess(int argc, const char **argv);\n int cmd__trace2(int argc, const char **argv);\n int cmd__truncate(int argc, const char **argv);\n int cmd__userdiff(int argc, const char **argv);\n-int cmd__urlmatch_normalization(int argc, const char **argv);\n int cmd__xml_encode(int argc, const char **argv);\n int cmd__wildmatch(int argc, const char **argv);\n #ifdef GIT_WINDOWS_NATIVE\ndiff --git a/t/helper/test-urlmatch-normalization.c b/t/helper/test-urlmatch-normalization.c\ndeleted file mode 100644\nindex 86edd454f5..0000000000\n--- a/t/helper/test-urlmatch-normalization.c\n+++ /dev/null\n@@ -1,56 +0,0 @@\n-#include \"test-tool.h\"\n-#include \"git-compat-util.h\"\n-#include \"urlmatch.h\"\n-\n-int cmd__urlmatch_normalization(int argc, const char **argv)\n-{\n-\tconst char usage[] = \"test-tool urlmatch-normalization [-p | -l] | \";\n-\tchar *url1 = NULL, *url2 = NULL;\n-\tint opt_p = 0, opt_l = 0;\n-\tint ret = 0;\n-\n-\t/*\n-\t * For one url, succeed if url_normalize succeeds on it, fail otherwise.\n-\t * For two urls, succeed only if url_normalize succeeds on both and\n-\t * the results compare equal with strcmp. If -p is given (one url only)\n-\t * and url_normalize succeeds, print the result followed by \"\\n\". If\n-\t * -l is given (one url only) and url_normalize succeeds, print the\n-\t * returned length in decimal followed by \"\\n\".\n-\t */\n-\n-\tif (argc > 1 && !strcmp(argv[1], \"-p\")) {\n-\t\topt_p = 1;\n-\t\targc--;\n-\t\targv++;\n-\t} else if (argc > 1 && !strcmp(argv[1], \"-l\")) {\n-\t\topt_l = 1;\n-\t\targc--;\n-\t\targv++;\n-\t}\n-\n-\tif (argc < 2 || argc > 3)\n-\t\tdie(\"%s\", usage);\n-\n-\tif (argc == 2) {\n-\t\tstruct url_info info;\n-\t\turl1 = url_normalize(argv[1], &info);\n-\t\tif (!url1)\n-\t\t\treturn 1;\n-\t\tif (opt_p)\n-\t\t\tprintf(\"%s\\n\", url1);\n-\t\tif (opt_l)\n-\t\t\tprintf(\"%u\\n\", (unsigned)info.url_len);\n-\t\tgoto cleanup;\n-\t}\n-\n-\tif (opt_p || opt_l)\n-\t\tdie(\"%s\", usage);\n-\n-\turl1 = url_normalize(argv[1], NULL);\n-\turl2 = url_normalize(argv[2], NULL);\n-\tret = (url1 && url2 && !strcmp(url1, url2)) ? 0 : 1;\n-cleanup:\n-\tfree(url1);\n-\tfree(url2);\n-\treturn ret;\n-}\ndiff --git a/t/t0110-urlmatch-normalization.sh b/t/t0110-urlmatch-normalization.sh\ndeleted file mode 100755\nindex 12d817fbd3..0000000000\n--- a/t/t0110-urlmatch-normalization.sh\n+++ /dev/null\n@@ -1,182 +0,0 @@\n-#!/bin/sh\n-\n-test_description='urlmatch URL normalization'\n-\n-TEST_PASSES_SANITIZE_LEAK=true\n-. ./test-lib.sh\n-\n-# The base name of the test url files\n-tu=\"$TEST_DIRECTORY/t0110/url\"\n-\n-# Note that only file: URLs should be allowed without a host\n-\n-test_expect_success 'url scheme' '\n-\t! test-tool urlmatch-normalization \"\" &&\n-\t! test-tool urlmatch-normalization \"_\" &&\n-\t! test-tool urlmatch-normalization \"scheme\" &&\n-\t! test-tool urlmatch-normalization \"scheme:\" &&\n-\t! test-tool urlmatch-normalization \"scheme:/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://\" &&\n-\t! test-tool urlmatch-normalization \"file\" &&\n-\t! test-tool urlmatch-normalization \"file:\" &&\n-\t! test-tool urlmatch-normalization \"file:/\" &&\n-\ttest-tool urlmatch-normalization \"file://\" &&\n-\t! test-tool urlmatch-normalization \"://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"x_test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"-test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"0test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"+test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \".test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"schem%6e://\" &&\n-\ttest-tool urlmatch-normalization \"x-Test+v1.0://acme.co\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"AbCdeF://x.Y\")\" = \"abcdef://x.y/\"\n-'\n-\n-test_expect_success 'url authority' '\n-\t! test-tool urlmatch-normalization \"scheme://user:pass@\" &&\n-\t! test-tool urlmatch-normalization \"scheme://?\" &&\n-\t! test-tool urlmatch-normalization \"scheme://#\" &&\n-\t! test-tool urlmatch-normalization \"scheme:///\" &&\n-\t! test-tool urlmatch-normalization \"scheme://:\" &&\n-\t! test-tool urlmatch-normalization \"scheme://:555\" &&\n-\ttest-tool urlmatch-normalization \"file://user:pass@\" &&\n-\ttest-tool urlmatch-normalization \"file://?\" &&\n-\ttest-tool urlmatch-normalization \"file://#\" &&\n-\ttest-tool urlmatch-normalization \"file:///\" &&\n-\ttest-tool urlmatch-normalization \"file://:\" &&\n-\t! test-tool urlmatch-normalization \"file://:555\" &&\n-\ttest-tool urlmatch-normalization \"scheme://user:pass@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://%00@host\" &&\n-\t! test-tool urlmatch-normalization \"scheme://%%@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host_\" &&\n-\ttest-tool urlmatch-normalization \"scheme://user:pass@host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://@host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host?x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host#x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host/@\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host?@x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host#@x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[::1]\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[::1]/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://hos%41/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[invalid....:/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://invalid....:]/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://invalid....:[/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://invalid....:[\"\n-'\n-\n-test_expect_success 'url port checks' '\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:0\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:0000000\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:0000001?\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:065535#\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:65535\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:65536\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:99999\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:100000\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:100001\" &&\n-\ttest-tool urlmatch-normalization \"http://q@some.host:80\" &&\n-\ttest-tool urlmatch-normalization \"https://q@some.host:443\" &&\n-\ttest-tool urlmatch-normalization \"http://q@some.host:80/\" &&\n-\ttest-tool urlmatch-normalization \"https://q@some.host:443?\" &&\n-\t! test-tool urlmatch-normalization \"http://q@:8008\" &&\n-\t! test-tool urlmatch-normalization \"http://:8080\" &&\n-\t! test-tool urlmatch-normalization \"http://:\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n-\ttest-tool urlmatch-normalization \"xyz://[::1]:456/\" &&\n-\ttest-tool urlmatch-normalization \"xyz://[::1]:/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:000/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:0%300/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:0x80/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:4294967297/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:030f/\"\n-'\n-\n-test_expect_success 'url port normalization' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:0800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:00000800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:065535\")\" = \"http://x:65535/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:1\")\" = \"http://x:1/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:80\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:080\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:000000080\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:443\")\" = \"https://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:0443\")\" = \"https://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:000000443\")\" = \"https://x/\"\n-'\n-\n-test_expect_success 'url general escapes' '\n-\t! test-tool urlmatch-normalization \"http://x.y?%fg\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/%7e%41^%3a\")\" = \"x://w/~A%5E%3A\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/:/?#[]@\")\" = \"x://w/:/?#[]@\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/$&()*+,;=\")\" = \"x://w/$&()*+,;=\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/'\\''\")\" = \"x://w/'\\''\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W?'\\!'\")\" = \"x://w/?'\\!'\"\n-'\n-\n-test_expect_success !MINGW 'url high-bit escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-1\")\")\" = \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-2\")\")\" = \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-3\")\")\" = \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-4\")\")\" = \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-5\")\")\" = \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-6\")\")\" = \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-7\")\")\" = \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-8\")\")\" = \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-9\")\")\" = \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-10\")\")\" = \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\"\n-'\n-\n-test_expect_success 'url utf-8 escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-11\")\")\" = \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\"\n-'\n-\n-test_expect_success 'url username/password escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://%41%62(^):%70+d@foo\")\" = \"x://Ab(%5E):p+d@foo/\"\n-'\n-\n-test_expect_success 'url normalized lengths' '\n-\ttest \"$(test-tool urlmatch-normalization -l \"Http://%4d%65:%4d^%70@The.Host\")\" = 25 &&\n-\ttest \"$(test-tool urlmatch-normalization -l \"http://%41:%42@x.y/%61/\")\" = 17 &&\n-\ttest \"$(test-tool urlmatch-normalization -l \"http://@x.y/^\")\" = 15\n-'\n-\n-test_expect_success 'url . and .. segments' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/.\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/./\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/.\")\" = \"x://y/a\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./\")\" = \"x://y/a/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/.?\")\" = \"x://y/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/./?\")\" = \"x://y/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/.?\")\" = \"x://y/a?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./?\")\" = \"x://y/a/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/.././../c\")\" = \"x://y/c\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/../.././c/\")\" = \"x://y/c/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/.././../c/././.././.\")\" = \"x://y/\" &&\n-\t! test-tool urlmatch-normalization \"x://y/a/./b/.././../c/././.././..\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./?/././..\")\" = \"x://y/a/?/././..\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/%2e/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/%2E/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/%2e./\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/b/.%2E/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/c/%2e%2E/\")\" = \"x://y/\"\n-'\n-\n-# http://@foo specifies an empty user name but does not specify a password\n-# http://foo specifies neither a user name nor a password\n-# So they should not be equivalent\n-test_expect_success 'url equivalents' '\n-\ttest-tool urlmatch-normalization \"httP://x\" \"Http://X/\" &&\n-\ttest-tool urlmatch-normalization \"Http://%4d%65:%4d^%70@The.Host\" \"hTTP://Me:%4D^p@the.HOST:80/\" &&\n-\t! test-tool urlmatch-normalization \"https://@x.y/^\" \"httpS://x.y:443/^\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^\" \"httpS://@x.y:0443/^\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^/../abc\" \"httpS://@x.y:0443/abc\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^/..\" \"httpS://@x.y:0443/\"\n-'\n-\n-test_done\ndiff --git a/t/t0110/README b/t/t0110/README\ndeleted file mode 100644\nindex ad4a50ecd8..0000000000\n--- a/t/t0110/README\n+++ /dev/null\n@@ -1,9 +0,0 @@\n-The url data files in this directory contain URLs with characters\n-in the range 0x01-0x1f and 0x7f-0xff to test the proper normalization\n-of unprintable characters.\n-\n-A select few characters in the 0x01-0x1f range are skipped to help\n-avoid problems running the test itself.\n-\n-The urls are in test files in this directory rather than being\n-embedded in the test script for portability.\ndiff --git a/t/t0110/url-1 b/t/t0110/url-1\ndeleted file mode 100644\nindex 519019c5ce6c58478f048a2f39e2321370d318c6..0000000000000000000000000000000000000000\nGIT binary patch\nliteral 0\nHcmV?d00001\n\nliteral 20\nbcmb=h($_E4XJle#VP#|I;Nuq%6ygE^Admtt\n\ndiff --git a/t/t0110/url-10 b/t/t0110/url-10\ndeleted file mode 100644\nindex b9965de6a5d74b122179821212b2c27c8ae03e80..0000000000000000000000000000000000000000\nGIT binary patch\nliteral 0\nHcmV?d00001\n\nliteral 23\nhcmV+y0O{|dCd5iLo_tE{cAuL^}d3^M=#\n\ndiff --git a/t/t0110/url-6 b/t/t0110/url-6\ndeleted file mode 100644\nindex e8283aac6dff049d3e02454db6e684c5790a5996..0000000000000000000000000000000000000000\nGIT binary patch\nliteral 0\nHcmV?d00001\n\nliteral 23\nhcmV+y0OXb%goKr&kC454l@7%\n\ndiff --git a/t/t0110/url-8 b/t/t0110/url-8\ndeleted file mode 100644\nindex 79a0ba836f5b8886b0a73f161eb292af2b105e65..0000000000000000000000000000000000000000\nGIT binary patch\nliteral 0\nHcmV?d00001\n\nliteral 23\nhcmV+y0OJG&\n\ndiff --git a/t/t0110/url-9 b/t/t0110/url-9\ndeleted file mode 100644\nindex 8b44bec48b94467c63e8e1ad18162e465da6d6dd..0000000000000000000000000000000000000000\nGIT binary patch\nliteral 0\nHcmV?d00001\n\nliteral 23\nhcmV+y0O+J3B?+U9u5HkP(\n\ndiff --git a/t/unit-tests/t-urlmatch-normalization.c b/t/unit-tests/t-urlmatch-normalization.c\nnew file mode 100644\nindex 0000000000..8e62b423cb\n--- /dev/null\n+++ b/t/unit-tests/t-urlmatch-normalization.c\n@@ -0,0 +1,271 @@\n+#include \"test-lib.h\"\n+#include \"urlmatch.h\"\n+\n+static void check_url_normalizable(const char *url, int normalizable)\n+{\n+\tchar *url_norm = url_normalize(url, NULL);\n+\n+\tif (!check_int(normalizable, ==, url_norm ? 1 : 0))\n+\t\ttest_msg(\"input url: %s\", url);\n+\tfree(url_norm);\n+}\n+\n+static void check_normalized_url(const char *url, const char *expect)\n+{\n+\tchar *url_norm = url_normalize(url, NULL);\n+\n+\tif (!check_str(url_norm, expect))\n+\t\ttest_msg(\"input url: %s\", url);\n+\tfree(url_norm);\n+}\n+\n+static void compare_normalized_urls(const char *url1, const char *url2,\n+\t\t\t\t size_t equal)\n+{\n+\tchar *url1_norm = url_normalize(url1, NULL);\n+\tchar *url2_norm = url_normalize(url2, NULL);\n+\n+\tif (equal) {\n+\t\tif (!check_str(url1_norm, url2_norm))\n+\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n+\t\t\t\t url2);\n+\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0)) {\n+\t\ttest_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n+\t\t\t \" input url1: %s\\n input url2: %s\",\n+\t\t\t url1_norm, url2_norm, url1, url2);\n+\t}\n+\tfree(url1_norm);\n+\tfree(url2_norm);\n+}\n+\n+static void check_normalized_url_length(const char *url, size_t len)\n+{\n+\tstruct url_info info;\n+\tchar *url_norm = url_normalize(url, &info);\n+\n+\tif (!check_int(info.url_len, ==, len))\n+\t\ttest_msg(\" input url: %s\\n normalized url: %s\", url,\n+\t\t\t url_norm);\n+\tfree(url_norm);\n+}\n+\n+/* Note that only file: URLs should be allowed without a host */\n+static void t_url_scheme(void)\n+{\n+\tcheck_url_normalizable(\"\", 0);\n+\tcheck_url_normalizable(\"_\", 0);\n+\tcheck_url_normalizable(\"scheme\", 0);\n+\tcheck_url_normalizable(\"scheme:\", 0);\n+\tcheck_url_normalizable(\"scheme:/\", 0);\n+\tcheck_url_normalizable(\"scheme://\", 0);\n+\tcheck_url_normalizable(\"file\", 0);\n+\tcheck_url_normalizable(\"file:\", 0);\n+\tcheck_url_normalizable(\"file:/\", 0);\n+\tcheck_url_normalizable(\"file://\", 1);\n+\tcheck_url_normalizable(\"://acme.co\", 0);\n+\tcheck_url_normalizable(\"x_test://acme.co\", 0);\n+\tcheck_url_normalizable(\"-test://acme.co\", 0);\n+\tcheck_url_normalizable(\"0test://acme.co\", 0);\n+\tcheck_url_normalizable(\"+test://acme.co\", 0);\n+\tcheck_url_normalizable(\".test://acme.co\", 0);\n+\tcheck_url_normalizable(\"schem%6e://\", 0);\n+\tcheck_url_normalizable(\"x-Test+v1.0://acme.co\", 1);\n+\tcheck_normalized_url(\"AbCdeF://x.Y\", \"abcdef://x.y/\");\n+}\n+\n+static void t_url_authority(void)\n+{\n+\tcheck_url_normalizable(\"scheme://user:pass@\", 0);\n+\tcheck_url_normalizable(\"scheme://?\", 0);\n+\tcheck_url_normalizable(\"scheme://#\", 0);\n+\tcheck_url_normalizable(\"scheme:///\", 0);\n+\tcheck_url_normalizable(\"scheme://:\", 0);\n+\tcheck_url_normalizable(\"scheme://:555\", 0);\n+\tcheck_url_normalizable(\"file://user:pass@\", 1);\n+\tcheck_url_normalizable(\"file://?\", 1);\n+\tcheck_url_normalizable(\"file://#\", 1);\n+\tcheck_url_normalizable(\"file:///\", 1);\n+\tcheck_url_normalizable(\"file://:\", 1);\n+\tcheck_url_normalizable(\"file://:555\", 0);\n+\tcheck_url_normalizable(\"scheme://user:pass@host\", 1);\n+\tcheck_url_normalizable(\"scheme://@host\", 1);\n+\tcheck_url_normalizable(\"scheme://%00@host\", 1);\n+\tcheck_url_normalizable(\"scheme://%%@host\", 0);\n+\tcheck_url_normalizable(\"scheme://host_\", 1);\n+\tcheck_url_normalizable(\"scheme://user:pass@host/\", 1);\n+\tcheck_url_normalizable(\"scheme://@host/\", 1);\n+\tcheck_url_normalizable(\"scheme://host/\", 1);\n+\tcheck_url_normalizable(\"scheme://host?x\", 1);\n+\tcheck_url_normalizable(\"scheme://host#x\", 1);\n+\tcheck_url_normalizable(\"scheme://host/@\", 1);\n+\tcheck_url_normalizable(\"scheme://host?@x\", 1);\n+\tcheck_url_normalizable(\"scheme://host#@x\", 1);\n+\tcheck_url_normalizable(\"scheme://[::1]\", 1);\n+\tcheck_url_normalizable(\"scheme://[::1]/\", 1);\n+\tcheck_url_normalizable(\"scheme://hos%41/\", 0);\n+\tcheck_url_normalizable(\"scheme://[invalid....:/\", 1);\n+\tcheck_url_normalizable(\"scheme://invalid....:]/\", 1);\n+\tcheck_url_normalizable(\"scheme://invalid....:[/\", 0);\n+\tcheck_url_normalizable(\"scheme://invalid....:[\", 0);\n+}\n+\n+static void t_url_port(void)\n+{\n+\tcheck_url_normalizable(\"xyz://q@some.host:\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0000000\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0000001?\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:065535#\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:65535\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:65536\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:99999\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:100000\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:100001\", 0);\n+\tcheck_url_normalizable(\"http://q@some.host:80\", 1);\n+\tcheck_url_normalizable(\"https://q@some.host:443\", 1);\n+\tcheck_url_normalizable(\"http://q@some.host:80/\", 1);\n+\tcheck_url_normalizable(\"https://q@some.host:443?\", 1);\n+\tcheck_url_normalizable(\"http://q@:8008\", 0);\n+\tcheck_url_normalizable(\"http://:8080\", 0);\n+\tcheck_url_normalizable(\"http://:\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:000/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:0%300/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:0x80/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:4294967297/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:030f/\", 0);\n+}\n+\n+static void t_url_port_normalization(void)\n+{\n+\tcheck_normalized_url(\"http://x:800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:0800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:00000800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:065535\", \"http://x:65535/\");\n+\tcheck_normalized_url(\"http://x:1\", \"http://x:1/\");\n+\tcheck_normalized_url(\"http://x:80\", \"http://x/\");\n+\tcheck_normalized_url(\"http://x:080\", \"http://x/\");\n+\tcheck_normalized_url(\"http://x:000000080\", \"http://x/\");\n+\tcheck_normalized_url(\"https://x:443\", \"https://x/\");\n+\tcheck_normalized_url(\"https://x:0443\", \"https://x/\");\n+\tcheck_normalized_url(\"https://x:000000443\", \"https://x/\");\n+}\n+\n+static void t_url_general_escape(void)\n+{\n+\tcheck_url_normalizable(\"http://x.y?%fg\", 0);\n+\tcheck_normalized_url(\"X://W/%7e%41^%3a\", \"x://w/~A%5E%3A\");\n+\tcheck_normalized_url(\"X://W/:/?#[]@\", \"x://w/:/?#[]@\");\n+\tcheck_normalized_url(\"X://W/$&()*+,;=\", \"x://w/$&()*+,;=\");\n+\tcheck_normalized_url(\"X://W/'\", \"x://w/'\");\n+\tcheck_normalized_url(\"X://W?!\", \"x://w/?!\");\n+}\n+\n+static void t_url_high_bit(void)\n+{\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08\\x0e\\x0f\\x10\\x11\\x12\",\n+\t\t\"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\x13\\x14\\x15\\x16\\x17\\x18\\x19\\x1b\\x1c\\x1d\\x1e\\x1f\\x7f\",\n+\t\t\"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\x80\\x81\\x82\\x83\\x84\\x85\\x86\\x87\\x88\\x89\\x8a\\x8b\\x8c\\x8d\\x8e\\x8f\",\n+\t\t\"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\x90\\x91\\x92\\x93\\x94\\x95\\x96\\x97\\x98\\x99\\x9a\\x9b\\x9c\\x9d\\x9e\\x9f\",\n+\t\t\"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xa0\\xa1\\xa2\\xa3\\xa4\\xa5\\xa6\\xa7\\xa8\\xa9\\xaa\\xab\\xac\\xad\\xae\\xaf\",\n+\t\t\"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xb0\\xb1\\xb2\\xb3\\xb4\\xb5\\xb6\\xb7\\xb8\\xb9\\xba\\xbb\\xbc\\xbd\\xbe\\xbf\",\n+\t\t\"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xc0\\xc1\\xc2\\xc3\\xc4\\xc5\\xc6\\xc7\\xc8\\xc9\\xca\\xcb\\xcc\\xcd\\xce\\xcf\",\n+\t\t\"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xd0\\xd1\\xd2\\xd3\\xd4\\xd5\\xd6\\xd7\\xd8\\xd9\\xda\\xdb\\xdc\\xdd\\xde\\xdf\",\n+\t\t\"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xe0\\xe1\\xe2\\xe3\\xe4\\xe5\\xe6\\xe7\\xe8\\xe9\\xea\\xeb\\xec\\xed\\xee\\xef\",\n+\t\t\"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xf0\\xf1\\xf2\\xf3\\xf4\\xf5\\xf6\\xf7\\xf8\\xf9\\xfa\\xfb\\xfc\\xfd\\xfe\\xff\",\n+\t\t\"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\");\n+}\n+\n+static void t_url_utf8_escape(void)\n+{\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xc2\\x80\\xdf\\xbf\\xe0\\xa0\\x80\\xef\\xbf\\xbd\\xf0\\x90\\x80\\x80\\xf0\\xaf\\xbf\\xbd\",\n+\t\t\"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\");\n+}\n+\n+static void t_url_username_pass(void)\n+{\n+\tcheck_normalized_url(\"x://%41%62(^):%70+d@foo\", \"x://Ab(%5E):p+d@foo/\");\n+}\n+\n+static void t_url_length(void)\n+{\n+\tcheck_normalized_url_length(\"Http://%4d%65:%4d^%70@The.Host\", 25);\n+\tcheck_normalized_url_length(\"http://%41:%42@x.y/%61/\", 17);\n+\tcheck_normalized_url_length(\"http://@x.y/^\", 15);\n+}\n+\n+static void t_url_dots(void)\n+{\n+\tcheck_normalized_url(\"x://y/.\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/./\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/a/.\", \"x://y/a\");\n+\tcheck_normalized_url(\"x://y/a/./\", \"x://y/a/\");\n+\tcheck_normalized_url(\"x://y/.?\", \"x://y/?\");\n+\tcheck_normalized_url(\"x://y/./?\", \"x://y/?\");\n+\tcheck_normalized_url(\"x://y/a/.?\", \"x://y/a?\");\n+\tcheck_normalized_url(\"x://y/a/./?\", \"x://y/a/?\");\n+\tcheck_normalized_url(\"x://y/a/./b/.././../c\", \"x://y/c\");\n+\tcheck_normalized_url(\"x://y/a/./b/../.././c/\", \"x://y/c/\");\n+\tcheck_normalized_url(\"x://y/a/./b/.././../c/././.././.\", \"x://y/\");\n+\tcheck_url_normalizable(\"x://y/a/./b/.././../c/././.././..\", 0);\n+\tcheck_normalized_url(\"x://y/a/./?/././..\", \"x://y/a/?/././..\");\n+\tcheck_normalized_url(\"x://y/%2e/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/%2E/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/a/%2e./\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/b/.%2E/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/c/%2e%2E/\", \"x://y/\");\n+}\n+\n+/*\n+ * http://@foo specifies an empty user name but does not specify a password\n+ * http://foo specifies neither a user name nor a password\n+ * So they should not be equivalent\n+ */\n+static void t_url_equivalents(void)\n+{\n+\tcompare_normalized_urls(\"httP://x\", \"Http://X/\", 1);\n+\tcompare_normalized_urls(\"Http://%4d%65:%4d^%70@The.Host\", \"hTTP://Me:%4D^p@the.HOST:80/\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^\", \"httpS://x.y:443/^\", 0);\n+\tcompare_normalized_urls(\"https://@x.y/^\", \"httpS://@x.y:0443/^\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^/../abc\", \"httpS://@x.y:0443/abc\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^/..\", \"httpS://@x.y:0443/\", 1);\n+}\n+\n+int cmd_main(int argc UNUSED, const char **argv UNUSED)\n+{\n+\tTEST(t_url_scheme(), \"url scheme\");\n+\tTEST(t_url_authority(), \"url authority\");\n+\tTEST(t_url_port(), \"url port checks\");\n+\tTEST(t_url_port_normalization(), \"url port normalization\");\n+\tTEST(t_url_general_escape(), \"url general escapes\");\n+\tTEST(t_url_high_bit(), \"url high-bit escapes\");\n+\tTEST(t_url_utf8_escape(), \"url utf8 escapes\");\n+\tTEST(t_url_username_pass(), \"url username/password escapes\");\n+\tTEST(t_url_length(), \"url normalized lengths\");\n+\tTEST(t_url_dots(), \"url . and .. segments\");\n+\tTEST(t_url_equivalents(), \"url equivalents\");\n+\treturn test_done();\n+}\n\nRange-diff against v2:\n1: cf47b17a33 ! 1: a73b89c8e0 t: migrate t0110-urlmatch-normalization to the new framework\n @@ Commit message\n pasted verbatim from the shellscript.\n \n Another change is the removal of MINGW prerequisite from one of the\n - test. It was there because[1] on Windows, the command line is a Unicode\n - string, it is not possible to pass arbitrary bytes to a program. But\n - in unit tests we don't have this limitation.\n + test. It was there because[1] on Windows, the command line is a\n + Unicode string, it is not possible to pass arbitrary bytes to a\n + program. But in unit tests we don't have this limitation.\n \n - With the addition of this unit test, we impose a new restriction of\n - running the unit tests from either 't/' or 't/unit-tests/bin/'\n - directories. This is to construct the path to files which contain some\n - input urls under the 't/t-urlmatch-normalization' directory. This\n - restriction is similar to one we have for end-to-end tests, where they\n - can be ran from only 't/'. Addition of 't/unit-tests/bin/' is to allow\n - for running individual tests which is not currently possible via any\n - 'make' targets and also 'unit-tests-test-tool' target is also ran from\n - the 't/unit-tests/bin' directory.\n + And since we can construct strings with arbitrary bytes in C, let's\n + also remove the test files which contain URLs with arbitrary bytes in\n + the 't/t0110' directory and instead embed those URLs in the unit test\n + code itself.\n \n [1]: https://lore.kernel.org/git/53CAC8EF.6020707@gmail.com/\n \n @@ t/t0110-urlmatch-normalization.sh (deleted)\n -\n -test_done\n \n + ## t/t0110/README (deleted) ##\n +@@\n +-The url data files in this directory contain URLs with characters\n +-in the range 0x01-0x1f and 0x7f-0xff to test the proper normalization\n +-of unprintable characters.\n +-\n +-A select few characters in the 0x01-0x1f range are skipped to help\n +-avoid problems running the test itself.\n +-\n +-The urls are in test files in this directory rather than being\n +-embedded in the test script for portability.\n +\n + ## t/t0110/url-1 (deleted) ##\n + Binary files t/t0110/url-1 and /dev/null differ\n +\n + ## t/t0110/url-10 (deleted) ##\n + Binary files t/t0110/url-10 and /dev/null differ\n +\n + ## t/t0110/url-11 (deleted) ##\n + Binary files t/t0110/url-11 and /dev/null differ\n +\n + ## t/t0110/url-2 (deleted) ##\n + Binary files t/t0110/url-2 and /dev/null differ\n +\n + ## t/t0110/url-3 (deleted) ##\n + Binary files t/t0110/url-3 and /dev/null differ\n +\n + ## t/t0110/url-4 (deleted) ##\n + Binary files t/t0110/url-4 and /dev/null differ\n +\n + ## t/t0110/url-5 (deleted) ##\n + Binary files t/t0110/url-5 and /dev/null differ\n +\n + ## t/t0110/url-6 (deleted) ##\n + Binary files t/t0110/url-6 and /dev/null differ\n +\n + ## t/t0110/url-7 (deleted) ##\n + Binary files t/t0110/url-7 and /dev/null differ\n +\n + ## t/t0110/url-8 (deleted) ##\n + Binary files t/t0110/url-8 and /dev/null differ\n +\n + ## t/t0110/url-9 (deleted) ##\n + Binary files t/t0110/url-9 and /dev/null differ\n +\n ## t/unit-tests/t-urlmatch-normalization.c (new) ##\n @@\n +#include \"test-lib.h\"\n +#include \"urlmatch.h\"\n -+#include \"strbuf.h\"\n +\n +static void check_url_normalizable(const char *url, int normalizable)\n +{\n @@ t/unit-tests/t-urlmatch-normalization.c (new)\n +\tfree(url2_norm);\n +}\n +\n -+static void check_normalized_url_from_file(const char *file, const char *expect)\n -+{\n -+\tstruct strbuf content = STRBUF_INIT, path = STRBUF_INIT;\n -+\tchar *cwd_basename;\n -+\n -+\tif (!check_int(strbuf_getcwd(&path), ==, 0))\n -+\t\treturn;\n -+\n -+\tcwd_basename = basename(path.buf);\n -+\tif (!check(!strcmp(cwd_basename, \"t\") || !strcmp(cwd_basename, \"bin\"))) {\n -+\t\ttest_msg(\"BUG: unit-tests should be run from either 't/' or 't/unit-tests/bin' directory\");\n -+\t\treturn;\n -+\t}\n -+\n -+\tstrbuf_strip_suffix(&path, \"/unit-tests/bin\");\n -+\tstrbuf_addf(&path, \"/unit-tests/t-urlmatch-normalization/%s\", file);\n -+\n -+\tif (!check_int(strbuf_read_file(&content, path.buf, 0), >, 0)) {\n -+\t\ttest_msg(\"failed to read from file '%s': %s\", file, strerror(errno));\n -+\t} else {\n -+\t\tchar *url_norm;\n -+\n -+\t\tstrbuf_trim_trailing_newline(&content);\n -+\t\turl_norm = url_normalize(content.buf, NULL);\n -+\t\tif (!check_str(url_norm, expect))\n -+\t\t\ttest_msg(\"input file: %s\", file);\n -+\t\tfree(url_norm);\n -+\t}\n -+\n -+\tstrbuf_release(&content);\n -+\tstrbuf_release(&path);\n -+}\n -+\n +static void check_normalized_url_length(const char *url, size_t len)\n +{\n +\tstruct url_info info;\n @@ t/unit-tests/t-urlmatch-normalization.c (new)\n +\n +static void t_url_high_bit(void)\n +{\n -+\tcheck_normalized_url_from_file(\"url-1\",\n -+\t\t\t \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\");\n -+\tcheck_normalized_url_from_file(\"url-2\",\n -+\t\t\t \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\");\n -+\tcheck_normalized_url_from_file(\"url-3\",\n -+\t\t\t \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\");\n -+\tcheck_normalized_url_from_file(\"url-4\",\n -+\t\t\t \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\");\n -+\tcheck_normalized_url_from_file(\"url-5\",\n -+\t\t\t \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\");\n -+\tcheck_normalized_url_from_file(\"url-6\",\n -+\t\t\t \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\");\n -+\tcheck_normalized_url_from_file(\"url-7\",\n -+\t\t\t \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\");\n -+\tcheck_normalized_url_from_file(\"url-8\",\n -+\t\t\t \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\");\n -+\tcheck_normalized_url_from_file(\"url-9\",\n -+\t\t\t \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\");\n -+\tcheck_normalized_url_from_file(\"url-10\",\n -+\t\t\t \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08\\x0e\\x0f\\x10\\x11\\x12\",\n ++\t\t\"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\x13\\x14\\x15\\x16\\x17\\x18\\x19\\x1b\\x1c\\x1d\\x1e\\x1f\\x7f\",\n ++\t\t\"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\x80\\x81\\x82\\x83\\x84\\x85\\x86\\x87\\x88\\x89\\x8a\\x8b\\x8c\\x8d\\x8e\\x8f\",\n ++\t\t\"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\x90\\x91\\x92\\x93\\x94\\x95\\x96\\x97\\x98\\x99\\x9a\\x9b\\x9c\\x9d\\x9e\\x9f\",\n ++\t\t\"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\xa0\\xa1\\xa2\\xa3\\xa4\\xa5\\xa6\\xa7\\xa8\\xa9\\xaa\\xab\\xac\\xad\\xae\\xaf\",\n ++\t\t\"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\xb0\\xb1\\xb2\\xb3\\xb4\\xb5\\xb6\\xb7\\xb8\\xb9\\xba\\xbb\\xbc\\xbd\\xbe\\xbf\",\n ++\t\t\"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\xc0\\xc1\\xc2\\xc3\\xc4\\xc5\\xc6\\xc7\\xc8\\xc9\\xca\\xcb\\xcc\\xcd\\xce\\xcf\",\n ++\t\t\"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\xd0\\xd1\\xd2\\xd3\\xd4\\xd5\\xd6\\xd7\\xd8\\xd9\\xda\\xdb\\xdc\\xdd\\xde\\xdf\",\n ++\t\t\"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\xe0\\xe1\\xe2\\xe3\\xe4\\xe5\\xe6\\xe7\\xe8\\xe9\\xea\\xeb\\xec\\xed\\xee\\xef\",\n ++\t\t\"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\xf0\\xf1\\xf2\\xf3\\xf4\\xf5\\xf6\\xf7\\xf8\\xf9\\xfa\\xfb\\xfc\\xfd\\xfe\\xff\",\n ++\t\t\"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\");\n +}\n +\n +static void t_url_utf8_escape(void)\n +{\n -+\tcheck_normalized_url_from_file(\"url-11\",\n -+\t\t\t \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\");\n ++\tcheck_normalized_url(\n ++\t\t\"x://q/\\xc2\\x80\\xdf\\xbf\\xe0\\xa0\\x80\\xef\\xbf\\xbd\\xf0\\x90\\x80\\x80\\xf0\\xaf\\xbf\\xbd\",\n ++\t\t\"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\");\n +}\n +\n +static void t_url_username_pass(void)\n @@ t/unit-tests/t-urlmatch-normalization.c (new)\n +\tTEST(t_url_equivalents(), \"url equivalents\");\n +\treturn test_done();\n +}\n -\n - ## t/t0110/README => t/unit-tests/t-urlmatch-normalization/README ##\n -\n - ## t/t0110/url-1 => t/unit-tests/t-urlmatch-normalization/url-1 ##\n -\n - ## t/t0110/url-10 => t/unit-tests/t-urlmatch-normalization/url-10 ##\n -\n - ## t/t0110/url-11 => t/unit-tests/t-urlmatch-normalization/url-11 ##\n -\n - ## t/t0110/url-2 => t/unit-tests/t-urlmatch-normalization/url-2 ##\n -\n - ## t/t0110/url-3 => t/unit-tests/t-urlmatch-normalization/url-3 ##\n -\n - ## t/t0110/url-4 => t/unit-tests/t-urlmatch-normalization/url-4 ##\n -\n - ## t/t0110/url-5 => t/unit-tests/t-urlmatch-normalization/url-5 ##\n -\n - ## t/t0110/url-6 => t/unit-tests/t-urlmatch-normalization/url-6 ##\n -\n - ## t/t0110/url-7 => t/unit-tests/t-urlmatch-normalization/url-7 ##\n -\n - ## t/t0110/url-8 => t/unit-tests/t-urlmatch-normalization/url-8 ##\n -\n - ## t/t0110/url-9 => t/unit-tests/t-urlmatch-normalization/url-9 ##\n-- \n2.46.0\n\n"},{"id":"500912","messageId":"D3FPBTED647O.7JKO1P410ZEE@gmail.com","threadId":"61693","inReplyTo":"xmqqh6bo448j.fsf@gitster.g","subject":"Re: [GSoC][PATCH v2] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Ghanshyam Thakkar","fromEmail":"shyamthakkar001@gmail.com","sentAt":"2024-08-14T14:24:38Z","receivedAt":"2024-08-14T14:24:45Z","isPatch":true,"sender":{"key":"shyamthakkar001@gmail.com","avatar":"https://avatars.githubusercontent.com/u/72698233?v=4"},"body":"Junio C Hamano wrote:\n> Ghanshyam Thakkar writes:\n>\n> > With the addition of this unit test, we impose a new restriction of\n> > running the unit tests from either 't/' or 't/unit-tests/bin/'\n> > directories. This is to construct the path to files which contain some\n> > input urls under the 't/t-urlmatch-normalization' directory. This\n> > restriction is similar to one we have for end-to-end tests, where they\n> > can be ran from only 't/'.\n> >\n> > Addition of 't/unit-tests/bin/' is to allow\n> > for running individual tests which is not currently possible via any\n> > 'make' targets and also 'unit-tests-test-tool' target is also ran from\n> > the 't/unit-tests/bin' directory.\n>\n> Sorry, but I do not quite follow. The above makes it sound as if\n> the 'bin' subdirectory is something that never existed before this\n> patch and this patch introduces the use of that directory, but that\n> is hardly the case. What does that \"Addition of\" really refer to?\n>\n> Do you mean \"we cannot run the tests from arbitrary places, and we\n> allow them to be run from t/, just like the normal tests\" followed\n> by \"in addition, we also allow them to be run from t/unit-tests/bin\n> directory because ...\"?\n\nYes, I meant that. I sent a v3 which embeds those URLs from files into\nthe code itself, which should not require such restriction.\n\nLink: https://lore.kernel.org/git/20240814142057.94671-1-shyamthakkar001@gmail.com/\n\nThanks.\n\n"},{"id":"500918","messageId":"xmqqa5hf11yj.fsf@gitster.g","threadId":"61693","inReplyTo":"20240814142057.94671-1-shyamthakkar001@gmail.com","subject":"Re: [GSoC][PATCH v3] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-14T16:52:36Z","receivedAt":"2024-08-14T16:52:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ghanshyam Thakkar writes:\n\n> And since we can construct strings with arbitrary bytes in C, let's\n> also remove the test files which contain URLs with arbitrary bytes in\n> the 't/t0110' directory and instead embed those URLs in the unit test\n> code itself.\n\nYeah, that was what I meant to say. As long as we do not have an\nembedded NUL in these test pattern strings, \"const char *\" literals\nare fine (and if we need to use embedded NUL, we'd need ).\n\nThanks. Will queue.\n"},{"id":"501269","messageId":"CAP8UFD2-VbyK-ZecDKEvgKicWrVe=e=z6mH_xjmrf=a4ZAYd8w@mail.gmail.com","threadId":"61693","inReplyTo":"20240814142057.94671-1-shyamthakkar001@gmail.com","subject":"Re: [GSoC][PATCH v3] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2024-08-19T12:46:11Z","receivedAt":"2024-08-19T12:46:25Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Wed, Aug 14, 2024 at 4:21 PM Ghanshyam Thakkar\n wrote:\n>\n> helper/test-urlmatch-normalization along with\n> t0110-urlmatch-normalization test the `url_normalize()` function from\n> 'urlmatch.h'. Migrate them to the unit testing framework for better\n> performance. And also add different test_msg()s for better debugging.\n>\n> In the migration, last two of the checks from `t_url_general_escape()`\n> were slightly changed compared to the shellscript. This involves changing\n\nNit: s/shellscript/shell script/\n\n>\n> '\\'' -> '\n> '\\!' -> !\n>\n> in the urls of those checks. This is because in C strings, we don't\n> need to escape \"'\" and \"!\". Other than these two, all the urls were\n> pasted verbatim from the shellscript.\n\nNit: s/shellscript/shell script/\n\n> Another change is the removal of MINGW prerequisite from one of the\n\nNit: s/of MINGW prerequisite/of a MINGW prerequisite/\n\n> test. It was there because[1] on Windows, the command line is a\n> Unicode string, it is not possible to pass arbitrary bytes to a\n> program. But in unit tests we don't have this limitation.\n>\n> And since we can construct strings with arbitrary bytes in C, let's\n> also remove the test files which contain URLs with arbitrary bytes in\n> the 't/t0110' directory and instead embed those URLs in the unit test\n> code itself.\n>\n> [1]: https://lore.kernel.org/git/53CAC8EF.6020707@gmail.com/\n>\n> Mentored-by: Christian Couder \n> Mentored-by: Kaartic Sivaraam \n> Signed-off-by: Ghanshyam Thakkar \n> ---\n> This version addresses Junio's review and removes the restriction\n> of running the unit tests in the 't/' and 't/unit-tests/bin'\n> introduced in v2 by embedding the URLs in the code itself.\n\nNice change.\n\n[...]\n\n> +static void compare_normalized_urls(const char *url1, const char *url2,\n> + size_t equal)\n\nNit: it's better to use 'unsigned int' or just 'int' for bool flags\nlike \"equal\". Or is there a reason to use 'size_t' instead?\n\n> +{\n> + char *url1_norm = url_normalize(url1, NULL);\n> + char *url2_norm = url_normalize(url2, NULL);\n> +\n> + if (equal) {\n> + if (!check_str(url1_norm, url2_norm))\n> + test_msg(\"input url1: %s\\n input url2: %s\", url1,\n> + url2);\n> + } else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0)) {\n> + test_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n> + \" input url1: %s\\n input url2: %s\",\n> + url1_norm, url2_norm, url1, url2);\n\nNit: something like \"normalized url1\" might be a bit better than\n\"url1_norm\" as for the 'url1' variable we use \"input url1\" instead of\njust \"url1\".\n\n> + }\n> + free(url1_norm);\n> + free(url2_norm);\n> +}\n> +\n> +static void check_normalized_url_length(const char *url, size_t len)\n> +{\n> + struct url_info info;\n> + char *url_norm = url_normalize(url, &info);\n> +\n> + if (!check_int(info.url_len, ==, len))\n> + test_msg(\" input url: %s\\n normalized url: %s\", url,\n> + url_norm);\n\nAbove \"normalized url\" is used for \"url_norm\" which is good.\n\n> + free(url_norm);\n> +}\n> +\n> +/* Note that only file: URLs should be allowed without a host */\n\nNit: maybe s/file:/\"file:\"/ would make things a bit clearer.\n\n[...]\n\n> +/*\n> + * http://@foo specifies an empty user name but does not specify a password\n> + * http://foo specifies neither a user name nor a password\n> + * So they should not be equivalent\n> + */\n\nNit: the above comment would be a bit better with URLs inside double\nquotes, with a full stop (period) at the end of each sentence and with\nonly one space character between \"http://foo\" and \"specifies\".\n\nExcept for the above nits, I am happy with this version. Thanks.\n"},{"id":"501378","messageId":"20240820152008.21354-2-shyamthakkar001@gmail.com","threadId":"61693","inReplyTo":"20240814142057.94671-1-shyamthakkar001@gmail.com","subject":"[GSoC][PATCH v4] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Ghanshyam Thakkar","fromEmail":"shyamthakkar001@gmail.com","sentAt":"2024-08-20T15:19:47Z","receivedAt":"2024-08-20T15:20:51Z","isPatch":true,"sender":{"key":"shyamthakkar001@gmail.com","avatar":"https://avatars.githubusercontent.com/u/72698233?v=4"},"body":"helper/test-urlmatch-normalization along with\nt0110-urlmatch-normalization test the `url_normalize()` function from\n'urlmatch.h'. Migrate them to the unit testing framework for better\nperformance. And also add different test_msg()s for better debugging.\n\nIn the migration, last two of the checks from `t_url_general_escape()`\nwere slightly changed compared to the shell script. This involves\nchanging\n\n'\\'' -> '\n'\\!' -> !\n\nin the urls of those checks. This is because in C strings, we don't\nneed to escape \"'\" and \"!\". Other than these two, all the urls were\npasted verbatim from the shell script.\n\nAnother change is the removal of a MINGW prerequisite from one of the\ntest. It was there because[1] on Windows, the command line is a\nUnicode string, it is not possible to pass arbitrary bytes to a\nprogram. But in unit tests we don't have this limitation.\n\nAnd since we can construct strings with arbitrary bytes in C, let's\nalso remove the test files which contain URLs with arbitrary bytes in\nthe 't/t0110' directory and instead embed those URLs in the unit test\ncode itself.\n\n[1]: https://lore.kernel.org/git/53CAC8EF.6020707@gmail.com/\n\nMentored-by: Christian Couder \nMentored-by: Kaartic Sivaraam \nSigned-off-by: Ghanshyam Thakkar \n---\n Makefile | 2 +-\n t/helper/test-tool.c | 1 -\n t/helper/test-tool.h | 1 -\n t/helper/test-urlmatch-normalization.c | 56 -----\n t/t0110-urlmatch-normalization.sh | 182 ----------------\n t/t0110/README | 9 -\n t/t0110/url-1 | Bin 20 -> 0 bytes\n t/t0110/url-10 | Bin 23 -> 0 bytes\n t/t0110/url-11 | Bin 25 -> 0 bytes\n t/t0110/url-2 | Bin 20 -> 0 bytes\n t/t0110/url-3 | Bin 23 -> 0 bytes\n t/t0110/url-4 | Bin 23 -> 0 bytes\n t/t0110/url-5 | Bin 23 -> 0 bytes\n t/t0110/url-6 | Bin 23 -> 0 bytes\n t/t0110/url-7 | Bin 23 -> 0 bytes\n t/t0110/url-8 | Bin 23 -> 0 bytes\n t/t0110/url-9 | Bin 23 -> 0 bytes\n t/unit-tests/t-urlmatch-normalization.c | 271 ++++++++++++++++++++++++\n 18 files changed, 272 insertions(+), 250 deletions(-)\n delete mode 100644 t/helper/test-urlmatch-normalization.c\n delete mode 100755 t/t0110-urlmatch-normalization.sh\n delete mode 100644 t/t0110/README\n delete mode 100644 t/t0110/url-1\n delete mode 100644 t/t0110/url-10\n delete mode 100644 t/t0110/url-11\n delete mode 100644 t/t0110/url-2\n delete mode 100644 t/t0110/url-3\n delete mode 100644 t/t0110/url-4\n delete mode 100644 t/t0110/url-5\n delete mode 100644 t/t0110/url-6\n delete mode 100644 t/t0110/url-7\n delete mode 100644 t/t0110/url-8\n delete mode 100644 t/t0110/url-9\n create mode 100644 t/unit-tests/t-urlmatch-normalization.c\n\ndiff --git a/Makefile b/Makefile\nindex 3863e60b66..d7bc19e823 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -843,7 +843,6 @@ TEST_BUILTINS_OBJS += test-submodule.o\n TEST_BUILTINS_OBJS += test-subprocess.o\n TEST_BUILTINS_OBJS += test-trace2.o\n TEST_BUILTINS_OBJS += test-truncate.o\n-TEST_BUILTINS_OBJS += test-urlmatch-normalization.o\n TEST_BUILTINS_OBJS += test-userdiff.o\n TEST_BUILTINS_OBJS += test-wildmatch.o\n TEST_BUILTINS_OBJS += test-windows-named-pipe.o\n@@ -1346,6 +1345,7 @@ UNIT_TEST_PROGRAMS += t-strbuf\n UNIT_TEST_PROGRAMS += t-strcmp-offset\n UNIT_TEST_PROGRAMS += t-strvec\n UNIT_TEST_PROGRAMS += t-trailer\n+UNIT_TEST_PROGRAMS += t-urlmatch-normalization\n UNIT_TEST_PROGS = $(patsubst %,$(UNIT_TEST_BIN)/%$X,$(UNIT_TEST_PROGRAMS))\n UNIT_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))\n UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex da3e69128a..f8a67df7de 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -83,7 +83,6 @@ static struct test_cmd cmds[] = {\n \t{ \"trace2\", cmd__trace2 },\n \t{ \"truncate\", cmd__truncate },\n \t{ \"userdiff\", cmd__userdiff },\n-\t{ \"urlmatch-normalization\", cmd__urlmatch_normalization },\n \t{ \"xml-encode\", cmd__xml_encode },\n \t{ \"wildmatch\", cmd__wildmatch },\n #ifdef GIT_WINDOWS_NATIVE\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex 642a34578c..e74bc0ffd4 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -76,7 +76,6 @@ int cmd__subprocess(int argc, const char **argv);\n int cmd__trace2(int argc, const char **argv);\n int cmd__truncate(int argc, const char **argv);\n int cmd__userdiff(int argc, const char **argv);\n-int cmd__urlmatch_normalization(int argc, const char **argv);\n int cmd__xml_encode(int argc, const char **argv);\n int cmd__wildmatch(int argc, const char **argv);\n #ifdef GIT_WINDOWS_NATIVE\ndiff --git a/t/helper/test-urlmatch-normalization.c b/t/helper/test-urlmatch-normalization.c\ndeleted file mode 100644\nindex 86edd454f5..0000000000\n--- a/t/helper/test-urlmatch-normalization.c\n+++ /dev/null\n@@ -1,56 +0,0 @@\n-#include \"test-tool.h\"\n-#include \"git-compat-util.h\"\n-#include \"urlmatch.h\"\n-\n-int cmd__urlmatch_normalization(int argc, const char **argv)\n-{\n-\tconst char usage[] = \"test-tool urlmatch-normalization [-p | -l] | \";\n-\tchar *url1 = NULL, *url2 = NULL;\n-\tint opt_p = 0, opt_l = 0;\n-\tint ret = 0;\n-\n-\t/*\n-\t * For one url, succeed if url_normalize succeeds on it, fail otherwise.\n-\t * For two urls, succeed only if url_normalize succeeds on both and\n-\t * the results compare equal with strcmp. If -p is given (one url only)\n-\t * and url_normalize succeeds, print the result followed by \"\\n\". If\n-\t * -l is given (one url only) and url_normalize succeeds, print the\n-\t * returned length in decimal followed by \"\\n\".\n-\t */\n-\n-\tif (argc > 1 && !strcmp(argv[1], \"-p\")) {\n-\t\topt_p = 1;\n-\t\targc--;\n-\t\targv++;\n-\t} else if (argc > 1 && !strcmp(argv[1], \"-l\")) {\n-\t\topt_l = 1;\n-\t\targc--;\n-\t\targv++;\n-\t}\n-\n-\tif (argc < 2 || argc > 3)\n-\t\tdie(\"%s\", usage);\n-\n-\tif (argc == 2) {\n-\t\tstruct url_info info;\n-\t\turl1 = url_normalize(argv[1], &info);\n-\t\tif (!url1)\n-\t\t\treturn 1;\n-\t\tif (opt_p)\n-\t\t\tprintf(\"%s\\n\", url1);\n-\t\tif (opt_l)\n-\t\t\tprintf(\"%u\\n\", (unsigned)info.url_len);\n-\t\tgoto cleanup;\n-\t}\n-\n-\tif (opt_p || opt_l)\n-\t\tdie(\"%s\", usage);\n-\n-\turl1 = url_normalize(argv[1], NULL);\n-\turl2 = url_normalize(argv[2], NULL);\n-\tret = (url1 && url2 && !strcmp(url1, url2)) ? 0 : 1;\n-cleanup:\n-\tfree(url1);\n-\tfree(url2);\n-\treturn ret;\n-}\ndiff --git a/t/t0110-urlmatch-normalization.sh b/t/t0110-urlmatch-normalization.sh\ndeleted file mode 100755\nindex 12d817fbd3..0000000000\n--- a/t/t0110-urlmatch-normalization.sh\n+++ /dev/null\n@@ -1,182 +0,0 @@\n-#!/bin/sh\n-\n-test_description='urlmatch URL normalization'\n-\n-TEST_PASSES_SANITIZE_LEAK=true\n-. ./test-lib.sh\n-\n-# The base name of the test url files\n-tu=\"$TEST_DIRECTORY/t0110/url\"\n-\n-# Note that only file: URLs should be allowed without a host\n-\n-test_expect_success 'url scheme' '\n-\t! test-tool urlmatch-normalization \"\" &&\n-\t! test-tool urlmatch-normalization \"_\" &&\n-\t! test-tool urlmatch-normalization \"scheme\" &&\n-\t! test-tool urlmatch-normalization \"scheme:\" &&\n-\t! test-tool urlmatch-normalization \"scheme:/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://\" &&\n-\t! test-tool urlmatch-normalization \"file\" &&\n-\t! test-tool urlmatch-normalization \"file:\" &&\n-\t! test-tool urlmatch-normalization \"file:/\" &&\n-\ttest-tool urlmatch-normalization \"file://\" &&\n-\t! test-tool urlmatch-normalization \"://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"x_test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"-test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"0test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"+test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \".test://acme.co\" &&\n-\t! test-tool urlmatch-normalization \"schem%6e://\" &&\n-\ttest-tool urlmatch-normalization \"x-Test+v1.0://acme.co\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"AbCdeF://x.Y\")\" = \"abcdef://x.y/\"\n-'\n-\n-test_expect_success 'url authority' '\n-\t! test-tool urlmatch-normalization \"scheme://user:pass@\" &&\n-\t! test-tool urlmatch-normalization \"scheme://?\" &&\n-\t! test-tool urlmatch-normalization \"scheme://#\" &&\n-\t! test-tool urlmatch-normalization \"scheme:///\" &&\n-\t! test-tool urlmatch-normalization \"scheme://:\" &&\n-\t! test-tool urlmatch-normalization \"scheme://:555\" &&\n-\ttest-tool urlmatch-normalization \"file://user:pass@\" &&\n-\ttest-tool urlmatch-normalization \"file://?\" &&\n-\ttest-tool urlmatch-normalization \"file://#\" &&\n-\ttest-tool urlmatch-normalization \"file:///\" &&\n-\ttest-tool urlmatch-normalization \"file://:\" &&\n-\t! test-tool urlmatch-normalization \"file://:555\" &&\n-\ttest-tool urlmatch-normalization \"scheme://user:pass@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://%00@host\" &&\n-\t! test-tool urlmatch-normalization \"scheme://%%@host\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host_\" &&\n-\ttest-tool urlmatch-normalization \"scheme://user:pass@host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://@host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host?x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host#x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host/@\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host?@x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://host#@x\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[::1]\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[::1]/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://hos%41/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://[invalid....:/\" &&\n-\ttest-tool urlmatch-normalization \"scheme://invalid....:]/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://invalid....:[/\" &&\n-\t! test-tool urlmatch-normalization \"scheme://invalid....:[\"\n-'\n-\n-test_expect_success 'url port checks' '\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:0\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:0000000\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:0000001?\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:065535#\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:65535\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:65536\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:99999\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:100000\" &&\n-\t! test-tool urlmatch-normalization \"xyz://q@some.host:100001\" &&\n-\ttest-tool urlmatch-normalization \"http://q@some.host:80\" &&\n-\ttest-tool urlmatch-normalization \"https://q@some.host:443\" &&\n-\ttest-tool urlmatch-normalization \"http://q@some.host:80/\" &&\n-\ttest-tool urlmatch-normalization \"https://q@some.host:443?\" &&\n-\t! test-tool urlmatch-normalization \"http://q@:8008\" &&\n-\t! test-tool urlmatch-normalization \"http://:8080\" &&\n-\t! test-tool urlmatch-normalization \"http://:\" &&\n-\ttest-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n-\ttest-tool urlmatch-normalization \"xyz://[::1]:456/\" &&\n-\ttest-tool urlmatch-normalization \"xyz://[::1]:/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:000/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:0%300/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:0x80/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:4294967297/\" &&\n-\t! test-tool urlmatch-normalization \"xyz://[::1]:030f/\"\n-'\n-\n-test_expect_success 'url port normalization' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:0800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:00000800\")\" = \"http://x:800/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:065535\")\" = \"http://x:65535/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:1\")\" = \"http://x:1/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:80\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:080\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"http://x:000000080\")\" = \"http://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:443\")\" = \"https://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:0443\")\" = \"https://x/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"https://x:000000443\")\" = \"https://x/\"\n-'\n-\n-test_expect_success 'url general escapes' '\n-\t! test-tool urlmatch-normalization \"http://x.y?%fg\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/%7e%41^%3a\")\" = \"x://w/~A%5E%3A\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/:/?#[]@\")\" = \"x://w/:/?#[]@\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/$&()*+,;=\")\" = \"x://w/$&()*+,;=\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W/'\\''\")\" = \"x://w/'\\''\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"X://W?'\\!'\")\" = \"x://w/?'\\!'\"\n-'\n-\n-test_expect_success !MINGW 'url high-bit escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-1\")\")\" = \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-2\")\")\" = \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-3\")\")\" = \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-4\")\")\" = \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-5\")\")\" = \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-6\")\")\" = \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-7\")\")\" = \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-8\")\")\" = \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-9\")\")\" = \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-10\")\")\" = \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\"\n-'\n-\n-test_expect_success 'url utf-8 escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-11\")\")\" = \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\"\n-'\n-\n-test_expect_success 'url username/password escapes' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://%41%62(^):%70+d@foo\")\" = \"x://Ab(%5E):p+d@foo/\"\n-'\n-\n-test_expect_success 'url normalized lengths' '\n-\ttest \"$(test-tool urlmatch-normalization -l \"Http://%4d%65:%4d^%70@The.Host\")\" = 25 &&\n-\ttest \"$(test-tool urlmatch-normalization -l \"http://%41:%42@x.y/%61/\")\" = 17 &&\n-\ttest \"$(test-tool urlmatch-normalization -l \"http://@x.y/^\")\" = 15\n-'\n-\n-test_expect_success 'url . and .. segments' '\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/.\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/./\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/.\")\" = \"x://y/a\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./\")\" = \"x://y/a/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/.?\")\" = \"x://y/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/./?\")\" = \"x://y/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/.?\")\" = \"x://y/a?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./?\")\" = \"x://y/a/?\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/.././../c\")\" = \"x://y/c\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/../.././c/\")\" = \"x://y/c/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/.././../c/././.././.\")\" = \"x://y/\" &&\n-\t! test-tool urlmatch-normalization \"x://y/a/./b/.././../c/././.././..\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/./?/././..\")\" = \"x://y/a/?/././..\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/%2e/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/%2E/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/a/%2e./\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/b/.%2E/\")\" = \"x://y/\" &&\n-\ttest \"$(test-tool urlmatch-normalization -p \"x://y/c/%2e%2E/\")\" = \"x://y/\"\n-'\n-\n-# http://@foo specifies an empty user name but does not specify a password\n-# http://foo specifies neither a user name nor a password\n-# So they should not be equivalent\n-test_expect_success 'url equivalents' '\n-\ttest-tool urlmatch-normalization \"httP://x\" \"Http://X/\" &&\n-\ttest-tool urlmatch-normalization \"Http://%4d%65:%4d^%70@The.Host\" \"hTTP://Me:%4D^p@the.HOST:80/\" &&\n-\t! test-tool urlmatch-normalization \"https://@x.y/^\" \"httpS://x.y:443/^\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^\" \"httpS://@x.y:0443/^\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^/../abc\" \"httpS://@x.y:0443/abc\" &&\n-\ttest-tool urlmatch-normalization \"https://@x.y/^/..\" \"httpS://@x.y:0443/\"\n-'\n-\n-test_done\ndiff --git a/t/t0110/README b/t/t0110/README\ndeleted file mode 100644\nindex ad4a50ecd8..0000000000\n--- a/t/t0110/README\n+++ /dev/null\n@@ -1,9 +0,0 @@\n-The url data files in this directory contain URLs with characters\n-in the range 0x01-0x1f and 0x7f-0xff to test the proper normalization\n-of unprintable characters.\n-\n-A select few characters in the 0x01-0x1f range are skipped to help\n-avoid problems running the test itself.\n-\n-The urls are in test files in this directory rather than being\n-embedded in the test script for portability.\ndiff --git a/t/t0110/url-1 b/t/t0110/url-1\ndeleted file mode 100644\nindex 519019c5ce6c58478f048a2f39e2321370d318c6..0000000000000000000000000000000000000000\nGIT binary patch\nliteral 0\nHcmV?d00001\n\nliteral 20\nbcmb=h($_E4XJle#VP#|I;Nuq%6ygE^Admtt\n\ndiff --git a/t/t0110/url-10 b/t/t0110/url-10\ndeleted file mode 100644\nindex b9965de6a5d74b122179821212b2c27c8ae03e80..0000000000000000000000000000000000000000\nGIT binary patch\nliteral 0\nHcmV?d00001\n\nliteral 23\nhcmV+y0O{|dCd5iLo_tE{cAuL^}d3^M=#\n\ndiff --git a/t/t0110/url-6 b/t/t0110/url-6\ndeleted file mode 100644\nindex e8283aac6dff049d3e02454db6e684c5790a5996..0000000000000000000000000000000000000000\nGIT binary patch\nliteral 0\nHcmV?d00001\n\nliteral 23\nhcmV+y0OXb%goKr&kC454l@7%\n\ndiff --git a/t/t0110/url-8 b/t/t0110/url-8\ndeleted file mode 100644\nindex 79a0ba836f5b8886b0a73f161eb292af2b105e65..0000000000000000000000000000000000000000\nGIT binary patch\nliteral 0\nHcmV?d00001\n\nliteral 23\nhcmV+y0OJG&\n\ndiff --git a/t/t0110/url-9 b/t/t0110/url-9\ndeleted file mode 100644\nindex 8b44bec48b94467c63e8e1ad18162e465da6d6dd..0000000000000000000000000000000000000000\nGIT binary patch\nliteral 0\nHcmV?d00001\n\nliteral 23\nhcmV+y0O+J3B?+U9u5HkP(\n\ndiff --git a/t/unit-tests/t-urlmatch-normalization.c b/t/unit-tests/t-urlmatch-normalization.c\nnew file mode 100644\nindex 0000000000..1769c357b9\n--- /dev/null\n+++ b/t/unit-tests/t-urlmatch-normalization.c\n@@ -0,0 +1,271 @@\n+#include \"test-lib.h\"\n+#include \"urlmatch.h\"\n+\n+static void check_url_normalizable(const char *url, unsigned int normalizable)\n+{\n+\tchar *url_norm = url_normalize(url, NULL);\n+\n+\tif (!check_int(normalizable, ==, url_norm ? 1 : 0))\n+\t\ttest_msg(\"input url: %s\", url);\n+\tfree(url_norm);\n+}\n+\n+static void check_normalized_url(const char *url, const char *expect)\n+{\n+\tchar *url_norm = url_normalize(url, NULL);\n+\n+\tif (!check_str(url_norm, expect))\n+\t\ttest_msg(\"input url: %s\", url);\n+\tfree(url_norm);\n+}\n+\n+static void compare_normalized_urls(const char *url1, const char *url2,\n+\t\t\t\t unsigned int equal)\n+{\n+\tchar *url1_norm = url_normalize(url1, NULL);\n+\tchar *url2_norm = url_normalize(url2, NULL);\n+\n+\tif (equal) {\n+\t\tif (!check_str(url1_norm, url2_norm))\n+\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n+\t\t\t\t url2);\n+\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0)) {\n+\t\ttest_msg(\" normalized url1: %s\\n normalized url2: %s\\n\"\n+\t\t\t \" input url1: %s\\n input url2: %s\",\n+\t\t\t url1_norm, url2_norm, url1, url2);\n+\t}\n+\tfree(url1_norm);\n+\tfree(url2_norm);\n+}\n+\n+static void check_normalized_url_length(const char *url, size_t len)\n+{\n+\tstruct url_info info;\n+\tchar *url_norm = url_normalize(url, &info);\n+\n+\tif (!check_int(info.url_len, ==, len))\n+\t\ttest_msg(\" input url: %s\\n normalized url: %s\", url,\n+\t\t\t url_norm);\n+\tfree(url_norm);\n+}\n+\n+/* Note that only \"file:\" URLs should be allowed without a host */\n+static void t_url_scheme(void)\n+{\n+\tcheck_url_normalizable(\"\", 0);\n+\tcheck_url_normalizable(\"_\", 0);\n+\tcheck_url_normalizable(\"scheme\", 0);\n+\tcheck_url_normalizable(\"scheme:\", 0);\n+\tcheck_url_normalizable(\"scheme:/\", 0);\n+\tcheck_url_normalizable(\"scheme://\", 0);\n+\tcheck_url_normalizable(\"file\", 0);\n+\tcheck_url_normalizable(\"file:\", 0);\n+\tcheck_url_normalizable(\"file:/\", 0);\n+\tcheck_url_normalizable(\"file://\", 1);\n+\tcheck_url_normalizable(\"://acme.co\", 0);\n+\tcheck_url_normalizable(\"x_test://acme.co\", 0);\n+\tcheck_url_normalizable(\"-test://acme.co\", 0);\n+\tcheck_url_normalizable(\"0test://acme.co\", 0);\n+\tcheck_url_normalizable(\"+test://acme.co\", 0);\n+\tcheck_url_normalizable(\".test://acme.co\", 0);\n+\tcheck_url_normalizable(\"schem%6e://\", 0);\n+\tcheck_url_normalizable(\"x-Test+v1.0://acme.co\", 1);\n+\tcheck_normalized_url(\"AbCdeF://x.Y\", \"abcdef://x.y/\");\n+}\n+\n+static void t_url_authority(void)\n+{\n+\tcheck_url_normalizable(\"scheme://user:pass@\", 0);\n+\tcheck_url_normalizable(\"scheme://?\", 0);\n+\tcheck_url_normalizable(\"scheme://#\", 0);\n+\tcheck_url_normalizable(\"scheme:///\", 0);\n+\tcheck_url_normalizable(\"scheme://:\", 0);\n+\tcheck_url_normalizable(\"scheme://:555\", 0);\n+\tcheck_url_normalizable(\"file://user:pass@\", 1);\n+\tcheck_url_normalizable(\"file://?\", 1);\n+\tcheck_url_normalizable(\"file://#\", 1);\n+\tcheck_url_normalizable(\"file:///\", 1);\n+\tcheck_url_normalizable(\"file://:\", 1);\n+\tcheck_url_normalizable(\"file://:555\", 0);\n+\tcheck_url_normalizable(\"scheme://user:pass@host\", 1);\n+\tcheck_url_normalizable(\"scheme://@host\", 1);\n+\tcheck_url_normalizable(\"scheme://%00@host\", 1);\n+\tcheck_url_normalizable(\"scheme://%%@host\", 0);\n+\tcheck_url_normalizable(\"scheme://host_\", 1);\n+\tcheck_url_normalizable(\"scheme://user:pass@host/\", 1);\n+\tcheck_url_normalizable(\"scheme://@host/\", 1);\n+\tcheck_url_normalizable(\"scheme://host/\", 1);\n+\tcheck_url_normalizable(\"scheme://host?x\", 1);\n+\tcheck_url_normalizable(\"scheme://host#x\", 1);\n+\tcheck_url_normalizable(\"scheme://host/@\", 1);\n+\tcheck_url_normalizable(\"scheme://host?@x\", 1);\n+\tcheck_url_normalizable(\"scheme://host#@x\", 1);\n+\tcheck_url_normalizable(\"scheme://[::1]\", 1);\n+\tcheck_url_normalizable(\"scheme://[::1]/\", 1);\n+\tcheck_url_normalizable(\"scheme://hos%41/\", 0);\n+\tcheck_url_normalizable(\"scheme://[invalid....:/\", 1);\n+\tcheck_url_normalizable(\"scheme://invalid....:]/\", 1);\n+\tcheck_url_normalizable(\"scheme://invalid....:[/\", 0);\n+\tcheck_url_normalizable(\"scheme://invalid....:[\", 0);\n+}\n+\n+static void t_url_port(void)\n+{\n+\tcheck_url_normalizable(\"xyz://q@some.host:\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0000000\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:0000001?\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:065535#\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:65535\", 1);\n+\tcheck_url_normalizable(\"xyz://q@some.host:65536\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:99999\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:100000\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:100001\", 0);\n+\tcheck_url_normalizable(\"http://q@some.host:80\", 1);\n+\tcheck_url_normalizable(\"https://q@some.host:443\", 1);\n+\tcheck_url_normalizable(\"http://q@some.host:80/\", 1);\n+\tcheck_url_normalizable(\"https://q@some.host:443?\", 1);\n+\tcheck_url_normalizable(\"http://q@:8008\", 0);\n+\tcheck_url_normalizable(\"http://:8080\", 0);\n+\tcheck_url_normalizable(\"http://:\", 0);\n+\tcheck_url_normalizable(\"xyz://q@some.host:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:456/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:/\", 1);\n+\tcheck_url_normalizable(\"xyz://[::1]:000/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:0%300/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:0x80/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:4294967297/\", 0);\n+\tcheck_url_normalizable(\"xyz://[::1]:030f/\", 0);\n+}\n+\n+static void t_url_port_normalization(void)\n+{\n+\tcheck_normalized_url(\"http://x:800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:0800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:00000800\", \"http://x:800/\");\n+\tcheck_normalized_url(\"http://x:065535\", \"http://x:65535/\");\n+\tcheck_normalized_url(\"http://x:1\", \"http://x:1/\");\n+\tcheck_normalized_url(\"http://x:80\", \"http://x/\");\n+\tcheck_normalized_url(\"http://x:080\", \"http://x/\");\n+\tcheck_normalized_url(\"http://x:000000080\", \"http://x/\");\n+\tcheck_normalized_url(\"https://x:443\", \"https://x/\");\n+\tcheck_normalized_url(\"https://x:0443\", \"https://x/\");\n+\tcheck_normalized_url(\"https://x:000000443\", \"https://x/\");\n+}\n+\n+static void t_url_general_escape(void)\n+{\n+\tcheck_url_normalizable(\"http://x.y?%fg\", 0);\n+\tcheck_normalized_url(\"X://W/%7e%41^%3a\", \"x://w/~A%5E%3A\");\n+\tcheck_normalized_url(\"X://W/:/?#[]@\", \"x://w/:/?#[]@\");\n+\tcheck_normalized_url(\"X://W/$&()*+,;=\", \"x://w/$&()*+,;=\");\n+\tcheck_normalized_url(\"X://W/'\", \"x://w/'\");\n+\tcheck_normalized_url(\"X://W?!\", \"x://w/?!\");\n+}\n+\n+static void t_url_high_bit(void)\n+{\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08\\x0e\\x0f\\x10\\x11\\x12\",\n+\t\t\"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\x13\\x14\\x15\\x16\\x17\\x18\\x19\\x1b\\x1c\\x1d\\x1e\\x1f\\x7f\",\n+\t\t\"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\x80\\x81\\x82\\x83\\x84\\x85\\x86\\x87\\x88\\x89\\x8a\\x8b\\x8c\\x8d\\x8e\\x8f\",\n+\t\t\"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\x90\\x91\\x92\\x93\\x94\\x95\\x96\\x97\\x98\\x99\\x9a\\x9b\\x9c\\x9d\\x9e\\x9f\",\n+\t\t\"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xa0\\xa1\\xa2\\xa3\\xa4\\xa5\\xa6\\xa7\\xa8\\xa9\\xaa\\xab\\xac\\xad\\xae\\xaf\",\n+\t\t\"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xb0\\xb1\\xb2\\xb3\\xb4\\xb5\\xb6\\xb7\\xb8\\xb9\\xba\\xbb\\xbc\\xbd\\xbe\\xbf\",\n+\t\t\"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xc0\\xc1\\xc2\\xc3\\xc4\\xc5\\xc6\\xc7\\xc8\\xc9\\xca\\xcb\\xcc\\xcd\\xce\\xcf\",\n+\t\t\"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xd0\\xd1\\xd2\\xd3\\xd4\\xd5\\xd6\\xd7\\xd8\\xd9\\xda\\xdb\\xdc\\xdd\\xde\\xdf\",\n+\t\t\"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xe0\\xe1\\xe2\\xe3\\xe4\\xe5\\xe6\\xe7\\xe8\\xe9\\xea\\xeb\\xec\\xed\\xee\\xef\",\n+\t\t\"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\");\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xf0\\xf1\\xf2\\xf3\\xf4\\xf5\\xf6\\xf7\\xf8\\xf9\\xfa\\xfb\\xfc\\xfd\\xfe\\xff\",\n+\t\t\"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\");\n+}\n+\n+static void t_url_utf8_escape(void)\n+{\n+\tcheck_normalized_url(\n+\t\t\"x://q/\\xc2\\x80\\xdf\\xbf\\xe0\\xa0\\x80\\xef\\xbf\\xbd\\xf0\\x90\\x80\\x80\\xf0\\xaf\\xbf\\xbd\",\n+\t\t\"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\");\n+}\n+\n+static void t_url_username_pass(void)\n+{\n+\tcheck_normalized_url(\"x://%41%62(^):%70+d@foo\", \"x://Ab(%5E):p+d@foo/\");\n+}\n+\n+static void t_url_length(void)\n+{\n+\tcheck_normalized_url_length(\"Http://%4d%65:%4d^%70@The.Host\", 25);\n+\tcheck_normalized_url_length(\"http://%41:%42@x.y/%61/\", 17);\n+\tcheck_normalized_url_length(\"http://@x.y/^\", 15);\n+}\n+\n+static void t_url_dots(void)\n+{\n+\tcheck_normalized_url(\"x://y/.\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/./\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/a/.\", \"x://y/a\");\n+\tcheck_normalized_url(\"x://y/a/./\", \"x://y/a/\");\n+\tcheck_normalized_url(\"x://y/.?\", \"x://y/?\");\n+\tcheck_normalized_url(\"x://y/./?\", \"x://y/?\");\n+\tcheck_normalized_url(\"x://y/a/.?\", \"x://y/a?\");\n+\tcheck_normalized_url(\"x://y/a/./?\", \"x://y/a/?\");\n+\tcheck_normalized_url(\"x://y/a/./b/.././../c\", \"x://y/c\");\n+\tcheck_normalized_url(\"x://y/a/./b/../.././c/\", \"x://y/c/\");\n+\tcheck_normalized_url(\"x://y/a/./b/.././../c/././.././.\", \"x://y/\");\n+\tcheck_url_normalizable(\"x://y/a/./b/.././../c/././.././..\", 0);\n+\tcheck_normalized_url(\"x://y/a/./?/././..\", \"x://y/a/?/././..\");\n+\tcheck_normalized_url(\"x://y/%2e/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/%2E/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/a/%2e./\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/b/.%2E/\", \"x://y/\");\n+\tcheck_normalized_url(\"x://y/c/%2e%2E/\", \"x://y/\");\n+}\n+\n+/*\n+ * \"http://@foo\" specifies an empty user name but does not specify a password.\n+ * \"http://foo\" specifies neither a user name nor a password.\n+ * So they should not be equivalent.\n+ */\n+static void t_url_equivalents(void)\n+{\n+\tcompare_normalized_urls(\"httP://x\", \"Http://X/\", 1);\n+\tcompare_normalized_urls(\"Http://%4d%65:%4d^%70@The.Host\", \"hTTP://Me:%4D^p@the.HOST:80/\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^\", \"httpS://x.y:443/^\", 0);\n+\tcompare_normalized_urls(\"https://@x.y/^\", \"httpS://@x.y:0443/^\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^/../abc\", \"httpS://@x.y:0443/abc\", 1);\n+\tcompare_normalized_urls(\"https://@x.y/^/..\", \"httpS://@x.y:0443/\", 1);\n+}\n+\n+int cmd_main(int argc UNUSED, const char **argv UNUSED)\n+{\n+\tTEST(t_url_scheme(), \"url scheme\");\n+\tTEST(t_url_authority(), \"url authority\");\n+\tTEST(t_url_port(), \"url port checks\");\n+\tTEST(t_url_port_normalization(), \"url port normalization\");\n+\tTEST(t_url_general_escape(), \"url general escapes\");\n+\tTEST(t_url_high_bit(), \"url high-bit escapes\");\n+\tTEST(t_url_utf8_escape(), \"url utf8 escapes\");\n+\tTEST(t_url_username_pass(), \"url username/password escapes\");\n+\tTEST(t_url_length(), \"url normalized lengths\");\n+\tTEST(t_url_dots(), \"url . and .. segments\");\n+\tTEST(t_url_equivalents(), \"url equivalents\");\n+\treturn test_done();\n+}\n\nRange-diff against v3:\n1: a73b89c8e0 ! 1: ef25954bf8 t: migrate t0110-urlmatch-normalization to the new framework\n @@ Commit message\n performance. And also add different test_msg()s for better debugging.\n \n In the migration, last two of the checks from `t_url_general_escape()`\n - were slightly changed compared to the shellscript. This involves changing\n + were slightly changed compared to the shell script. This involves\n + changing\n \n '\\'' -> '\n '\\!' -> !\n \n in the urls of those checks. This is because in C strings, we don't\n need to escape \"'\" and \"!\". Other than these two, all the urls were\n - pasted verbatim from the shellscript.\n + pasted verbatim from the shell script.\n \n - Another change is the removal of MINGW prerequisite from one of the\n + Another change is the removal of a MINGW prerequisite from one of the\n test. It was there because[1] on Windows, the command line is a\n Unicode string, it is not possible to pass arbitrary bytes to a\n program. But in unit tests we don't have this limitation.\n @@ t/unit-tests/t-urlmatch-normalization.c (new)\n +#include \"test-lib.h\"\n +#include \"urlmatch.h\"\n +\n -+static void check_url_normalizable(const char *url, int normalizable)\n ++static void check_url_normalizable(const char *url, unsigned int normalizable)\n +{\n +\tchar *url_norm = url_normalize(url, NULL);\n +\n @@ t/unit-tests/t-urlmatch-normalization.c (new)\n +}\n +\n +static void compare_normalized_urls(const char *url1, const char *url2,\n -+\t\t\t\t size_t equal)\n ++\t\t\t\t unsigned int equal)\n +{\n +\tchar *url1_norm = url_normalize(url1, NULL);\n +\tchar *url2_norm = url_normalize(url2, NULL);\n @@ t/unit-tests/t-urlmatch-normalization.c (new)\n +\t\t\ttest_msg(\"input url1: %s\\n input url2: %s\", url1,\n +\t\t\t\t url2);\n +\t} else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0)) {\n -+\t\ttest_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n ++\t\ttest_msg(\" normalized url1: %s\\n normalized url2: %s\\n\"\n +\t\t\t \" input url1: %s\\n input url2: %s\",\n +\t\t\t url1_norm, url2_norm, url1, url2);\n +\t}\n @@ t/unit-tests/t-urlmatch-normalization.c (new)\n +\tfree(url_norm);\n +}\n +\n -+/* Note that only file: URLs should be allowed without a host */\n ++/* Note that only \"file:\" URLs should be allowed without a host */\n +static void t_url_scheme(void)\n +{\n +\tcheck_url_normalizable(\"\", 0);\n @@ t/unit-tests/t-urlmatch-normalization.c (new)\n +}\n +\n +/*\n -+ * http://@foo specifies an empty user name but does not specify a password\n -+ * http://foo specifies neither a user name nor a password\n -+ * So they should not be equivalent\n ++ * \"http://@foo\" specifies an empty user name but does not specify a password.\n ++ * \"http://foo\" specifies neither a user name nor a password.\n ++ * So they should not be equivalent.\n + */\n +static void t_url_equivalents(void)\n +{\n-- \n2.46.0\n\n"},{"id":"501380","messageId":"D3KUCRL9BIY3.1N5ZVTM4T9UR9@gmail.com","threadId":"61693","inReplyTo":"20240820152008.21354-2-shyamthakkar001@gmail.com","subject":"Re: [GSoC][PATCH v4] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Ghanshyam Thakkar","fromEmail":"shyamthakkar001@gmail.com","sentAt":"2024-08-20T15:24:18Z","receivedAt":"2024-08-20T15:24:26Z","isPatch":true,"sender":{"key":"shyamthakkar001@gmail.com","avatar":"https://avatars.githubusercontent.com/u/72698233?v=4"},"body":"Ghanshyam Thakkar wrote:\n> helper/test-urlmatch-normalization along with\n> t0110-urlmatch-normalization test the `url_normalize()` function from\n> 'urlmatch.h'. Migrate them to the unit testing framework for better\n> performance. And also add different test_msg()s for better debugging.\n>\n> In the migration, last two of the checks from `t_url_general_escape()`\n> were slightly changed compared to the shell script. This involves\n> changing\n>\n> '\\'' -> '\n> '\\!' -> !\n>\n> in the urls of those checks. This is because in C strings, we don't\n> need to escape \"'\" and \"!\". Other than these two, all the urls were\n> pasted verbatim from the shell script.\n>\n> Another change is the removal of a MINGW prerequisite from one of the\n> test. It was there because[1] on Windows, the command line is a\n> Unicode string, it is not possible to pass arbitrary bytes to a\n> program. But in unit tests we don't have this limitation.\n>\n> And since we can construct strings with arbitrary bytes in C, let's\n> also remove the test files which contain URLs with arbitrary bytes in\n> the 't/t0110' directory and instead embed those URLs in the unit test\n> code itself.\n>\n> [1]: https://lore.kernel.org/git/53CAC8EF.6020707@gmail.com/\n>\n> Mentored-by: Christian Couder \n> Mentored-by: Kaartic Sivaraam \n> Signed-off-by: Ghanshyam Thakkar \n> ---\nChanges in v4:\n\n- fix typos and grammatical inconsistencies in the commit\n message, test_msg()s and comments.\n\n- change int and size_t to unsigned int, where ever applicable.\n\nThanks.\n\n> Makefile | 2 +-\n> t/helper/test-tool.c | 1 -\n> t/helper/test-tool.h | 1 -\n> t/helper/test-urlmatch-normalization.c | 56 -----\n> t/t0110-urlmatch-normalization.sh | 182 ----------------\n> t/t0110/README | 9 -\n> t/t0110/url-1 | Bin 20 -> 0 bytes\n> t/t0110/url-10 | Bin 23 -> 0 bytes\n> t/t0110/url-11 | Bin 25 -> 0 bytes\n> t/t0110/url-2 | Bin 20 -> 0 bytes\n> t/t0110/url-3 | Bin 23 -> 0 bytes\n> t/t0110/url-4 | Bin 23 -> 0 bytes\n> t/t0110/url-5 | Bin 23 -> 0 bytes\n> t/t0110/url-6 | Bin 23 -> 0 bytes\n> t/t0110/url-7 | Bin 23 -> 0 bytes\n> t/t0110/url-8 | Bin 23 -> 0 bytes\n> t/t0110/url-9 | Bin 23 -> 0 bytes\n> t/unit-tests/t-urlmatch-normalization.c | 271 ++++++++++++++++++++++++\n> 18 files changed, 272 insertions(+), 250 deletions(-)\n> delete mode 100644 t/helper/test-urlmatch-normalization.c\n> delete mode 100755 t/t0110-urlmatch-normalization.sh\n> delete mode 100644 t/t0110/README\n> delete mode 100644 t/t0110/url-1\n> delete mode 100644 t/t0110/url-10\n> delete mode 100644 t/t0110/url-11\n> delete mode 100644 t/t0110/url-2\n> delete mode 100644 t/t0110/url-3\n> delete mode 100644 t/t0110/url-4\n> delete mode 100644 t/t0110/url-5\n> delete mode 100644 t/t0110/url-6\n> delete mode 100644 t/t0110/url-7\n> delete mode 100644 t/t0110/url-8\n> delete mode 100644 t/t0110/url-9\n> create mode 100644 t/unit-tests/t-urlmatch-normalization.c\n>\n> diff --git a/Makefile b/Makefile\n> index 3863e60b66..d7bc19e823 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -843,7 +843,6 @@ TEST_BUILTINS_OBJS += test-submodule.o\n> TEST_BUILTINS_OBJS += test-subprocess.o\n> TEST_BUILTINS_OBJS += test-trace2.o\n> TEST_BUILTINS_OBJS += test-truncate.o\n> -TEST_BUILTINS_OBJS += test-urlmatch-normalization.o\n> TEST_BUILTINS_OBJS += test-userdiff.o\n> TEST_BUILTINS_OBJS += test-wildmatch.o\n> TEST_BUILTINS_OBJS += test-windows-named-pipe.o\n> @@ -1346,6 +1345,7 @@ UNIT_TEST_PROGRAMS += t-strbuf\n> UNIT_TEST_PROGRAMS += t-strcmp-offset\n> UNIT_TEST_PROGRAMS += t-strvec\n> UNIT_TEST_PROGRAMS += t-trailer\n> +UNIT_TEST_PROGRAMS += t-urlmatch-normalization\n> UNIT_TEST_PROGS = $(patsubst\n> %,$(UNIT_TEST_BIN)/%$X,$(UNIT_TEST_PROGRAMS))\n> UNIT_TEST_OBJS = $(patsubst\n> %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))\n> UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o\n> diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\n> index da3e69128a..f8a67df7de 100644\n> --- a/t/helper/test-tool.c\n> +++ b/t/helper/test-tool.c\n> @@ -83,7 +83,6 @@ static struct test_cmd cmds[] = {\n> { \"trace2\", cmd__trace2 },\n> { \"truncate\", cmd__truncate },\n> { \"userdiff\", cmd__userdiff },\n> - { \"urlmatch-normalization\", cmd__urlmatch_normalization },\n> { \"xml-encode\", cmd__xml_encode },\n> { \"wildmatch\", cmd__wildmatch },\n> #ifdef GIT_WINDOWS_NATIVE\n> diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\n> index 642a34578c..e74bc0ffd4 100644\n> --- a/t/helper/test-tool.h\n> +++ b/t/helper/test-tool.h\n> @@ -76,7 +76,6 @@ int cmd__subprocess(int argc, const char **argv);\n> int cmd__trace2(int argc, const char **argv);\n> int cmd__truncate(int argc, const char **argv);\n> int cmd__userdiff(int argc, const char **argv);\n> -int cmd__urlmatch_normalization(int argc, const char **argv);\n> int cmd__xml_encode(int argc, const char **argv);\n> int cmd__wildmatch(int argc, const char **argv);\n> #ifdef GIT_WINDOWS_NATIVE\n> diff --git a/t/helper/test-urlmatch-normalization.c\n> b/t/helper/test-urlmatch-normalization.c\n> deleted file mode 100644\n> index 86edd454f5..0000000000\n> --- a/t/helper/test-urlmatch-normalization.c\n> +++ /dev/null\n> @@ -1,56 +0,0 @@\n> -#include \"test-tool.h\"\n> -#include \"git-compat-util.h\"\n> -#include \"urlmatch.h\"\n> -\n> -int cmd__urlmatch_normalization(int argc, const char **argv)\n> -{\n> - const char usage[] = \"test-tool urlmatch-normalization [-p | -l]\n> | \";\n> - char *url1 = NULL, *url2 = NULL;\n> - int opt_p = 0, opt_l = 0;\n> - int ret = 0;\n> -\n> - /*\n> - * For one url, succeed if url_normalize succeeds on it, fail\n> otherwise.\n> - * For two urls, succeed only if url_normalize succeeds on both and\n> - * the results compare equal with strcmp. If -p is given (one url only)\n> - * and url_normalize succeeds, print the result followed by \"\\n\". If\n> - * -l is given (one url only) and url_normalize succeeds, print the\n> - * returned length in decimal followed by \"\\n\".\n> - */\n> -\n> - if (argc > 1 && !strcmp(argv[1], \"-p\")) {\n> - opt_p = 1;\n> - argc--;\n> - argv++;\n> - } else if (argc > 1 && !strcmp(argv[1], \"-l\")) {\n> - opt_l = 1;\n> - argc--;\n> - argv++;\n> - }\n> -\n> - if (argc < 2 || argc > 3)\n> - die(\"%s\", usage);\n> -\n> - if (argc == 2) {\n> - struct url_info info;\n> - url1 = url_normalize(argv[1], &info);\n> - if (!url1)\n> - return 1;\n> - if (opt_p)\n> - printf(\"%s\\n\", url1);\n> - if (opt_l)\n> - printf(\"%u\\n\", (unsigned)info.url_len);\n> - goto cleanup;\n> - }\n> -\n> - if (opt_p || opt_l)\n> - die(\"%s\", usage);\n> -\n> - url1 = url_normalize(argv[1], NULL);\n> - url2 = url_normalize(argv[2], NULL);\n> - ret = (url1 && url2 && !strcmp(url1, url2)) ? 0 : 1;\n> -cleanup:\n> - free(url1);\n> - free(url2);\n> - return ret;\n> -}\n> diff --git a/t/t0110-urlmatch-normalization.sh\n> b/t/t0110-urlmatch-normalization.sh\n> deleted file mode 100755\n> index 12d817fbd3..0000000000\n> --- a/t/t0110-urlmatch-normalization.sh\n> +++ /dev/null\n> @@ -1,182 +0,0 @@\n> -#!/bin/sh\n> -\n> -test_description='urlmatch URL normalization'\n> -\n> -TEST_PASSES_SANITIZE_LEAK=true\n> -. ./test-lib.sh\n> -\n> -# The base name of the test url files\n> -tu=\"$TEST_DIRECTORY/t0110/url\"\n> -\n> -# Note that only file: URLs should be allowed without a host\n> -\n> -test_expect_success 'url scheme' '\n> - ! test-tool urlmatch-normalization \"\" &&\n> - ! test-tool urlmatch-normalization \"_\" &&\n> - ! test-tool urlmatch-normalization \"scheme\" &&\n> - ! test-tool urlmatch-normalization \"scheme:\" &&\n> - ! test-tool urlmatch-normalization \"scheme:/\" &&\n> - ! test-tool urlmatch-normalization \"scheme://\" &&\n> - ! test-tool urlmatch-normalization \"file\" &&\n> - ! test-tool urlmatch-normalization \"file:\" &&\n> - ! test-tool urlmatch-normalization \"file:/\" &&\n> - test-tool urlmatch-normalization \"file://\" &&\n> - ! test-tool urlmatch-normalization \"://acme.co\" &&\n> - ! test-tool urlmatch-normalization \"x_test://acme.co\" &&\n> - ! test-tool urlmatch-normalization \"-test://acme.co\" &&\n> - ! test-tool urlmatch-normalization \"0test://acme.co\" &&\n> - ! test-tool urlmatch-normalization \"+test://acme.co\" &&\n> - ! test-tool urlmatch-normalization \".test://acme.co\" &&\n> - ! test-tool urlmatch-normalization \"schem%6e://\" &&\n> - test-tool urlmatch-normalization \"x-Test+v1.0://acme.co\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"AbCdeF://x.Y\")\" =\n> \"abcdef://x.y/\"\n> -'\n> -\n> -test_expect_success 'url authority' '\n> - ! test-tool urlmatch-normalization \"scheme://user:pass@\" &&\n> - ! test-tool urlmatch-normalization \"scheme://?\" &&\n> - ! test-tool urlmatch-normalization \"scheme://#\" &&\n> - ! test-tool urlmatch-normalization \"scheme:///\" &&\n> - ! test-tool urlmatch-normalization \"scheme://:\" &&\n> - ! test-tool urlmatch-normalization \"scheme://:555\" &&\n> - test-tool urlmatch-normalization \"file://user:pass@\" &&\n> - test-tool urlmatch-normalization \"file://?\" &&\n> - test-tool urlmatch-normalization \"file://#\" &&\n> - test-tool urlmatch-normalization \"file:///\" &&\n> - test-tool urlmatch-normalization \"file://:\" &&\n> - ! test-tool urlmatch-normalization \"file://:555\" &&\n> - test-tool urlmatch-normalization \"scheme://user:pass@host\" &&\n> - test-tool urlmatch-normalization \"scheme://@host\" &&\n> - test-tool urlmatch-normalization \"scheme://%00@host\" &&\n> - ! test-tool urlmatch-normalization \"scheme://%%@host\" &&\n> - test-tool urlmatch-normalization \"scheme://host_\" &&\n> - test-tool urlmatch-normalization \"scheme://user:pass@host/\" &&\n> - test-tool urlmatch-normalization \"scheme://@host/\" &&\n> - test-tool urlmatch-normalization \"scheme://host/\" &&\n> - test-tool urlmatch-normalization \"scheme://host?x\" &&\n> - test-tool urlmatch-normalization \"scheme://host#x\" &&\n> - test-tool urlmatch-normalization \"scheme://host/@\" &&\n> - test-tool urlmatch-normalization \"scheme://host?@x\" &&\n> - test-tool urlmatch-normalization \"scheme://host#@x\" &&\n> - test-tool urlmatch-normalization \"scheme://[::1]\" &&\n> - test-tool urlmatch-normalization \"scheme://[::1]/\" &&\n> - ! test-tool urlmatch-normalization \"scheme://hos%41/\" &&\n> - test-tool urlmatch-normalization \"scheme://[invalid....:/\" &&\n> - test-tool urlmatch-normalization \"scheme://invalid....:]/\" &&\n> - ! test-tool urlmatch-normalization \"scheme://invalid....:[/\" &&\n> - ! test-tool urlmatch-normalization \"scheme://invalid....:[\"\n> -'\n> -\n> -test_expect_success 'url port checks' '\n> - test-tool urlmatch-normalization \"xyz://q@some.host:\" &&\n> - test-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:0\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:0000000\" &&\n> - test-tool urlmatch-normalization \"xyz://q@some.host:0000001?\" &&\n> - test-tool urlmatch-normalization \"xyz://q@some.host:065535#\" &&\n> - test-tool urlmatch-normalization \"xyz://q@some.host:65535\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:65536\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:99999\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:100000\" &&\n> - ! test-tool urlmatch-normalization \"xyz://q@some.host:100001\" &&\n> - test-tool urlmatch-normalization \"http://q@some.host:80\" &&\n> - test-tool urlmatch-normalization \"https://q@some.host:443\" &&\n> - test-tool urlmatch-normalization \"http://q@some.host:80/\" &&\n> - test-tool urlmatch-normalization \"https://q@some.host:443?\" &&\n> - ! test-tool urlmatch-normalization \"http://q@:8008\" &&\n> - ! test-tool urlmatch-normalization \"http://:8080\" &&\n> - ! test-tool urlmatch-normalization \"http://:\" &&\n> - test-tool urlmatch-normalization \"xyz://q@some.host:456/\" &&\n> - test-tool urlmatch-normalization \"xyz://[::1]:456/\" &&\n> - test-tool urlmatch-normalization \"xyz://[::1]:/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://[::1]:000/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://[::1]:0%300/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://[::1]:0x80/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://[::1]:4294967297/\" &&\n> - ! test-tool urlmatch-normalization \"xyz://[::1]:030f/\"\n> -'\n> -\n> -test_expect_success 'url port normalization' '\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:800\")\" =\n> \"http://x:800/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:0800\")\" =\n> \"http://x:800/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:00000800\")\" =\n> \"http://x:800/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:065535\")\" =\n> \"http://x:65535/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:1\")\" =\n> \"http://x:1/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:80\")\" =\n> \"http://x/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:080\")\" =\n> \"http://x/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"http://x:000000080\")\" =\n> \"http://x/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"https://x:443\")\" =\n> \"https://x/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"https://x:0443\")\" =\n> \"https://x/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"https://x:000000443\")\" =\n> \"https://x/\"\n> -'\n> -\n> -test_expect_success 'url general escapes' '\n> - ! test-tool urlmatch-normalization \"http://x.y?%fg\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"X://W/%7e%41^%3a\")\" =\n> \"x://w/~A%5E%3A\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"X://W/:/?#[]@\")\" =\n> \"x://w/:/?#[]@\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"X://W/$&()*+,;=\")\" =\n> \"x://w/$&()*+,;=\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"X://W/'\\''\")\" =\n> \"x://w/'\\''\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"X://W?'\\!'\")\" =\n> \"x://w/?'\\!'\"\n> -'\n> -\n> -test_expect_success !MINGW 'url high-bit escapes' '\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-1\")\")\" =\n> \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-2\")\")\" =\n> \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-3\")\")\" =\n> \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-4\")\")\" =\n> \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-5\")\")\" =\n> \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-6\")\")\" =\n> \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-7\")\")\" =\n> \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-8\")\")\" =\n> \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-9\")\")\" =\n> \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-10\")\")\" =\n> \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\"\n> -'\n> -\n> -test_expect_success 'url utf-8 escapes' '\n> - test \"$(test-tool urlmatch-normalization -p \"$(cat \"$tu-11\")\")\" =\n> \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\"\n> -'\n> -\n> -test_expect_success 'url username/password escapes' '\n> - test \"$(test-tool urlmatch-normalization -p\n> \"x://%41%62(^):%70+d@foo\")\" = \"x://Ab(%5E):p+d@foo/\"\n> -'\n> -\n> -test_expect_success 'url normalized lengths' '\n> - test \"$(test-tool urlmatch-normalization -l\n> \"Http://%4d%65:%4d^%70@The.Host\")\" = 25 &&\n> - test \"$(test-tool urlmatch-normalization -l\n> \"http://%41:%42@x.y/%61/\")\" = 17 &&\n> - test \"$(test-tool urlmatch-normalization -l \"http://@x.y/^\")\" = 15\n> -'\n> -\n> -test_expect_success 'url . and .. segments' '\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/.\")\" = \"x://y/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/./\")\" = \"x://y/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/.\")\" = \"x://y/a\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/./\")\" =\n> \"x://y/a/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/.?\")\" = \"x://y/?\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/./?\")\" = \"x://y/?\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/.?\")\" =\n> \"x://y/a?\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/./?\")\" =\n> \"x://y/a/?\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/.././../c\")\"\n> = \"x://y/c\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/./b/../.././c/\")\"\n> = \"x://y/c/\" &&\n> - test \"$(test-tool urlmatch-normalization -p\n> \"x://y/a/./b/.././../c/././.././.\")\" = \"x://y/\" &&\n> - ! test-tool urlmatch-normalization \"x://y/a/./b/.././../c/././.././..\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/./?/././..\")\" =\n> \"x://y/a/?/././..\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/%2e/\")\" = \"x://y/\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/%2E/\")\" = \"x://y/\"\n> &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/a/%2e./\")\" =\n> \"x://y/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/b/.%2E/\")\" =\n> \"x://y/\" &&\n> - test \"$(test-tool urlmatch-normalization -p \"x://y/c/%2e%2E/\")\" =\n> \"x://y/\"\n> -'\n> -\n> -# http://@foo specifies an empty user name but does not specify a\n> password\n> -# http://foo specifies neither a user name nor a password\n> -# So they should not be equivalent\n> -test_expect_success 'url equivalents' '\n> - test-tool urlmatch-normalization \"httP://x\" \"Http://X/\" &&\n> - test-tool urlmatch-normalization \"Http://%4d%65:%4d^%70@The.Host\"\n> \"hTTP://Me:%4D^p@the.HOST:80/\" &&\n> - ! test-tool urlmatch-normalization \"https://@x.y/^\"\n> \"httpS://x.y:443/^\" &&\n> - test-tool urlmatch-normalization \"https://@x.y/^\"\n> \"httpS://@x.y:0443/^\" &&\n> - test-tool urlmatch-normalization \"https://@x.y/^/../abc\"\n> \"httpS://@x.y:0443/abc\" &&\n> - test-tool urlmatch-normalization \"https://@x.y/^/..\"\n> \"httpS://@x.y:0443/\"\n> -'\n> -\n> -test_done\n> diff --git a/t/t0110/README b/t/t0110/README\n> deleted file mode 100644\n> index ad4a50ecd8..0000000000\n> --- a/t/t0110/README\n> +++ /dev/null\n> @@ -1,9 +0,0 @@\n> -The url data files in this directory contain URLs with characters\n> -in the range 0x01-0x1f and 0x7f-0xff to test the proper normalization\n> -of unprintable characters.\n> -\n> -A select few characters in the 0x01-0x1f range are skipped to help\n> -avoid problems running the test itself.\n> -\n> -The urls are in test files in this directory rather than being\n> -embedded in the test script for portability.\n> diff --git a/t/t0110/url-1 b/t/t0110/url-1\n> deleted file mode 100644\n> index\n> 519019c5ce6c58478f048a2f39e2321370d318c6..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 20\n> bcmb=h($_E4XJle#VP#|I;Nuq%6ygE^Admtt\n>\n> diff --git a/t/t0110/url-10 b/t/t0110/url-10\n> deleted file mode 100644\n> index\n> b9965de6a5d74b122179821212b2c27c8ae03e80..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 23\n> hcmV+y0O{|dCd5i\n> diff --git a/t/t0110/url-11 b/t/t0110/url-11\n> deleted file mode 100644\n> index\n> f0a50f10096a20d597f40c775f09a71276e0050a..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 25\n> hcmb=h($_E4Kh$u4|APe$@AvQhFrlI0!}|Suxd5(W4xs=5\n>\n> diff --git a/t/t0110/url-2 b/t/t0110/url-2\n> deleted file mode 100644\n> index\n> 43334b05b2de3794d6020abd96e634a4e9e49cb0..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 20\n> bcmb=h($_E47Zwo}6PJ*bmXVc{ujc{)C{+Vx\n>\n> diff --git a/t/t0110/url-3 b/t/t0110/url-3\n> deleted file mode 100644\n> index\n> 7378c7bec247b996bc67b00a05ed89cf47d4b7a7..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 23\n> ecmb=h($_E4Z)j|4ZfR|6@96C6?&\n> diff --git a/t/t0110/url-4 b/t/t0110/url-4\n> deleted file mode 100644\n> index\n> 220b198c97f942fea4960f51a2105cc42261061a..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 23\n> hcmV+y0O\n> diff --git a/t/t0110/url-5 b/t/t0110/url-5\n> deleted file mode 100644\n> index\n> 1ccd9277792840955bb124bdde21f4b08bcccb63..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 23\n> hcmV+y0OLo_tE{cAuL^}d3^M=#\n>\n> diff --git a/t/t0110/url-6 b/t/t0110/url-6\n> deleted file mode 100644\n> index\n> e8283aac6dff049d3e02454db6e684c5790a5996..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 23\n> hcmV+y0O\n> diff --git a/t/t0110/url-7 b/t/t0110/url-7\n> deleted file mode 100644\n> index\n> fa7c10b615259deefd15b638b021da7c60eba1b2..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 23\n> hcmV+y0OXb%goKr&kC454l@7%\n>\n> diff --git a/t/t0110/url-8 b/t/t0110/url-8\n> deleted file mode 100644\n> index\n> 79a0ba836f5b8886b0a73f161eb292af2b105e65..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 23\n> hcmV+y0OJG&\n>\n> diff --git a/t/t0110/url-9 b/t/t0110/url-9\n> deleted file mode 100644\n> index\n> 8b44bec48b94467c63e8e1ad18162e465da6d6dd..0000000000000000000000000000000000000000\n> GIT binary patch\n> literal 0\n> HcmV?d00001\n>\n> literal 23\n> hcmV+y0O+J3B?+U9u5HkP(\n>\n> diff --git a/t/unit-tests/t-urlmatch-normalization.c\n> b/t/unit-tests/t-urlmatch-normalization.c\n> new file mode 100644\n> index 0000000000..1769c357b9\n> --- /dev/null\n> +++ b/t/unit-tests/t-urlmatch-normalization.c\n> @@ -0,0 +1,271 @@\n> +#include \"test-lib.h\"\n> +#include \"urlmatch.h\"\n> +\n> +static void check_url_normalizable(const char *url, unsigned int\n> normalizable)\n> +{\n> + char *url_norm = url_normalize(url, NULL);\n> +\n> + if (!check_int(normalizable, ==, url_norm ? 1 : 0))\n> + test_msg(\"input url: %s\", url);\n> + free(url_norm);\n> +}\n> +\n> +static void check_normalized_url(const char *url, const char *expect)\n> +{\n> + char *url_norm = url_normalize(url, NULL);\n> +\n> + if (!check_str(url_norm, expect))\n> + test_msg(\"input url: %s\", url);\n> + free(url_norm);\n> +}\n> +\n> +static void compare_normalized_urls(const char *url1, const char *url2,\n> + unsigned int equal)\n> +{\n> + char *url1_norm = url_normalize(url1, NULL);\n> + char *url2_norm = url_normalize(url2, NULL);\n> +\n> + if (equal) {\n> + if (!check_str(url1_norm, url2_norm))\n> + test_msg(\"input url1: %s\\n input url2: %s\", url1,\n> + url2);\n> + } else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0)) {\n> + test_msg(\" normalized url1: %s\\n normalized url2: %s\\n\"\n> + \" input url1: %s\\n input url2: %s\",\n> + url1_norm, url2_norm, url1, url2);\n> + }\n> + free(url1_norm);\n> + free(url2_norm);\n> +}\n> +\n> +static void check_normalized_url_length(const char *url, size_t len)\n> +{\n> + struct url_info info;\n> + char *url_norm = url_normalize(url, &info);\n> +\n> + if (!check_int(info.url_len, ==, len))\n> + test_msg(\" input url: %s\\n normalized url: %s\", url,\n> + url_norm);\n> + free(url_norm);\n> +}\n> +\n> +/* Note that only \"file:\" URLs should be allowed without a host */\n> +static void t_url_scheme(void)\n> +{\n> + check_url_normalizable(\"\", 0);\n> + check_url_normalizable(\"_\", 0);\n> + check_url_normalizable(\"scheme\", 0);\n> + check_url_normalizable(\"scheme:\", 0);\n> + check_url_normalizable(\"scheme:/\", 0);\n> + check_url_normalizable(\"scheme://\", 0);\n> + check_url_normalizable(\"file\", 0);\n> + check_url_normalizable(\"file:\", 0);\n> + check_url_normalizable(\"file:/\", 0);\n> + check_url_normalizable(\"file://\", 1);\n> + check_url_normalizable(\"://acme.co\", 0);\n> + check_url_normalizable(\"x_test://acme.co\", 0);\n> + check_url_normalizable(\"-test://acme.co\", 0);\n> + check_url_normalizable(\"0test://acme.co\", 0);\n> + check_url_normalizable(\"+test://acme.co\", 0);\n> + check_url_normalizable(\".test://acme.co\", 0);\n> + check_url_normalizable(\"schem%6e://\", 0);\n> + check_url_normalizable(\"x-Test+v1.0://acme.co\", 1);\n> + check_normalized_url(\"AbCdeF://x.Y\", \"abcdef://x.y/\");\n> +}\n> +\n> +static void t_url_authority(void)\n> +{\n> + check_url_normalizable(\"scheme://user:pass@\", 0);\n> + check_url_normalizable(\"scheme://?\", 0);\n> + check_url_normalizable(\"scheme://#\", 0);\n> + check_url_normalizable(\"scheme:///\", 0);\n> + check_url_normalizable(\"scheme://:\", 0);\n> + check_url_normalizable(\"scheme://:555\", 0);\n> + check_url_normalizable(\"file://user:pass@\", 1);\n> + check_url_normalizable(\"file://?\", 1);\n> + check_url_normalizable(\"file://#\", 1);\n> + check_url_normalizable(\"file:///\", 1);\n> + check_url_normalizable(\"file://:\", 1);\n> + check_url_normalizable(\"file://:555\", 0);\n> + check_url_normalizable(\"scheme://user:pass@host\", 1);\n> + check_url_normalizable(\"scheme://@host\", 1);\n> + check_url_normalizable(\"scheme://%00@host\", 1);\n> + check_url_normalizable(\"scheme://%%@host\", 0);\n> + check_url_normalizable(\"scheme://host_\", 1);\n> + check_url_normalizable(\"scheme://user:pass@host/\", 1);\n> + check_url_normalizable(\"scheme://@host/\", 1);\n> + check_url_normalizable(\"scheme://host/\", 1);\n> + check_url_normalizable(\"scheme://host?x\", 1);\n> + check_url_normalizable(\"scheme://host#x\", 1);\n> + check_url_normalizable(\"scheme://host/@\", 1);\n> + check_url_normalizable(\"scheme://host?@x\", 1);\n> + check_url_normalizable(\"scheme://host#@x\", 1);\n> + check_url_normalizable(\"scheme://[::1]\", 1);\n> + check_url_normalizable(\"scheme://[::1]/\", 1);\n> + check_url_normalizable(\"scheme://hos%41/\", 0);\n> + check_url_normalizable(\"scheme://[invalid....:/\", 1);\n> + check_url_normalizable(\"scheme://invalid....:]/\", 1);\n> + check_url_normalizable(\"scheme://invalid....:[/\", 0);\n> + check_url_normalizable(\"scheme://invalid....:[\", 0);\n> +}\n> +\n> +static void t_url_port(void)\n> +{\n> + check_url_normalizable(\"xyz://q@some.host:\", 1);\n> + check_url_normalizable(\"xyz://q@some.host:456/\", 1);\n> + check_url_normalizable(\"xyz://q@some.host:0\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:0000000\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:0000001?\", 1);\n> + check_url_normalizable(\"xyz://q@some.host:065535#\", 1);\n> + check_url_normalizable(\"xyz://q@some.host:65535\", 1);\n> + check_url_normalizable(\"xyz://q@some.host:65536\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:99999\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:100000\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:100001\", 0);\n> + check_url_normalizable(\"http://q@some.host:80\", 1);\n> + check_url_normalizable(\"https://q@some.host:443\", 1);\n> + check_url_normalizable(\"http://q@some.host:80/\", 1);\n> + check_url_normalizable(\"https://q@some.host:443?\", 1);\n> + check_url_normalizable(\"http://q@:8008\", 0);\n> + check_url_normalizable(\"http://:8080\", 0);\n> + check_url_normalizable(\"http://:\", 0);\n> + check_url_normalizable(\"xyz://q@some.host:456/\", 1);\n> + check_url_normalizable(\"xyz://[::1]:456/\", 1);\n> + check_url_normalizable(\"xyz://[::1]:/\", 1);\n> + check_url_normalizable(\"xyz://[::1]:000/\", 0);\n> + check_url_normalizable(\"xyz://[::1]:0%300/\", 0);\n> + check_url_normalizable(\"xyz://[::1]:0x80/\", 0);\n> + check_url_normalizable(\"xyz://[::1]:4294967297/\", 0);\n> + check_url_normalizable(\"xyz://[::1]:030f/\", 0);\n> +}\n> +\n> +static void t_url_port_normalization(void)\n> +{\n> + check_normalized_url(\"http://x:800\", \"http://x:800/\");\n> + check_normalized_url(\"http://x:0800\", \"http://x:800/\");\n> + check_normalized_url(\"http://x:00000800\", \"http://x:800/\");\n> + check_normalized_url(\"http://x:065535\", \"http://x:65535/\");\n> + check_normalized_url(\"http://x:1\", \"http://x:1/\");\n> + check_normalized_url(\"http://x:80\", \"http://x/\");\n> + check_normalized_url(\"http://x:080\", \"http://x/\");\n> + check_normalized_url(\"http://x:000000080\", \"http://x/\");\n> + check_normalized_url(\"https://x:443\", \"https://x/\");\n> + check_normalized_url(\"https://x:0443\", \"https://x/\");\n> + check_normalized_url(\"https://x:000000443\", \"https://x/\");\n> +}\n> +\n> +static void t_url_general_escape(void)\n> +{\n> + check_url_normalizable(\"http://x.y?%fg\", 0);\n> + check_normalized_url(\"X://W/%7e%41^%3a\", \"x://w/~A%5E%3A\");\n> + check_normalized_url(\"X://W/:/?#[]@\", \"x://w/:/?#[]@\");\n> + check_normalized_url(\"X://W/$&()*+,;=\", \"x://w/$&()*+,;=\");\n> + check_normalized_url(\"X://W/'\", \"x://w/'\");\n> + check_normalized_url(\"X://W?!\", \"x://w/?!\");\n> +}\n> +\n> +static void t_url_high_bit(void)\n> +{\n> + check_normalized_url(\n> + \"x://q/\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08\\x0e\\x0f\\x10\\x11\\x12\",\n> + \"x://q/%01%02%03%04%05%06%07%08%0E%0F%10%11%12\");\n> + check_normalized_url(\n> + \"x://q/\\x13\\x14\\x15\\x16\\x17\\x18\\x19\\x1b\\x1c\\x1d\\x1e\\x1f\\x7f\",\n> + \"x://q/%13%14%15%16%17%18%19%1B%1C%1D%1E%1F%7F\");\n> + check_normalized_url(\n> +\n> \"x://q/\\x80\\x81\\x82\\x83\\x84\\x85\\x86\\x87\\x88\\x89\\x8a\\x8b\\x8c\\x8d\\x8e\\x8f\",\n> + \"x://q/%80%81%82%83%84%85%86%87%88%89%8A%8B%8C%8D%8E%8F\");\n> + check_normalized_url(\n> +\n> \"x://q/\\x90\\x91\\x92\\x93\\x94\\x95\\x96\\x97\\x98\\x99\\x9a\\x9b\\x9c\\x9d\\x9e\\x9f\",\n> + \"x://q/%90%91%92%93%94%95%96%97%98%99%9A%9B%9C%9D%9E%9F\");\n> + check_normalized_url(\n> +\n> \"x://q/\\xa0\\xa1\\xa2\\xa3\\xa4\\xa5\\xa6\\xa7\\xa8\\xa9\\xaa\\xab\\xac\\xad\\xae\\xaf\",\n> + \"x://q/%A0%A1%A2%A3%A4%A5%A6%A7%A8%A9%AA%AB%AC%AD%AE%AF\");\n> + check_normalized_url(\n> +\n> \"x://q/\\xb0\\xb1\\xb2\\xb3\\xb4\\xb5\\xb6\\xb7\\xb8\\xb9\\xba\\xbb\\xbc\\xbd\\xbe\\xbf\",\n> + \"x://q/%B0%B1%B2%B3%B4%B5%B6%B7%B8%B9%BA%BB%BC%BD%BE%BF\");\n> + check_normalized_url(\n> +\n> \"x://q/\\xc0\\xc1\\xc2\\xc3\\xc4\\xc5\\xc6\\xc7\\xc8\\xc9\\xca\\xcb\\xcc\\xcd\\xce\\xcf\",\n> + \"x://q/%C0%C1%C2%C3%C4%C5%C6%C7%C8%C9%CA%CB%CC%CD%CE%CF\");\n> + check_normalized_url(\n> +\n> \"x://q/\\xd0\\xd1\\xd2\\xd3\\xd4\\xd5\\xd6\\xd7\\xd8\\xd9\\xda\\xdb\\xdc\\xdd\\xde\\xdf\",\n> + \"x://q/%D0%D1%D2%D3%D4%D5%D6%D7%D8%D9%DA%DB%DC%DD%DE%DF\");\n> + check_normalized_url(\n> +\n> \"x://q/\\xe0\\xe1\\xe2\\xe3\\xe4\\xe5\\xe6\\xe7\\xe8\\xe9\\xea\\xeb\\xec\\xed\\xee\\xef\",\n> + \"x://q/%E0%E1%E2%E3%E4%E5%E6%E7%E8%E9%EA%EB%EC%ED%EE%EF\");\n> + check_normalized_url(\n> +\n> \"x://q/\\xf0\\xf1\\xf2\\xf3\\xf4\\xf5\\xf6\\xf7\\xf8\\xf9\\xfa\\xfb\\xfc\\xfd\\xfe\\xff\",\n> + \"x://q/%F0%F1%F2%F3%F4%F5%F6%F7%F8%F9%FA%FB%FC%FD%FE%FF\");\n> +}\n> +\n> +static void t_url_utf8_escape(void)\n> +{\n> + check_normalized_url(\n> +\n> \"x://q/\\xc2\\x80\\xdf\\xbf\\xe0\\xa0\\x80\\xef\\xbf\\xbd\\xf0\\x90\\x80\\x80\\xf0\\xaf\\xbf\\xbd\",\n> + \"x://q/%C2%80%DF%BF%E0%A0%80%EF%BF%BD%F0%90%80%80%F0%AF%BF%BD\");\n> +}\n> +\n> +static void t_url_username_pass(void)\n> +{\n> + check_normalized_url(\"x://%41%62(^):%70+d@foo\",\n> \"x://Ab(%5E):p+d@foo/\");\n> +}\n> +\n> +static void t_url_length(void)\n> +{\n> + check_normalized_url_length(\"Http://%4d%65:%4d^%70@The.Host\", 25);\n> + check_normalized_url_length(\"http://%41:%42@x.y/%61/\", 17);\n> + check_normalized_url_length(\"http://@x.y/^\", 15);\n> +}\n> +\n> +static void t_url_dots(void)\n> +{\n> + check_normalized_url(\"x://y/.\", \"x://y/\");\n> + check_normalized_url(\"x://y/./\", \"x://y/\");\n> + check_normalized_url(\"x://y/a/.\", \"x://y/a\");\n> + check_normalized_url(\"x://y/a/./\", \"x://y/a/\");\n> + check_normalized_url(\"x://y/.?\", \"x://y/?\");\n> + check_normalized_url(\"x://y/./?\", \"x://y/?\");\n> + check_normalized_url(\"x://y/a/.?\", \"x://y/a?\");\n> + check_normalized_url(\"x://y/a/./?\", \"x://y/a/?\");\n> + check_normalized_url(\"x://y/a/./b/.././../c\", \"x://y/c\");\n> + check_normalized_url(\"x://y/a/./b/../.././c/\", \"x://y/c/\");\n> + check_normalized_url(\"x://y/a/./b/.././../c/././.././.\", \"x://y/\");\n> + check_url_normalizable(\"x://y/a/./b/.././../c/././.././..\", 0);\n> + check_normalized_url(\"x://y/a/./?/././..\", \"x://y/a/?/././..\");\n> + check_normalized_url(\"x://y/%2e/\", \"x://y/\");\n> + check_normalized_url(\"x://y/%2E/\", \"x://y/\");\n> + check_normalized_url(\"x://y/a/%2e./\", \"x://y/\");\n> + check_normalized_url(\"x://y/b/.%2E/\", \"x://y/\");\n> + check_normalized_url(\"x://y/c/%2e%2E/\", \"x://y/\");\n> +}\n> +\n> +/*\n> + * \"http://@foo\" specifies an empty user name but does not specify a\n> password.\n> + * \"http://foo\" specifies neither a user name nor a password.\n> + * So they should not be equivalent.\n> + */\n> +static void t_url_equivalents(void)\n> +{\n> + compare_normalized_urls(\"httP://x\", \"Http://X/\", 1);\n> + compare_normalized_urls(\"Http://%4d%65:%4d^%70@The.Host\",\n> \"hTTP://Me:%4D^p@the.HOST:80/\", 1);\n> + compare_normalized_urls(\"https://@x.y/^\", \"httpS://x.y:443/^\", 0);\n> + compare_normalized_urls(\"https://@x.y/^\", \"httpS://@x.y:0443/^\", 1);\n> + compare_normalized_urls(\"https://@x.y/^/../abc\",\n> \"httpS://@x.y:0443/abc\", 1);\n> + compare_normalized_urls(\"https://@x.y/^/..\", \"httpS://@x.y:0443/\", 1);\n> +}\n> +\n> +int cmd_main(int argc UNUSED, const char **argv UNUSED)\n> +{\n> + TEST(t_url_scheme(), \"url scheme\");\n> + TEST(t_url_authority(), \"url authority\");\n> + TEST(t_url_port(), \"url port checks\");\n> + TEST(t_url_port_normalization(), \"url port normalization\");\n> + TEST(t_url_general_escape(), \"url general escapes\");\n> + TEST(t_url_high_bit(), \"url high-bit escapes\");\n> + TEST(t_url_utf8_escape(), \"url utf8 escapes\");\n> + TEST(t_url_username_pass(), \"url username/password escapes\");\n> + TEST(t_url_length(), \"url normalized lengths\");\n> + TEST(t_url_dots(), \"url . and .. segments\");\n> + TEST(t_url_equivalents(), \"url equivalents\");\n> + return test_done();\n> +}\n>\n> Range-diff against v3:\n> 1: a73b89c8e0 ! 1: ef25954bf8 t: migrate t0110-urlmatch-normalization to\n> the new framework\n> @@ Commit message\n> performance. And also add different test_msg()s for better debugging.\n> \n> In the migration, last two of the checks from `t_url_general_escape()`\n> - were slightly changed compared to the shellscript. This involves\n> changing\n> + were slightly changed compared to the shell script. This involves\n> + changing\n> \n> '\\'' -> '\n> '\\!' -> !\n> \n> in the urls of those checks. This is because in C strings, we don't\n> need to escape \"'\" and \"!\". Other than these two, all the urls were\n> - pasted verbatim from the shellscript.\n> + pasted verbatim from the shell script.\n> \n> - Another change is the removal of MINGW prerequisite from one of the\n> + Another change is the removal of a MINGW prerequisite from one of the\n> test. It was there because[1] on Windows, the command line is a\n> Unicode string, it is not possible to pass arbitrary bytes to a\n> program. But in unit tests we don't have this limitation.\n> @@ t/unit-tests/t-urlmatch-normalization.c (new)\n> +#include \"test-lib.h\"\n> +#include \"urlmatch.h\"\n> +\n> -+static void check_url_normalizable(const char *url, int normalizable)\n> ++static void check_url_normalizable(const char *url, unsigned int\n> normalizable)\n> +{\n> + char *url_norm = url_normalize(url, NULL);\n> +\n> @@ t/unit-tests/t-urlmatch-normalization.c (new)\n> +}\n> +\n> +static void compare_normalized_urls(const char *url1, const char *url2,\n> -+ size_t equal)\n> ++ unsigned int equal)\n> +{\n> + char *url1_norm = url_normalize(url1, NULL);\n> + char *url2_norm = url_normalize(url2, NULL);\n> @@ t/unit-tests/t-urlmatch-normalization.c (new)\n> + test_msg(\"input url1: %s\\n input url2: %s\", url1,\n> + url2);\n> + } else if (!check_int(strcmp(url1_norm, url2_norm), !=, 0)) {\n> -+ test_msg(\" url1_norm: %s\\n url2_norm: %s\\n\"\n> ++ test_msg(\" normalized url1: %s\\n normalized url2: %s\\n\"\n> + \" input url1: %s\\n input url2: %s\",\n> + url1_norm, url2_norm, url1, url2);\n> + }\n> @@ t/unit-tests/t-urlmatch-normalization.c (new)\n> + free(url_norm);\n> +}\n> +\n> -+/* Note that only file: URLs should be allowed without a host */\n> ++/* Note that only \"file:\" URLs should be allowed without a host */\n> +static void t_url_scheme(void)\n> +{\n> + check_url_normalizable(\"\", 0);\n> @@ t/unit-tests/t-urlmatch-normalization.c (new)\n> +}\n> +\n> +/*\n> -+ * http://@foo specifies an empty user name but does not specify a\n> password\n> -+ * http://foo specifies neither a user name nor a password\n> -+ * So they should not be equivalent\n> ++ * \"http://@foo\" specifies an empty user name but does not specify a\n> password.\n> ++ * \"http://foo\" specifies neither a user name nor a password.\n> ++ * So they should not be equivalent.\n> + */\n> +static void t_url_equivalents(void)\n> +{\n> --\n> 2.46.0\n\n"},{"id":"501411","messageId":"CAP8UFD1WJqKUe-SA=w0oGQNgpLdaQNkc5Uh0rZtmCvDD1oQk7A@mail.gmail.com","threadId":"61693","inReplyTo":"20240820152008.21354-2-shyamthakkar001@gmail.com","subject":"Re: [GSoC][PATCH v4] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2024-08-21T10:06:43Z","receivedAt":"2024-08-21T10:06:57Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Tue, Aug 20, 2024 at 5:20 PM Ghanshyam Thakkar\n wrote:\n>\n> helper/test-urlmatch-normalization along with\n> t0110-urlmatch-normalization test the `url_normalize()` function from\n> 'urlmatch.h'. Migrate them to the unit testing framework for better\n> performance. And also add different test_msg()s for better debugging.\n\nThis version addresses all the suggestions (nits actually) to improve\nthe previous version, so it seems to me that it is good to go.\n\nThanks.\n"},{"id":"501441","messageId":"xmqqplq1270h.fsf@gitster.g","threadId":"61693","inReplyTo":"CAP8UFD1WJqKUe-SA=w0oGQNgpLdaQNkc5Uh0rZtmCvDD1oQk7A@mail.gmail.com","subject":"Re: [GSoC][PATCH v4] t: migrate t0110-urlmatch-normalization to the new framework","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-21T16:08:30Z","receivedAt":"2024-08-21T16:08:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Christian Couder writes:\n\n> On Tue, Aug 20, 2024 at 5:20 PM Ghanshyam Thakkar\n> wrote:\n>>\n>> helper/test-urlmatch-normalization along with\n>> t0110-urlmatch-normalization test the `url_normalize()` function from\n>> 'urlmatch.h'. Migrate them to the unit testing framework for better\n>> performance. And also add different test_msg()s for better debugging.\n>\n> This version addresses all the suggestions (nits actually) to improve\n> the previous version, so it seems to me that it is good to go.\n\nLooking good to me, too. Thanks, both.\n\nWill queue. Let me mark it for 'next' in my \"What's cooking\" draft.\n"}]}