{"thread":{"id":"52082","subject":"[PATCH 1/5] Documentation: mention more worktree-specific exceptions","startedAt":"2019-10-21T16:01:01Z","lastAt":"2019-10-28T21:30:48Z","messageCount":14,"participants":["SZEDER Gábor","David Turner","Junio C Hamano","Johannes Schindelin"],"isPatch":true,"patchVersion":1,"patchTotal":5},"messages":[{"id":"384529","messageId":"20191021160043.701-2-szeder.dev@gmail.com","threadId":"52082","inReplyTo":"20191021160043.701-1-szeder.dev@gmail.com","subject":"[PATCH 1/5] Documentation: mention more worktree-specific exceptions","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-10-21T16:00:39Z","receivedAt":"2019-10-21T16:01:01Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"If a directory in $GIT_DIR is overridden when $GIT_COMMON_DIR is set,\nthen usually all paths within that directory are overridden as well.\nThere are a couple of exceptions, though, and two of them, namely\n'refs/rewritten' and 'logs/HEAD' are not mentioned in\n'gitrepository-layout'.  Document them as well.\n\nSigned-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n---\n Documentation/gitrepository-layout.txt | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/gitrepository-layout.txt b/Documentation/gitrepository-layout.txt\nindex d6388f10bb..1a2ef4c150 100644\n--- a/Documentation/gitrepository-layout.txt\n+++ b/Documentation/gitrepository-layout.txt\n@@ -96,9 +96,9 @@ refs::\n \tdirectory.  The 'git prune' command knows to preserve\n \tobjects reachable from refs found in this directory and\n \tits subdirectories.\n-\tThis directory is ignored (except refs/bisect and\n-\trefs/worktree) if $GIT_COMMON_DIR is set and\n-\t\"$GIT_COMMON_DIR/refs\" will be used instead.\n+\tThis directory is ignored (except refs/bisect,\n+\trefs/rewritten and refs/worktree) if $GIT_COMMON_DIR is\n+\tset and \"$GIT_COMMON_DIR/refs\" will be used instead.\n \n refs/heads/`name`::\n \trecords tip-of-the-tree commit objects of branch `name`\n@@ -240,8 +240,8 @@ remotes::\n logs::\n \tRecords of changes made to refs are stored in this directory.\n \tSee linkgit:git-update-ref[1] for more information. This\n-\tdirectory is ignored if $GIT_COMMON_DIR is set and\n-\t\"$GIT_COMMON_DIR/logs\" will be used instead.\n+\tdirectory is ignored (except logs/HEAD) if $GIT_COMMON_DIR is\n+\tset and \"$GIT_COMMON_DIR/logs\" will be used instead.\n \n logs/refs/heads/`name`::\n \tRecords all changes made to the branch tip named `name`.\n-- \n2.24.0.rc0.472.ga6f06c86b4\n\n"},{"id":"384530","messageId":"20191021160043.701-1-szeder.dev@gmail.com","threadId":"52082","inReplyTo":"20191018113557.GA29845@szeder.dev","subject":"[PATCH 0/5] path.c: a couple of common dir/trie fixes","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-10-21T16:00:38Z","receivedAt":"2019-10-21T16:01:01Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Fri, Oct 18, 2019 at 01:35:57PM +0200, SZEDER Gábor wrote:\n> > unfortunately, see two more bugs,\n\nAnd there are documentation bugs as well, both user-visible (i.e. in\na man page) and in in-code comment.\n\n> > and one of them is a \"proper\" bug leading to bogus\n> > output:\n> >\n> >   $ git -C WT/ rev-parse --git-path logs/refs --git-path logs/refs/\n> >   /home/szeder/src/git/.git/logs/refs\n> >   /home/szeder/src/git/.git/worktrees/WT/logs/refs/\n> \n> This one-liner below fixes it, but I haven't yet made up my mind about\n> whether this is the right fix or whether there could be any fallout\n> (at least the test suite doesn't show any).\n> \n>   $ ./git -C WT/ rev-parse --git-path logs/refs --git-path logs/refs/\n>   /home/szeder/src/git/.git/logs/refs\n>   /home/szeder/src/git/.git/logs/refs/\n> \n> \n> diff --git a/path.c b/path.c\n> index 04b69b9feb..9019169418 100644\n> --- a/path.c\n> +++ b/path.c\n> @@ -335,7 +335,7 @@ static int check_common(const char *unmatched, void *value, void *baton)\n>       struct common_dir *dir = value;\n>  \n>       if (!dir)\n> -             return 0;\n> +             return -1;\n>  \n>       if (dir->is_dir && (unmatched[0] == 0 || unmatched[0] == '/'))\n>               return !dir->exclude;\n\nNow I made up mind: this isn't the right fix :)\nThe proper fix is in the last patch of this series.\n\nCc-ing David Turner, the trie's author; if I misunderstood anything,\nthen hopefully he can spot and clarify it.\n\nSZEDER Gábor (5):\n  Documentation: mention more worktree-specific exceptions\n  path.c: clarify trie_find()'s in-code comment\n  path.c: mark 'logs/HEAD' in 'common_list' as file\n  path.c: clarify two field names in 'struct common_dir'\n  path.c: don't call the match function without value in trie_find()\n\n Documentation/gitrepository-layout.txt |  10 +-\n path.c                                 | 122 ++++++++++++++-----------\n t/t0060-path-utils.sh                  |   2 +\n 3 files changed, 74 insertions(+), 60 deletions(-)\n\n-- \n2.24.0.rc0.472.ga6f06c86b4\n\n"},{"id":"384531","messageId":"20191021160043.701-4-szeder.dev@gmail.com","threadId":"52082","inReplyTo":"20191021160043.701-1-szeder.dev@gmail.com","subject":"[PATCH 3/5] path.c: mark 'logs/HEAD' in 'common_list' as file","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-10-21T16:00:41Z","receivedAt":"2019-10-21T16:01:04Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"'logs/HEAD', i.e. HEAD's reflog, is a file, but its entry in\n'common_list' has the 'is_dir' bit set.\n\nUnset that bit to make it consistent with what 'logs/HEAD' is supposed\nto be.\n\nThis doesn't make a difference in behavior: check_common() is the only\nfunction that looks at the 'is_dir' bit, and that function either\nreturns 0, or '!exclude', which for 'logs/HEAD' results in 0 as well.\n\nSigned-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n---\n path.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/path.c b/path.c\nindex 4ac9a101f5..3fac5f5726 100644\n--- a/path.c\n+++ b/path.c\n@@ -113,7 +113,7 @@ static struct common_dir common_list[] = {\n \t{ 0, 1, 0, \"info\" },\n \t{ 0, 0, 1, \"info/sparse-checkout\" },\n \t{ 1, 1, 0, \"logs\" },\n-\t{ 1, 1, 1, \"logs/HEAD\" },\n+\t{ 1, 0, 1, \"logs/HEAD\" },\n \t{ 0, 1, 1, \"logs/refs/bisect\" },\n \t{ 0, 1, 1, \"logs/refs/rewritten\" },\n \t{ 0, 1, 1, \"logs/refs/worktree\" },\n-- \n2.24.0.rc0.472.ga6f06c86b4\n\n"},{"id":"384532","messageId":"20191021160043.701-3-szeder.dev@gmail.com","threadId":"52082","inReplyTo":"20191021160043.701-1-szeder.dev@gmail.com","subject":"[PATCH 2/5] path.c: clarify trie_find()'s in-code comment","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-10-21T16:00:40Z","receivedAt":"2019-10-21T16:01:07Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"A fairly long comment describes trie_find()'s behavior and shows\nexamples, but it's slightly incomplete/inaccurate.  Update this\ncomment to specify how trie_find() handles a negative return value\nfrom the given match function.\n\nFurthermore, update the list of examples to include not only two but\nthree levels of path components.  This makes the examples slightly\nmore complicated, but it can illustrate the behavior in more corner\ncases.\n\nFinally, basically everything refers to the data stored for a key as\n\"value\", with two confusing exceptions:\n\n  - The type definition of the match function calls its corresponding\n    parameter 'data'.\n    Rename that parameter to 'value'.  (check_common(), the only\n    function of this type already calls it 'value').\n\n  - The table of examples above trie_find() has a \"val from node\"\n    column, which has nothing to do with the value stored in the trie:\n    it's a \"prefix of the key for which the trie contains a value\"\n    that led to that node.\n    Rename that column header to \"prefix to node\".\n\nNote that neither the original nor the updated description and\nexamples correspond 100% to the current implementation, because the\nimplementation is a bit buggy, but the comment describes the desired\nbehavior.  The bug will be fixed in the last patch of this series.\n\nSigned-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n---\n path.c | 45 ++++++++++++++++++++++++++++-----------------\n 1 file changed, 28 insertions(+), 17 deletions(-)\n\ndiff --git a/path.c b/path.c\nindex e3da1f3c4e..4ac9a101f5 100644\n--- a/path.c\n+++ b/path.c\n@@ -236,30 +236,41 @@ static void *add_to_trie(struct trie *root, const char *key, void *value)\n \treturn old;\n }\n \n-typedef int (*match_fn)(const char *unmatched, void *data, void *baton);\n+typedef int (*match_fn)(const char *unmatched, void *value, void *baton);\n \n /*\n  * Search a trie for some key.  Find the longest /-or-\\0-terminated\n- * prefix of the key for which the trie contains a value.  Call fn\n- * with the unmatched portion of the key and the found value, and\n- * return its return value.  If there is no such prefix, return -1.\n+ * prefix of the key for which the trie contains a value.  If there is\n+ * no such prefix, return -1.  Otherwise call fn with the unmatched\n+ * portion of the key and the found value.  If fn returns 0 or\n+ * positive, then return its return value.  If fn returns negative,\n+ * then call fn with the next-longest /-terminated prefix of the key\n+ * (i.e. a parent directory) for which the trie contains a value, and\n+ * handle its return value the same way.  If there is no shorter\n+ * /-terminated prefix with a value left, then return the negative\n+ * return value of the most recent fn invocation.\n  *\n  * The key is partially normalized: consecutive slashes are skipped.\n  *\n- * For example, consider the trie containing only [refs,\n- * refs/worktree] (both with values).\n- *\n- * | key             | unmatched  | val from node | return value |\n- * |-----------------|------------|---------------|--------------|\n- * | a               | not called | n/a           | -1           |\n- * | refs            | \\0         | refs          | as per fn    |\n- * | refs/           | /          | refs          | as per fn    |\n- * | refs/w          | /w         | refs          | as per fn    |\n- * | refs/worktree   | \\0         | refs/worktree | as per fn    |\n- * | refs/worktree/  | /          | refs/worktree | as per fn    |\n- * | refs/worktree/a | /a         | refs/worktree | as per fn    |\n- * |-----------------|------------|---------------|--------------|\n+ * For example, consider the trie containing only [logs,\n+ * logs/refs/bisect], both with values, but not logs/refs.\n  *\n+ * | key                | unmatched      | prefix to node   | return value |\n+ * |--------------------|----------------|------------------|--------------|\n+ * | a                  | not called     | n/a              | -1           |\n+ * | logstore           | not called     | n/a              | -1           |\n+ * | logs               | \\0             | logs             | as per fn    |\n+ * | logs/              | /              | logs             | as per fn    |\n+ * | logs/refs          | /refs          | logs             | as per fn    |\n+ * | logs/refs/         | /refs/         | logs             | as per fn    |\n+ * | logs/refs/b        | /refs/b        | logs             | as per fn    |\n+ * | logs/refs/bisected | /refs/bisected | logs             | as per fn    |\n+ * | logs/refs/bisect   | \\0             | logs/refs/bisect | as per fn    |\n+ * | logs/refs/bisect/  | /              | logs/refs/bisect | as per fn    |\n+ * | logs/refs/bisect/a | /a             | logs/refs/bisect | as per fn    |\n+ * | (If fn in the previous line returns -1, then fn is called once more:) |\n+ * | logs/refs/bisect/a | /refs/bisect/a | logs             | as per fn    |\n+ * |--------------------|----------------|------------------|--------------|\n  */\n static int trie_find(struct trie *root, const char *key, match_fn fn,\n \t\t     void *baton)\n-- \n2.24.0.rc0.472.ga6f06c86b4\n\n"},{"id":"384533","messageId":"20191021160043.701-5-szeder.dev@gmail.com","threadId":"52082","inReplyTo":"20191021160043.701-1-szeder.dev@gmail.com","subject":"[PATCH 4/5] path.c: clarify two field names in 'struct common_dir'","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-10-21T16:00:42Z","receivedAt":"2019-10-21T16:01:09Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"An array of 'struct common_dir' instances is used to specify whether\nvarious paths in $GIT_DIR are specific to a worktree, or are common,\ni.e. belong to main worktree.  The names of two fields in this\nstruct are somewhat confusing or ambigious:\n\n  - The path is recorded in the struct's 'dirname' field, even though\n    several entries are regular files e.g. 'gc.pid', 'packed-refs',\n    etc.\n\n    Rename this field to 'path' to reduce confusion.\n\n  - The field 'exclude' tells whether the path is excluded...  from\n    where?  Excluded from the common dir or from the worktree?  It\n    means the former, but it's ambigious.\n\n    Rename this field to 'is_common' to make it unambigious what it\n    means.  This, however, means the exact opposite of what 'exclude'\n    meant, so we have to negate the field's value in all entries as\n    well.\n\nThe diff is best viewed with '--color-words'.\n\nSigned-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n---\n path.c | 66 +++++++++++++++++++++++++++++-----------------------------\n 1 file changed, 33 insertions(+), 33 deletions(-)\n\ndiff --git a/path.c b/path.c\nindex 3fac5f5726..cf57bd52dd 100644\n--- a/path.c\n+++ b/path.c\n@@ -101,36 +101,36 @@ struct common_dir {\n \t/* Not considered garbage for report_linked_checkout_garbage */\n \tunsigned ignore_garbage:1;\n \tunsigned is_dir:1;\n-\t/* Not common even though its parent is */\n-\tunsigned exclude:1;\n-\tconst char *dirname;\n+\t/* Belongs to the common dir, though it may contain paths that don't */\n+\tunsigned is_common:1;\n+\tconst char *path;\n };\n \n static struct common_dir common_list[] = {\n-\t{ 0, 1, 0, \"branches\" },\n-\t{ 0, 1, 0, \"common\" },\n-\t{ 0, 1, 0, \"hooks\" },\n-\t{ 0, 1, 0, \"info\" },\n-\t{ 0, 0, 1, \"info/sparse-checkout\" },\n-\t{ 1, 1, 0, \"logs\" },\n-\t{ 1, 0, 1, \"logs/HEAD\" },\n-\t{ 0, 1, 1, \"logs/refs/bisect\" },\n-\t{ 0, 1, 1, \"logs/refs/rewritten\" },\n-\t{ 0, 1, 1, \"logs/refs/worktree\" },\n-\t{ 0, 1, 0, \"lost-found\" },\n-\t{ 0, 1, 0, \"objects\" },\n-\t{ 0, 1, 0, \"refs\" },\n-\t{ 0, 1, 1, \"refs/bisect\" },\n-\t{ 0, 1, 1, \"refs/rewritten\" },\n-\t{ 0, 1, 1, \"refs/worktree\" },\n-\t{ 0, 1, 0, \"remotes\" },\n-\t{ 0, 1, 0, \"worktrees\" },\n-\t{ 0, 1, 0, \"rr-cache\" },\n-\t{ 0, 1, 0, \"svn\" },\n-\t{ 0, 0, 0, \"config\" },\n-\t{ 1, 0, 0, \"gc.pid\" },\n-\t{ 0, 0, 0, \"packed-refs\" },\n-\t{ 0, 0, 0, \"shallow\" },\n+\t{ 0, 1, 1, \"branches\" },\n+\t{ 0, 1, 1, \"common\" },\n+\t{ 0, 1, 1, \"hooks\" },\n+\t{ 0, 1, 1, \"info\" },\n+\t{ 0, 0, 0, \"info/sparse-checkout\" },\n+\t{ 1, 1, 1, \"logs\" },\n+\t{ 1, 0, 0, \"logs/HEAD\" },\n+\t{ 0, 1, 0, \"logs/refs/bisect\" },\n+\t{ 0, 1, 0, \"logs/refs/rewritten\" },\n+\t{ 0, 1, 0, \"logs/refs/worktree\" },\n+\t{ 0, 1, 1, \"lost-found\" },\n+\t{ 0, 1, 1, \"objects\" },\n+\t{ 0, 1, 1, \"refs\" },\n+\t{ 0, 1, 0, \"refs/bisect\" },\n+\t{ 0, 1, 0, \"refs/rewritten\" },\n+\t{ 0, 1, 0, \"refs/worktree\" },\n+\t{ 0, 1, 1, \"remotes\" },\n+\t{ 0, 1, 1, \"worktrees\" },\n+\t{ 0, 1, 1, \"rr-cache\" },\n+\t{ 0, 1, 1, \"svn\" },\n+\t{ 0, 0, 1, \"config\" },\n+\t{ 1, 0, 1, \"gc.pid\" },\n+\t{ 0, 0, 1, \"packed-refs\" },\n+\t{ 0, 0, 1, \"shallow\" },\n \t{ 0, 0, 0, NULL }\n };\n \n@@ -331,8 +331,8 @@ static void init_common_trie(void)\n \tif (common_trie_done_setup)\n \t\treturn;\n \n-\tfor (p = common_list; p->dirname; p++)\n-\t\tadd_to_trie(&common_trie, p->dirname, p);\n+\tfor (p = common_list; p->path; p++)\n+\t\tadd_to_trie(&common_trie, p->path, p);\n \n \tcommon_trie_done_setup = 1;\n }\n@@ -349,10 +349,10 @@ static int check_common(const char *unmatched, void *value, void *baton)\n \t\treturn 0;\n \n \tif (dir->is_dir && (unmatched[0] == 0 || unmatched[0] == '/'))\n-\t\treturn !dir->exclude;\n+\t\treturn dir->is_common;\n \n \tif (!dir->is_dir && unmatched[0] == 0)\n-\t\treturn !dir->exclude;\n+\t\treturn dir->is_common;\n \n \treturn 0;\n }\n@@ -376,8 +376,8 @@ void report_linked_checkout_garbage(void)\n \t\treturn;\n \tstrbuf_addf(&sb, \"%s/\", get_git_dir());\n \tlen = sb.len;\n-\tfor (p = common_list; p->dirname; p++) {\n-\t\tconst char *path = p->dirname;\n+\tfor (p = common_list; p->path; p++) {\n+\t\tconst char *path = p->path;\n \t\tif (p->ignore_garbage)\n \t\t\tcontinue;\n \t\tstrbuf_setlen(&sb, len);\n-- \n2.24.0.rc0.472.ga6f06c86b4\n\n"},{"id":"384534","messageId":"20191021160043.701-6-szeder.dev@gmail.com","threadId":"52082","inReplyTo":"20191021160043.701-1-szeder.dev@gmail.com","subject":"[PATCH 5/5] path.c: don't call the match function without value in trie_find()","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-10-21T16:00:43Z","receivedAt":"2019-10-21T16:01:10Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"'logs/refs' is not a working tree-specific path, but since commit\nb9317d55a3 (Make sure refs/rewritten/ is per-worktree, 2019-03-07)\n'git rev-parse --git-path' has been returning a bogus path if a\ntrailing '/' is present:\n\n  $ git -C WT/ rev-parse --git-path logs/refs --git-path logs/refs/\n  /home/szeder/src/git/.git/logs/refs\n  /home/szeder/src/git/.git/worktrees/WT/logs/refs/\n\nWe use a trie data structure to efficiently decide whether a path\nbelongs to the common dir or is working tree-specific.  As it happens\nb9317d55a3 triggered a bug that is as old as the trie implementation\nitself, added in 4e09cf2acf (path: optimize common dir checking,\n2015-08-31).\n\n  - According to the comment describing trie_find(), it should only\n    call the given match function 'fn' for a \"/-or-\\0-terminated\n    prefix of the key for which the trie contains a value\".  This is\n    not true: there are three places where trie_find() calls the match\n    function, but one of them is missing the check for value's\n    existence.\n\n  - b9317d55a3 added two new keys to the trie: 'logs/refs/rewritten'\n    and 'logs/refs/worktree', next to the already existing\n    'logs/refs/bisect'.  This resulted in a trie node with the path\n    'logs/refs', which didn't exist before, and which doesn't have a\n    value attached.  A query for 'logs/refs/' finds this node and then\n    hits that one callsite of the match function which doesn't check\n    for the value's existence, and thus invokes the match function\n    with NULL as value.\n\n  - When the match function check_common() is invoked with a NULL\n    value, it returns 0, which indicates that the queried path doesn't\n    belong to the common directory, ultimately resulting the bogus\n    path shown above.\n\nAdd the missing condition to trie_find() so it will never invoke the\nmatch function with a non-existing value.  check_common() will then no\nlonger have to check that it got a non-NULL value, so remove that\ncondition.\n\nI believe that there are no other paths that could cause similar bogus\noutput.  AFAICT the only other key resulting in the match function\nbeing called with a NULL value is 'co' (because of the keys 'common'\nand 'config').  However, as they are not in a directory that belongs\nto the common directory the resulting working tree-specific path is\nexpected.\n\nSigned-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n---\n path.c                | 11 ++++++-----\n t/t0060-path-utils.sh |  2 ++\n 2 files changed, 8 insertions(+), 5 deletions(-)\n\ndiff --git a/path.c b/path.c\nindex cf57bd52dd..e21b00c4d4 100644\n--- a/path.c\n+++ b/path.c\n@@ -299,9 +299,13 @@ static int trie_find(struct trie *root, const char *key, match_fn fn,\n \n \t/* Matched the entire compressed section */\n \tkey += i;\n-\tif (!*key)\n+\tif (!*key) {\n \t\t/* End of key */\n-\t\treturn fn(key, root->value, baton);\n+\t\tif (root->value)\n+\t\t\treturn fn(key, root->value, baton);\n+\t\telse\n+\t\t\treturn -1;\n+\t}\n \n \t/* Partial path normalization: skip consecutive slashes */\n \twhile (key[0] == '/' && key[1] == '/')\n@@ -345,9 +349,6 @@ static int check_common(const char *unmatched, void *value, void *baton)\n {\n \tstruct common_dir *dir = value;\n \n-\tif (!dir)\n-\t\treturn 0;\n-\n \tif (dir->is_dir && (unmatched[0] == 0 || unmatched[0] == '/'))\n \t\treturn dir->is_common;\n \ndiff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh\nindex c7b53e494b..501e1a288d 100755\n--- a/t/t0060-path-utils.sh\n+++ b/t/t0060-path-utils.sh\n@@ -288,6 +288,8 @@ test_git_path GIT_COMMON_DIR=bar index                    .git/index\n test_git_path GIT_COMMON_DIR=bar HEAD                     .git/HEAD\n test_git_path GIT_COMMON_DIR=bar logs/HEAD                .git/logs/HEAD\n test_git_path GIT_COMMON_DIR=bar logs/refs/bisect/foo     .git/logs/refs/bisect/foo\n+test_git_path GIT_COMMON_DIR=bar logs/refs                bar/logs/refs\n+test_git_path GIT_COMMON_DIR=bar logs/refs/               bar/logs/refs/\n test_git_path GIT_COMMON_DIR=bar logs/refs/bisec/foo      bar/logs/refs/bisec/foo\n test_git_path GIT_COMMON_DIR=bar logs/refs/bisec          bar/logs/refs/bisec\n test_git_path GIT_COMMON_DIR=bar logs/refs/bisectfoo      bar/logs/refs/bisectfoo\n-- \n2.24.0.rc0.472.ga6f06c86b4\n\n"},{"id":"384537","messageId":"0f62325e46901322346184b47329940f7700a1e3.camel@novalis.org","threadId":"52082","inReplyTo":"20191021160043.701-6-szeder.dev@gmail.com","subject":"Re: [PATCH 5/5] path.c: don't call the match function without value in trie_find()","fromName":"David Turner","fromEmail":"novalis@novalis.org","sentAt":"2019-10-21T17:39:53Z","receivedAt":"2019-10-21T17:40:02Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Mon, 2019-10-21 at 18:00 +0200, SZEDER Gábor wrote:\n> Add the missing condition to trie_find() so it will never invoke the\n> match function with a non-existing value.  check_common() will then\n> no\n> longer have to check that it got a non-NULL value, so remove that\n> condition.\n...\n>  \n>  \t/* Partial path normalization: skip consecutive slashes */\n>  \twhile (key[0] == '/' && key[1] == '/')\n> @@ -345,9 +349,6 @@ static int check_common(const char *unmatched,\n> void *value, void *baton)\n>  {\n>  \tstruct common_dir *dir = value;\n>  \n> -\tif (!dir)\n> -\t\treturn 0;\n\n\nDo we want to assert(dir) here?\n\nOverall, LGTM.  Thanks for the clean-up.\n\n"},{"id":"384566","messageId":"20191021205703.GB4348@szeder.dev","threadId":"52082","inReplyTo":"20191021160043.701-6-szeder.dev@gmail.com","subject":"Re: [PATCH 5/5] path.c: don't call the match function without value in trie_find()","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-10-21T20:57:03Z","receivedAt":"2019-10-21T20:57:09Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Mon, Oct 21, 2019 at 06:00:43PM +0200, SZEDER Gábor wrote:\n> 'logs/refs' is not a working tree-specific path, but since commit\n> b9317d55a3 (Make sure refs/rewritten/ is per-worktree, 2019-03-07)\n> 'git rev-parse --git-path' has been returning a bogus path if a\n> trailing '/' is present:\n> \n>   $ git -C WT/ rev-parse --git-path logs/refs --git-path logs/refs/\n>   /home/szeder/src/git/.git/logs/refs\n>   /home/szeder/src/git/.git/worktrees/WT/logs/refs/\n> \n> We use a trie data structure to efficiently decide whether a path\n> belongs to the common dir or is working tree-specific.  As it happens\n> b9317d55a3 triggered a bug that is as old as the trie implementation\n> itself, added in 4e09cf2acf (path: optimize common dir checking,\n> 2015-08-31).\n> \n>   - According to the comment describing trie_find(), it should only\n>     call the given match function 'fn' for a \"/-or-\\0-terminated\n>     prefix of the key for which the trie contains a value\".  This is\n>     not true: there are three places where trie_find() calls the match\n>     function, but one of them is missing the check for value's\n>     existence.\n> \n>   - b9317d55a3 added two new keys to the trie: 'logs/refs/rewritten'\n>     and 'logs/refs/worktree', next to the already existing\n>     'logs/refs/bisect'.  This resulted in a trie node with the path\n>     'logs/refs', which didn't exist before, and which doesn't have a\n\nOops, I missed the trailing slash, that must be 'logs/refs/'!\n\n>     value attached.  A query for 'logs/refs/' finds this node and then\n>     hits that one callsite of the match function which doesn't check\n>     for the value's existence, and thus invokes the match function\n>     with NULL as value.\n"},{"id":"384671","messageId":"xmqqa79si003.fsf@gitster-ct.c.googlers.com","threadId":"52082","inReplyTo":"20191021205703.GB4348@szeder.dev","subject":"Re: [PATCH 5/5] path.c: don't call the match function without value in trie_find()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-10-23T04:01:00Z","receivedAt":"2019-10-23T04:01:08Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"SZEDER Gábor <szeder.dev@gmail.com> writes:\n\n>>   - b9317d55a3 added two new keys to the trie: 'logs/refs/rewritten'\n>>     and 'logs/refs/worktree', next to the already existing\n>>     'logs/refs/bisect'.  This resulted in a trie node with the path\n>>     'logs/refs', which didn't exist before, and which doesn't have a\n>\n> Oops, I missed the trailing slash, that must be 'logs/refs/'!\n>\n>>     value attached.  A query for 'logs/refs/' finds this node and then\n>>     hits that one callsite of the match function which doesn't check\n>>     for the value's existence, and thus invokes the match function\n>>     with NULL as value.\n\nGiven that the trie is maintained by hand in common_list[], I wonder\nif we can mechanically catch errors like the one b9317d55a3 added,\nby perhaps having a self-test function that a t/helper/ program\ncalls to perform consistency check after the \"git\" gets built.\n\nThanks.\n\n\n\n"},{"id":"384711","messageId":"20191023162048.GI4348@szeder.dev","threadId":"52082","inReplyTo":"xmqqa79si003.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 5/5] path.c: don't call the match function without value in trie_find()","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-10-23T16:20:48Z","receivedAt":"2019-10-23T16:20:54Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Wed, Oct 23, 2019 at 01:01:00PM +0900, Junio C Hamano wrote:\n> SZEDER Gábor <szeder.dev@gmail.com> writes:\n> \n> >>   - b9317d55a3 added two new keys to the trie: 'logs/refs/rewritten'\n> >>     and 'logs/refs/worktree', next to the already existing\n> >>     'logs/refs/bisect'.  This resulted in a trie node with the path\n> >>     'logs/refs', which didn't exist before, and which doesn't have a\n> >\n> > Oops, I missed the trailing slash, that must be 'logs/refs/'!\n> >\n> >>     value attached.  A query for 'logs/refs/' finds this node and then\n> >>     hits that one callsite of the match function which doesn't check\n> >>     for the value's existence, and thus invokes the match function\n> >>     with NULL as value.\n> \n> Given that the trie is maintained by hand in common_list[], I wonder\n> if we can mechanically catch errors like the one b9317d55a3 added,\n> by perhaps having a self-test function that a t/helper/ program\n> calls to perform consistency check after the \"git\" gets built.\n\nI'm not sure what you mean by \"consistency check\".  The resulting trie\nlooked as expected both before and after b9317d55a3, i.e. each trie\nnode had the right contents, value, and children, as far as I could\ntell.  The issue was in the lookup function.\n\n"},{"id":"384742","messageId":"xmqq5zkehld3.fsf@gitster-ct.c.googlers.com","threadId":"52082","inReplyTo":"20191023162048.GI4348@szeder.dev","subject":"Re: [PATCH 5/5] path.c: don't call the match function without value in trie_find()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-10-24T03:29:28Z","receivedAt":"2019-10-24T03:29:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"SZEDER Gábor <szeder.dev@gmail.com> writes:\n\n> I'm not sure what you mean by \"consistency check\".  The resulting trie\n> looked as expected both before and after b9317d55a3, i.e. each trie\n> node had the right contents, value, and children, as far as I could\n> tell.  The issue was in the lookup function.\n\nI saw the change to the code as a band-aid, which wouldn't have been\nnecessary if we had the missing refs/ entry.  Fully populating the\nleading levels explicitly may give the application more flexibility\nby yielding results different from hardcoded -1 like the patched\ncode gives.  \n\nBut perhaps treating a path that matches a missing intermediate\nlevel just like a path that did not match anything in the trie is\nwhat we want anyway, so I guess the code change was the right thing\n(as opposed to a band-aid).\n\nThanks.\n\n"},{"id":"384974","messageId":"nycvar.QRO.7.76.6.1910281155220.46@tvgsbejvaqbjf.bet","threadId":"52082","inReplyTo":"20191021160043.701-6-szeder.dev@gmail.com","subject":"Re: [PATCH 5/5] path.c: don't call the match function without value in trie_find()","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-28T10:57:10Z","receivedAt":"2019-10-28T10:57:39Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Gábor,\n\nOn Mon, 21 Oct 2019, SZEDER Gábor wrote:\n\n> 'logs/refs' is not a working tree-specific path, but since commit\n> b9317d55a3 (Make sure refs/rewritten/ is per-worktree, 2019-03-07)\n> 'git rev-parse --git-path' has been returning a bogus path if a\n> trailing '/' is present:\n>\n>   $ git -C WT/ rev-parse --git-path logs/refs --git-path logs/refs/\n>   /home/szeder/src/git/.git/logs/refs\n>   /home/szeder/src/git/.git/worktrees/WT/logs/refs/\n>\n> We use a trie data structure to efficiently decide whether a path\n> belongs to the common dir or is working tree-specific.  As it happens\n> b9317d55a3 triggered a bug that is as old as the trie implementation\n> itself, added in 4e09cf2acf (path: optimize common dir checking,\n> 2015-08-31).\n>\n>   - According to the comment describing trie_find(), it should only\n>     call the given match function 'fn' for a \"/-or-\\0-terminated\n>     prefix of the key for which the trie contains a value\".  This is\n>     not true: there are three places where trie_find() calls the match\n>     function, but one of them is missing the check for value's\n>     existence.\n>\n>   - b9317d55a3 added two new keys to the trie: 'logs/refs/rewritten'\n>     and 'logs/refs/worktree', next to the already existing\n>     'logs/refs/bisect'.  This resulted in a trie node with the path\n>     'logs/refs', which didn't exist before, and which doesn't have a\n>     value attached.  A query for 'logs/refs/' finds this node and then\n>     hits that one callsite of the match function which doesn't check\n>     for the value's existence, and thus invokes the match function\n>     with NULL as value.\n>\n>   - When the match function check_common() is invoked with a NULL\n>     value, it returns 0, which indicates that the queried path doesn't\n>     belong to the common directory, ultimately resulting the bogus\n>     path shown above.\n>\n> Add the missing condition to trie_find() so it will never invoke the\n> match function with a non-existing value.  check_common() will then no\n> longer have to check that it got a non-NULL value, so remove that\n> condition.\n>\n> I believe that there are no other paths that could cause similar bogus\n> output.  AFAICT the only other key resulting in the match function\n> being called with a NULL value is 'co' (because of the keys 'common'\n> and 'config').  However, as they are not in a directory that belongs\n> to the common directory the resulting working tree-specific path is\n> expected.\n>\n> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n\nThank you for this entire patch series. Just one nit:\n\n\n> diff --git a/path.c b/path.c\n> index cf57bd52dd..e21b00c4d4 100644\n> --- a/path.c\n> +++ b/path.c\n> @@ -299,9 +299,13 @@ static int trie_find(struct trie *root, const char *key, match_fn fn,\n>\n>  \t/* Matched the entire compressed section */\n>  \tkey += i;\n> -\tif (!*key)\n> +\tif (!*key) {\n>  \t\t/* End of key */\n> -\t\treturn fn(key, root->value, baton);\n> +\t\tif (root->value)\n> +\t\t\treturn fn(key, root->value, baton);\n> +\t\telse\n> +\t\t\treturn -1;\n\nI would have preferred this:\n\n+\t\tif (!root->value)\n+\t\t\treturn -1;\n+\t\treturn fn(key, root->value, baton);\n\n... as it would more accurately reflect my mental model of an \"early\nout\".\n\nBut as I said, this is just a nit-pick.\n\nThank you for working on this!\nDscho\n\n> +\t}\n>\n>  \t/* Partial path normalization: skip consecutive slashes */\n>  \twhile (key[0] == '/' && key[1] == '/')\n> @@ -345,9 +349,6 @@ static int check_common(const char *unmatched, void *value, void *baton)\n>  {\n>  \tstruct common_dir *dir = value;\n>\n> -\tif (!dir)\n> -\t\treturn 0;\n> -\n>  \tif (dir->is_dir && (unmatched[0] == 0 || unmatched[0] == '/'))\n>  \t\treturn dir->is_common;\n>\n> diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh\n> index c7b53e494b..501e1a288d 100755\n> --- a/t/t0060-path-utils.sh\n> +++ b/t/t0060-path-utils.sh\n> @@ -288,6 +288,8 @@ test_git_path GIT_COMMON_DIR=bar index                    .git/index\n>  test_git_path GIT_COMMON_DIR=bar HEAD                     .git/HEAD\n>  test_git_path GIT_COMMON_DIR=bar logs/HEAD                .git/logs/HEAD\n>  test_git_path GIT_COMMON_DIR=bar logs/refs/bisect/foo     .git/logs/refs/bisect/foo\n> +test_git_path GIT_COMMON_DIR=bar logs/refs                bar/logs/refs\n> +test_git_path GIT_COMMON_DIR=bar logs/refs/               bar/logs/refs/\n>  test_git_path GIT_COMMON_DIR=bar logs/refs/bisec/foo      bar/logs/refs/bisec/foo\n>  test_git_path GIT_COMMON_DIR=bar logs/refs/bisec          bar/logs/refs/bisec\n>  test_git_path GIT_COMMON_DIR=bar logs/refs/bisectfoo      bar/logs/refs/bisectfoo\n> --\n> 2.24.0.rc0.472.ga6f06c86b4\n>\n>\n"},{"id":"384981","messageId":"20191028120054.GS4348@szeder.dev","threadId":"52082","inReplyTo":"nycvar.QRO.7.76.6.1910281155220.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH 5/5] path.c: don't call the match function without value in trie_find()","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-10-28T12:00:54Z","receivedAt":"2019-10-28T12:01:01Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Mon, Oct 28, 2019 at 11:57:10AM +0100, Johannes Schindelin wrote:\n> >   - According to the comment describing trie_find(), it should only\n> >     call the given match function 'fn' for a \"/-or-\\0-terminated\n> >     prefix of the key for which the trie contains a value\".  This is\n> >     not true: there are three places where trie_find() calls the match\n> >     function, but one of them is missing the check for value's\n> >     existence.\n\n> Thank you for this entire patch series. Just one nit:\n> \n> \n> > diff --git a/path.c b/path.c\n> > index cf57bd52dd..e21b00c4d4 100644\n> > --- a/path.c\n> > +++ b/path.c\n> > @@ -299,9 +299,13 @@ static int trie_find(struct trie *root, const char *key, match_fn fn,\n> >\n> >  \t/* Matched the entire compressed section */\n> >  \tkey += i;\n> > -\tif (!*key)\n> > +\tif (!*key) {\n> >  \t\t/* End of key */\n> > -\t\treturn fn(key, root->value, baton);\n> > +\t\tif (root->value)\n> > +\t\t\treturn fn(key, root->value, baton);\n> > +\t\telse\n> > +\t\t\treturn -1;\n> \n> I would have preferred this:\n> \n> +\t\tif (!root->value)\n> +\t\t\treturn -1;\n> +\t\treturn fn(key, root->value, baton);\n> \n> ... as it would more accurately reflect my mental model of an \"early\n> out\".\n\nThe checks at the other two of those three callsites look like this,\nand I just followed suit for the sake of consistency.\n\n"},{"id":"385015","messageId":"nycvar.QRO.7.76.6.1910282229480.46@tvgsbejvaqbjf.bet","threadId":"52082","inReplyTo":"20191028120054.GS4348@szeder.dev","subject":"Re: [PATCH 5/5] path.c: don't call the match function without value in trie_find()","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-10-28T21:30:17Z","receivedAt":"2019-10-28T21:30:48Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Gábor,\n\nOn Mon, 28 Oct 2019, SZEDER Gábor wrote:\n\n> On Mon, Oct 28, 2019 at 11:57:10AM +0100, Johannes Schindelin wrote:\n> > >   - According to the comment describing trie_find(), it should only\n> > >     call the given match function 'fn' for a \"/-or-\\0-terminated\n> > >     prefix of the key for which the trie contains a value\".  This is\n> > >     not true: there are three places where trie_find() calls the match\n> > >     function, but one of them is missing the check for value's\n> > >     existence.\n>\n> > Thank you for this entire patch series. Just one nit:\n> >\n> >\n> > > diff --git a/path.c b/path.c\n> > > index cf57bd52dd..e21b00c4d4 100644\n> > > --- a/path.c\n> > > +++ b/path.c\n> > > @@ -299,9 +299,13 @@ static int trie_find(struct trie *root, const char *key, match_fn fn,\n> > >\n> > >  \t/* Matched the entire compressed section */\n> > >  \tkey += i;\n> > > -\tif (!*key)\n> > > +\tif (!*key) {\n> > >  \t\t/* End of key */\n> > > -\t\treturn fn(key, root->value, baton);\n> > > +\t\tif (root->value)\n> > > +\t\t\treturn fn(key, root->value, baton);\n> > > +\t\telse\n> > > +\t\t\treturn -1;\n> >\n> > I would have preferred this:\n> >\n> > +\t\tif (!root->value)\n> > +\t\t\treturn -1;\n> > +\t\treturn fn(key, root->value, baton);\n> >\n> > ... as it would more accurately reflect my mental model of an \"early\n> > out\".\n>\n> The checks at the other two of those three callsites look like this,\n> and I just followed suit for the sake of consistency.\n\nOh, okay. Sorry for the noise, then.\n\nThanks,\nDscho\n"}]}