{"thread":{"id":"63284","subject":"Bug: Git sometimes disregards wildcards in pathspecs if a file name matches exactly","startedAt":"2025-04-11T19:08:21Z","lastAt":"2025-04-12T09:29:23Z","messageCount":5,"participants":["Piotr Siupa","K Jayatheerth","JAYATHEERTH K","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"516010","messageId":"CAPM0=yBnaXojeC9WkHg08deR-VpjaVQwyrqt8mk+54qLXqSaAQ@mail.gmail.com","threadId":"63284","inReplyTo":null,"subject":"Bug: Git sometimes disregards wildcards in pathspecs if a file name matches exactly","fromName":"Piotr Siupa","fromEmail":"piotrsiupa@gmail.com","sentAt":"2025-04-11T19:08:03Z","receivedAt":"2025-04-11T19:08:21Z","isPatch":false,"sender":{"key":"piotrsiupa@gmail.com","avatar":"https://gravatar.com/avatar/5970da465a147bda2b821a29e966ed328cca93067a42d05cf33ee38aac4acd6f?d=mp&s=160"},"body":"Hi! I think I've found a bug in the command \"git add\".\nIt can be reproduced in a fresh repository by running:\n\ngit init\ntouch 'foo' 'f*'\ngit add 'f*'\n\nThe last command should add both files \"f*\" and \"foo\" to the index but\nit adds only \"f*\".\nRunning it the second time works as expected. (It adds \"foo\" on the\nsecond attempt.)\n\nI'm using Git 2.43.2. The current \"next\" (2.49.0.805.g082f7c87e0)\nseems to have the same behavior if I'm testing it correctly.\n"},{"id":"516025","messageId":"20250412015748.7177-1-jayatheerthkulkarni2005@gmail.com","threadId":"63284","inReplyTo":"CAPM0=yBnaXojeC9WkHg08deR-VpjaVQwyrqt8mk+54qLXqSaAQ@mail.gmail.com","subject":"Re: Bug: Git sometimes disregards wildcards in pathspecs if a file name matches exactly","fromName":"K Jayatheerth","fromEmail":"jayatheerthkulkarni2005@gmail.com","sentAt":"2025-04-12T01:57:48Z","receivedAt":"2025-04-12T01:58:10Z","isPatch":false,"sender":{"key":"jayatheerthkulkarni2005@gmail.com","avatar":"https://avatars.githubusercontent.com/u/148841023?v=4"},"body":"Hi Piotr, Hello everyone,\n\nThanks for the clear bug report, Piotr. I can reproduce the behavior\nyou described in 2.49:\n\nOn Fri, Apr 11, 2025 at 9:08 PM Piotr Siupa <piotrsiupa@gmail.com> wrote:\n>\n> Hi! I think I've found a bug in the command \"git add\".\n> It can be reproduced in a fresh repository by running:\n>\n> git init\n> touch 'foo' 'f*'\n> git add 'f*'\n>\n> The last command should add both files \"f*\" and \"foo\" to the index but\n> it adds only \"f*\".\n> Running it the second time works as expected. (It adds \"foo\" on the\n> second attempt.)\n\nFollowing the code path down from 'cmd_add' (in 'builtin/add.c'),\nthe issue appears to stem from how pathspecs are matched against\ndirectory entries. This happens specifically within the 'prune_directory'\nfunction which uses 'do_match_pathspec' internally (likely called via\n'dir_path_match' -> 'match_pathspec' -> 'match_pathspec_with_flags').\n\nHere's a breakdown of what seems to be happening during that first\n'git add ''f*''' call:\n\nFirst, 'cmd_add' sees it needs to add new files. Then, 'fill_directory'\nfinds both untracked files: 'foo' and the literal 'f*'.\n\nNext, 'prune_directory' is called to filter these using the pathspec ''f*''.\nInside 'prune_directory', the 'do_match_pathspec' function is called for\neach file ('foo', then 'f*', or vice-versa) against the pathspec list\n(which just contains ''f*''). These calls share a common marker array\n(often called 'seen') to track which pathspecs have found a match so far.\n\nWhen 'do_match_pathspec' processes the literal file 'f*' against the\npathspec item ''f*'', it calls 'match_pathspec_item'. This helper function\nlikely returns a code like 'MATCHED_EXACTLY' because the pattern ''f*''\nhappens to exactly match the filename '\"f*\"'. Consequently,\n'do_match_pathspec' updates the 'seen' array for the ''f*'' pathspec to\nmark it as exactly matched. Since a match was found, 'prune_directory'\ndecides to keep the 'f*' entry.\n\nThe problem arises when 'do_match_pathspec' processes the other file, 'foo',\nagainst the same pathspec item ''f*''. Before doing the actual comparison,\nit checks the 'seen' array and finds that the ''f*'' pathspec was already\nmarked 'MATCHED_EXACTLY' (from processing the literal 'f*' file).\nAn optimization check like 'if (seen && seen[i] == MATCHED_EXACTLY)'\nthen evaluates to true. This causes the loop to 'continue', skipping the\ncall to 'match_pathspec_item' entirely for the 'foo' file against the ''f*''\npattern. Because no match was found *in this specific call*, 'do_match_pathspec'\nreturns 0, and 'prune_directory' discards the 'foo' entry.\n\nFinally, 'prune_directory' returns the filtered list, now containing only 'f*',\nand 'add_files' adds only that file to the index.\n\nOn the *second* 'git add ''f*''' call, 'fill_directory' only finds the\nuntracked 'foo'. 'do_match_pathspec' runs with a fresh 'seen' array,\nso the 'MATCHED_EXACTLY' check is initially false. 'match_pathspec_item'\nis called for 'foo', returns 'MATCHED_FNMATCH' (a glob match), and 'foo'\nis correctly added.\n\n> I'm using Git 2.43.2. The current \"next\" (2.49.0.805.g082f7c87e0)\n> seems to have the same behavior if I'm testing it correctly.\n\nYes, the relevant code structures in 'do_match_pathspec' appear similar\nin recent versions, suggesting the behavior is likely consistent.\n\nConclusion:\n\nThe core issue seems to be that optimization check within 'do_match_pathspec':\n\n  // inside do_match_pathspec loop:\n  if (seen && seen[i] == MATCHED_EXACTLY)\n          continue;\n\nThis optimization assumes that once a pathspec item has achieved an\n\"exact\" match against *some* file, it doesn't need to be checked\nagainst *any other* files during the same directory scan operation.\n\nHowever, when a pathspec contains glob characters (like ''f*'') but\nhappens to *also* exactly match a literal filename ('f*'),\n'match_pathspec_item' appears to return 'MATCHED_EXACTLY'. This triggers\nthe optimization, incorrectly preventing the *same* pathspec pattern ''f*''\nfrom matching *other* files (like 'foo') via its intended glob behavior\nduring that initial scan.\n\nA potential fix might involve adjusting the logic in 'match_pathspec_item'\nto perhaps not return 'MATCHED_EXACTLY' if the match involved globbing,\nor modifying the 'seen' check in 'do_match_pathspec' to account for\nthis ambiguity.\n\nThanks again for spotting this subtle behavior!\n\n-Jayatheerth\n"},{"id":"516026","messageId":"CA+rGoLd-bSbaqfswD+qVrCv1fvG69QX+3BO1htbPJtinFkEQAA@mail.gmail.com","threadId":"63284","inReplyTo":"20250412015748.7177-1-jayatheerthkulkarni2005@gmail.com","subject":"Re: Bug: Git sometimes disregards wildcards in pathspecs if a file name matches exactly","fromName":"JAYATHEERTH K","fromEmail":"jayatheerthkulkarni2005@gmail.com","sentAt":"2025-04-12T03:00:00Z","receivedAt":"2025-04-12T03:00:12Z","isPatch":false,"sender":{"key":"jayatheerthkulkarni2005@gmail.com","avatar":"https://avatars.githubusercontent.com/u/148841023?v=4"},"body":"Upon a much closer look into the Git source code, specifically\ninit_pathspec_item within pathspec.c, my previous hypothesis regarding\nthe optimization check in do_match_pathspec appears to be incorrect,\nor at least not the root cause.\n\nThe debugging output confirmed that the core issue seems to originate\nearlier, during the pathspec parsing phase. It revealed that when\nprocessing the input string 'f*' (without any explicit magic like\n:(glob)), the resulting pathspec_item.magic field does not have the ,\neven though the string clearly contains the * glob metacharacter. (To\nclarify I did multiple fprintf checks in the do_match_pathspec fun)\n\nLooking at the logic in init_pathspec_item, it seems the PATHSPEC_GLOB\nflag is generally only added to item->magic if:\na) The user explicitly uses a magic prefix like :(glob)f*.\nb) A global setting like GIT_GLOB_PATHSPECS_ENVIRONMENT is active.\n\nThe code does not appear to automatically set PATHSPEC_GLOB on\nitem->magic simply based on scanning the content of the path string\nfor unescaped *, ?, or [ characters if no explicit magic prefix is\npresent.\n\nThis explains why the fix attempted in do_match_pathspec (which relied\non checking the PATHSPEC_GLOB flag) was ineffective – the flag wasn't\nset in the first place for the 'f*' input. It also explains the\noriginal inconsistent behavior: because the pathspec lacks the\nPATHSPEC_GLOB magic flag, parts of Git relying on these flags treat it\nas literal by default, even if lower-level matching functions might\nstill perform some wildcard expansion based on the string content\nitself.\n\nSo, this seems less like a bug in a specific conditional check and\nmore like a consequence of the current design where the magic flags\nindicating glob behavior are opt-in (via prefixes or environment\nvariables) rather than implicitly inferred from the pathspec string's\ncontent.\n\n-Jayatheerth\n"},{"id":"516029","messageId":"20250412062753.GA3763415@coredump.intra.peff.net","threadId":"63284","inReplyTo":"CA+rGoLd-bSbaqfswD+qVrCv1fvG69QX+3BO1htbPJtinFkEQAA@mail.gmail.com","subject":"Re: Bug: Git sometimes disregards wildcards in pathspecs if a file name matches exactly","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-04-12T06:27:53Z","receivedAt":"2025-04-12T06:28:00Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Apr 12, 2025 at 08:30:00AM +0530, JAYATHEERTH K wrote:\n\n> Upon a much closer look into the Git source code, specifically\n> init_pathspec_item within pathspec.c, my previous hypothesis regarding\n> the optimization check in do_match_pathspec appears to be incorrect,\n> or at least not the root cause.\n\nI think you're still on the right track, but are just mis-interpreting\nthe item->magic field. It's about user-specified \"magic\" flags, one of\nwhich is \"treat this pathspec like a glob, even if the default (or an\nearlier \"literal\" magic flag) would tell you not to do so\".\n\nI don't think there is a bit flag in the pathspec item for \"this item\nhas glob meta characters\". But we compute and record the offset of the\nfirst such character in the \"nowildcard_len\" field (which is for most\ncases much better, since we can optimize prefix matches for stuff like\n\"foo/bar/*\").\n\nAnd you can compare that to \"len\" to see if it does any globbing at all\n(which should also naturally handle stuff like \":(literal)\" because we\nthen mark the whole thing as \"nowildcard\").\n\nSo something like this probably works:\n\ndiff --git a/dir.c b/dir.c\nindex cbd82be6c9..85cc08f4fc 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -519,7 +519,8 @@ static int do_match_pathspec(struct index_state *istate,\n \t\t    ( exclude && !(ps->items[i].magic & PATHSPEC_EXCLUDE)))\n \t\t\tcontinue;\n \n-\t\tif (seen && seen[i] == MATCHED_EXACTLY)\n+\t\tif (seen && seen[i] == MATCHED_EXACTLY &&\n+\t\t    ps->items[i].nowildcard_len == ps->items[i].len)\n \t\t\tcontinue;\n \t\t/*\n \t\t * Make exclude patterns optional and never report\n\nI think if you grep around for 'nowildcard_len ==' you'll find one or\ntwo similar spots.\n\n-Peff\n"},{"id":"516032","messageId":"CA+rGoLe=A61CRNUztHvDpPTUxW1TobS4vknyF3OGTEP0imnwjQ@mail.gmail.com","threadId":"63284","inReplyTo":"20250412062753.GA3763415@coredump.intra.peff.net","subject":"Re: Bug: Git sometimes disregards wildcards in pathspecs if a file name matches exactly","fromName":"JAYATHEERTH K","fromEmail":"jayatheerthkulkarni2005@gmail.com","sentAt":"2025-04-12T09:29:11Z","receivedAt":"2025-04-12T09:29:23Z","isPatch":false,"sender":{"key":"jayatheerthkulkarni2005@gmail.com","avatar":"https://avatars.githubusercontent.com/u/148841023?v=4"},"body":"On Sat, Apr 12, 2025 at 11:57 AM Jeff King <peff@peff.net> wrote:\n>\n> On Sat, Apr 12, 2025 at 08:30:00AM +0530, JAYATHEERTH K wrote:\n>\n> > Upon a much closer look into the Git source code, specifically\n> > init_pathspec_item within pathspec.c, my previous hypothesis regarding\n> > the optimization check in do_match_pathspec appears to be incorrect,\n> > or at least not the root cause.\n>\n> I think you're still on the right track, but are just mis-interpreting\n> the item->magic field. It's about user-specified \"magic\" flags, one of\n> which is \"treat this pathspec like a glob, even if the default (or an\n> earlier \"literal\" magic flag) would tell you not to do so\".\n>\n\nOk that makes a lot of sense I was actually tweaking around with\nhas_wildcard but was confused with the chronology in cmd_add\n\n> I don't think there is a bit flag in the pathspec item for \"this item\n> has glob meta characters\". But we compute and record the offset of the\n> first such character in the \"nowildcard_len\" field (which is for most\n> cases much better, since we can optimize prefix matches for stuff like\n> \"foo/bar/*\").\n>\n> And you can compare that to \"len\" to see if it does any globbing at all\n> (which should also naturally handle stuff like \":(literal)\" because we\n> then mark the whole thing as \"nowildcard\").\n>\n\nUnderstood!! makes sense again.\n\n> So something like this probably works:\n>\n> diff --git a/dir.c b/dir.c\n> index cbd82be6c9..85cc08f4fc 100644\n> --- a/dir.c\n> +++ b/dir.c\n> @@ -519,7 +519,8 @@ static int do_match_pathspec(struct index_state *istate,\n>                     ( exclude && !(ps->items[i].magic & PATHSPEC_EXCLUDE)))\n>                         continue;\n>\n> -               if (seen && seen[i] == MATCHED_EXACTLY)\n> +               if (seen && seen[i] == MATCHED_EXACTLY &&\n> +                   ps->items[i].nowildcard_len == ps->items[i].len)\n>                         continue;\n>                 /*\n>                  * Make exclude patterns optional and never report\n>\n> I think if you grep around for 'nowildcard_len ==' you'll find one or\n> two similar spots.\n>\n\nI did and found this (nowildcard_len) in attr files after tracing it up!\nWill send a patch in a new thread as soon as possible.\n\n\n> -Peff\n\nThank you,\nJayatheerth\n"}]}