{"thread":{"id":"17730","subject":"How to use path limiting (using a glob)?","startedAt":"2009-02-11T19:14:32Z","lastAt":"2009-02-12T11:09:36Z","messageCount":6,"participants":["Peter Baumann","Linus Torvalds","Junio C Hamano","Nanako Shiraishi","Sitaram Chamarty"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"104274","messageId":"20090211191432.GC27232@m62s10.vlinux.de","threadId":"17730","inReplyTo":null,"subject":"How to use path limiting (using a glob)?","fromName":"Peter Baumann","fromEmail":"waste.manager@gmx.de","sentAt":"2009-02-11T19:14:32Z","receivedAt":"2009-02-11T19:14:32Z","isPatch":false,"sender":{"key":"waste.manager@gmx.de","avatar":null},"body":"Hallo,\n\nafter reading Junio's nice blog today where he explained how to use git grep\nefficiently, I saw him using a glob to match for the interesting files:\n\n\t $ git grep -e ';;' -- '*.c'\n\nIs it possible to have the same feature in git diff and the revision\nmachinery? Because I tried\n\n\t$ cd $path_to_your_git_src_dir\n\t$ git log master -p -- '*.h'\n\t.... No commit shown \n\n\t$ git diff --name-only v1.5.0  v1.6.0 -- '*.c'\n\nand both don't return anything.\n\nGrettings,\nPeter Baumann\n"},{"id":"104284","messageId":"alpine.LFD.2.00.0902111129190.3590@localhost.localdomain","threadId":"17730","inReplyTo":"20090211191432.GC27232@m62s10.vlinux.de","subject":"Re: How to use path limiting (using a glob)?","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2009-02-11T19:40:44Z","receivedAt":"2009-02-11T19:40:44Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 11 Feb 2009, Peter Baumann wrote:\n\n> after reading Junio's nice blog today where he explained how to use git grep\n> efficiently, I saw him using a glob to match for the interesting files:\n> \n> \t $ git grep -e ';;' -- '*.c'\n> \n> Is it possible to have the same feature in git diff and the revision\n> machinery?\n\nNot really. Git has two different kinds of path limiters, and they are \nreally really different.\n\n - the \"walk current index/directory recursively\" kind that \"git ls-files\" \n   uses, which takes a 'fnmatch()' type path regexp (not a real regexp, \n   but the kind you're used to with shell)\n\n   NOTE! On purpose, we don't set the FNM_PATHNAME, so \"*.c\" here is \n   different from *.c in shell (it's more like \"**.c\" in tcsh). IOW, * \n   matches '/' too, and will walk subdirectories.\n\n - the \"revision limiter\" pathspec. This is *not* a regexp, it's a pure \n   prefix matcher, for a very simple reason: performance.\n\n> \t$ cd $path_to_your_git_src_dir\n> \t$ git log master -p -- '*.h'\n> \t.... No commit shown \n> \n> \t$ git diff --name-only v1.5.0  v1.6.0 -- '*.c'\n> \n> and both don't return anything.\n\nYeah, in the revision matcher you can still depend on the shell \nexpansion, and it will do _almost_ the right thing. So if you do\n\n\tgit log master -p *.c\n\nwithout the quotes, the shell expansion will work, and that in turn will \ngive a set of filenames that \"git log\" will restrict the log to. HOWEVER, \nit's not a real wildcard - it's literally looking at what you have now in \nyour current working directory, and saying \"give me the logs of those \npathnames\", not \"give me the logs of everything ending with .c\".\n\nWe _could_ make the revision limiter understand fnmatch-style patterns, \nbut quite frankly, it's very very expensive - too expensive to be useful \nfor big repositories. The point about only matching prefixes is that it \nallows the revision limiter to not even walk into subdirectories that \ndon't match, but if you do the \"*.c\" kind of pattern, now the revision \ncode has to look up every tree recursively. That code is also _extremely_ \nperformance-critical, so we really don't want to use fnmatch() when we can \ncurrently use just \"memcmp()\".\n\nSo yes, it's kind of odd how we have two totally different concepts of \npathname patterns, but it's probably easiest to remember that \"'git grep' \nis just special\". \n\n\t\tLinus\n"},{"id":"104286","messageId":"7v1vu4eouz.fsf@gitster.siamese.dyndns.org","threadId":"17730","inReplyTo":"20090211191432.GC27232@m62s10.vlinux.de","subject":"Re: How to use path limiting (using a glob)?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-02-11T19:48:52Z","receivedAt":"2009-02-11T19:48:52Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Peter Baumann <waste.manager@gmx.de> writes:\n\n> after reading Junio's nice blog today where he explained how to use git grep\n> efficiently, I saw him using a glob to match for the interesting files:\n>\n> \t $ git grep -e ';;' -- '*.c'\n>\n> Is it possible to have the same feature in git diff and the revision\n> machinery? Because I tried\n>\n> \t$ cd $path_to_your_git_src_dir\n> \t$ git log master -p -- '*.h'\n> \t.... No commit shown \n>\n> \t$ git diff --name-only v1.5.0  v1.6.0 -- '*.c'\n>\n> and both don't return anything.\n\nThere was a recent discussion on this.  The index family uses glob, the\ntree family uses leading-path only.  The one implemented for grep can do\nboth, and attempts to unify both by providing possibly reusable interface\nso that the other two families can be ported to, but we haven't managed to\ntrick anybody to take up the task ;-).\n"},{"id":"104297","messageId":"20090212060900.6117@nanako3.lavabit.com","threadId":"17730","inReplyTo":"7v1vu4eouz.fsf@gitster.siamese.dyndns.org","subject":"Re: How to use path limiting (using a glob)?","fromName":"Nanako Shiraishi","fromEmail":"nanako3@lavabit.com","sentAt":"2009-02-11T21:09:00Z","receivedAt":"2009-02-11T21:09:00Z","isPatch":false,"sender":{"key":"nanako3@lavabit.com","avatar":"https://gravatar.com/avatar/3777b9e201c5883a62b1a6fdf7c53f2d712d1d80989146063ea861e33aad72a8?d=mp&s=160"},"body":"Quoting Junio C Hamano <gitster@pobox.com>:\n> Peter Baumann <waste.manager@gmx.de> writes:\n>> Hallo,\n>>\n>> after reading Junio's nice blog today where he explained how to use git grep\n>> efficiently, I saw him using a glob to match for the interesting files:\n>>\n>> \t $ git grep -e ';;' -- '*.c'\n>>\n>> Is it possible to have the same feature in git diff and the revision\n>> machinery? Because I tried\n>>\n>> \t$ cd $path_to_your_git_src_dir\n>> \t$ git log master -p -- '*.h'\n>> \t.... No commit shown \n>>\n>> \t$ git diff --name-only v1.5.0  v1.6.0 -- '*.c'\n>>\n>> and both don't return anything.\n> There was a recent discussion on this.  The index family uses glob, the\n> tree family uses leading-path only.  The one implemented for grep can do\n> both, and attempts to unify both by providing possibly reusable interface\n> so that the other two families can be ported to, but we haven't managed to\n> trick anybody to take up the task ;-).\n\nThe list archive has nicely written summaries on the issues and suggestions on how to make this possible:\n\n    http://article.gmane.org/gmane.comp.version-control.git/94628\n    http://thread.gmane.org/gmane.comp.version-control.git/105638/focus=105679\n\n-- \nNanako Shiraishi, the unofficial project secratary of the git project\nhttp://ivory.ap.teacup.com/nanako3/\n"},{"id":"104365","messageId":"20090212102719.GD27232@m62s10.vlinux.de","threadId":"17730","inReplyTo":"alpine.LFD.2.00.0902111129190.3590@localhost.localdomain","subject":"Re: How to use path limiting (using a glob)?","fromName":"Peter Baumann","fromEmail":"waste.manager@gmx.de","sentAt":"2009-02-12T10:27:20Z","receivedAt":"2009-02-12T10:27:20Z","isPatch":false,"sender":{"key":"waste.manager@gmx.de","avatar":null},"body":"On Wed, Feb 11, 2009 at 11:40:44AM -0800, Linus Torvalds wrote:\n> \n> \n> On Wed, 11 Feb 2009, Peter Baumann wrote:\n> \n> > after reading Junio's nice blog today where he explained how to use git grep\n> > efficiently, I saw him using a glob to match for the interesting files:\n> > \n> > \t $ git grep -e ';;' -- '*.c'\n> > \n> > Is it possible to have the same feature in git diff and the revision\n> > machinery?\n> \n> Not really. Git has two different kinds of path limiters, and they are \n> really really different.\n> \n>  - the \"walk current index/directory recursively\" kind that \"git ls-files\" \n>    uses, which takes a 'fnmatch()' type path regexp (not a real regexp, \n>    but the kind you're used to with shell)\n> \n>    NOTE! On purpose, we don't set the FNM_PATHNAME, so \"*.c\" here is \n>    different from *.c in shell (it's more like \"**.c\" in tcsh). IOW, * \n>    matches '/' too, and will walk subdirectories.\n> \n\nHm. But if git does only anchor the * at the current directory, wouldn't\nthis solve (or at least reduce) the performance problems you described in the\nlater paragraph? Having the \"**.c\" do a recurisve search for every .c\nfile would then be used to do a recusrive search. \n\n>  - the \"revision limiter\" pathspec. This is *not* a regexp, it's a pure \n>    prefix matcher, for a very simple reason: performance.\n> \n> > \t$ cd $path_to_your_git_src_dir\n> > \t$ git log master -p -- '*.h'\n> > \t.... No commit shown \n> > \n> > \t$ git diff --name-only v1.5.0  v1.6.0 -- '*.c'\n> > \n> > and both don't return anything.\n> \n> Yeah, in the revision matcher you can still depend on the shell \n> expansion, and it will do _almost_ the right thing. So if you do\n> \n> \tgit log master -p *.c\n> \n> without the quotes, the shell expansion will work, and that in turn will \n> give a set of filenames that \"git log\" will restrict the log to. HOWEVER, \n> it's not a real wildcard - it's literally looking at what you have now in \n> your current working directory, and saying \"give me the logs of those \n> pathnames\", not \"give me the logs of everything ending with .c\".\n> \n\nOk. Thats actually the reason why I asked for this, because if a file\ngot removed it wouldn't be found by this.\n\n> We _could_ make the revision limiter understand fnmatch-style patterns, \n> but quite frankly, it's very very expensive - too expensive to be useful \n> for big repositories. The point about only matching prefixes is that it \n> allows the revision limiter to not even walk into subdirectories that \n> don't match, but if you do the \"*.c\" kind of pattern, now the revision \n> code has to look up every tree recursively. That code is also _extremely_ \n> performance-critical, so we really don't want to use fnmatch() when we can \n> currently use just \"memcmp()\".\n> \n> So yes, it's kind of odd how we have two totally different concepts of \n> pathname patterns, but it's probably easiest to remember that \"'git grep' \n> is just special\". \n> \n> \t\tLinus\n"},{"id":"104370","messageId":"slrngp80rg.rmg.sitaramc@sitaramc.homelinux.net","threadId":"17730","inReplyTo":"20090212102719.GD27232@m62s10.vlinux.de","subject":"Re: How to use path limiting (using a glob)?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2009-02-12T11:09:36Z","receivedAt":"2009-02-12T11:09:36Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On 2009-02-12, Peter Baumann <waste.manager@gmx.de> wrote:\n> On Wed, Feb 11, 2009 at 11:40:44AM -0800, Linus Torvalds wrote:\n\n>> On Wed, 11 Feb 2009, Peter Baumann wrote:\n>> \n>> > after reading Junio's nice blog today where he\n>> > explained how to use git grep efficiently, I saw him\n>> > using a glob to match for the interesting files:\n>> > \n>> > \t $ git grep -e ';;' -- '*.c'\n>> > \n>> > Is it possible to have the same feature in git diff and the revision\n>> > machinery?\n>> \n>> Not really. Git has two different kinds of path limiters, and they are \n>> really really different.\n>> \n>>  - the \"walk current index/directory recursively\" kind that \"git ls-files\" \n>>    uses, which takes a 'fnmatch()' type path regexp (not a real regexp, \n>>    but the kind you're used to with shell)\n>> \n>>    NOTE! On purpose, we don't set the FNM_PATHNAME, so \"*.c\" here is \n>>    different from *.c in shell (it's more like \"**.c\" in tcsh). IOW, * \n>>    matches '/' too, and will walk subdirectories.\n>> \n>\n> Hm. But if git does only anchor the * at the current directory, wouldn't\n> this solve (or at least reduce) the performance problems you described in the\n> later paragraph? Having the \"**.c\" do a recurisve search for every .c\n> file would then be used to do a recusrive search. \n\nI think Linus meant that it's expensive to look for all *.c\nfiles at any depth in the tree, for every commit in\nrepository.\n\nYou can have either a prefix matcher to limit the search\n*within* a tree so you can afford to walk all revs in the\nrepo, or you stick to just one tree (or a few explicitly\nnamed ones).\n\nYou seem to be saying 'fine, I know, and I'm willing to\nindicate that I'm accepting this cost by using a different\nsyntax'.\n\nBut the syntax is not the point.  You can certainly do that\nright now, if you really wish to.  Just don't try it on a\nlarge repo :-)\n\n    git grep -e pattern $(git rev-list --all) -- *.c\n\nMake suitable modifications to the '--all' in the git\nrev-list to limit the revs you want to search.\n\nRegardless of whether there is a simple syntax to support it\nor not, this is probably not what you want, most of the time\n:-)\n\nSitaram\n"}]}