{"thread":{"id":"19972","subject":"RFH - git-log variant that _does_ search through diffs","startedAt":"2009-06-30T00:08:47Z","lastAt":"2009-07-01T08:27:46Z","messageCount":12,"participants":["Eric Raible","Jeff King","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"117198","messageId":"279b37b20906291708g67da3a75p316ea4893f02666a@mail.gmail.com","threadId":"19972","inReplyTo":null,"subject":"RFH - git-log variant that _does_ search through diffs","fromName":"Eric Raible","fromEmail":"raible@gmail.com","sentAt":"2009-06-30T00:08:47Z","receivedAt":"2009-06-30T00:08:47Z","isPatch":false,"sender":{"key":"raible@gmail.com","avatar":null},"body":"[Surely this has been address before,  but I wasn't able to find it...]\n\nThe documentation for git-log -S includes:\n\n\"Look for differences that introduce or remove an instance of <string>.\n Note that this is different than the string simply appearing in diff output\"\n\nBut I want to do that \"different\" thing (IOW I want search the diff output).\n\nSo must I loop through git-rev-list, grepping git-diff output on each commit?\n\nOr if it _is_ possible to search the diff output directly then it\nmight be useful\nto link to the relevant description instead of saying what -S doesn't do.\n\nThanks - Eric\n"},{"id":"117203","messageId":"20090630040337.GA23741@sigio.peff.net","threadId":"19972","inReplyTo":"279b37b20906291708g67da3a75p316ea4893f02666a@mail.gmail.com","subject":"Re: RFH - git-log variant that _does_ search through diffs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-06-30T04:03:37Z","receivedAt":"2009-06-30T04:03:37Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jun 29, 2009 at 05:08:47PM -0700, Eric Raible wrote:\n\n> [Surely this has been address before,  but I wasn't able to find it...]\n\nThere is some discussion here:\n\n  http://article.gmane.org/gmane.comp.version-control.git/112077\n\n> The documentation for git-log -S includes:\n> \n> \"Look for differences that introduce or remove an instance of <string>.\n>  Note that this is different than the string simply appearing in diff output\"\n> \n> But I want to do that \"different\" thing (IOW I want search the diff output).\n> \n> So must I loop through git-rev-list, grepping git-diff output on each commit?\n\nCurrently, yes. There is no way to do it internally. A patch to\nimplement it would probably be accepted, though (see the thread I\nmentioned above for more details).\n\nYou can at least combine rev-list and diff into one command, and grep\nlike this (for 'foo'):\n\n  git log -z -p | perl -0ne 'print if /^[-+].*foo/m' | tr '\\0' '\\n'\n\n-Peff\n"},{"id":"117218","messageId":"7vd48m9m8i.fsf@alter.siamese.dyndns.org","threadId":"19972","inReplyTo":"20090630040337.GA23741@sigio.peff.net","subject":"Re: RFH - git-log variant that _does_ search through diffs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-06-30T08:09:17Z","receivedAt":"2009-06-30T08:09:17Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> On Mon, Jun 29, 2009 at 05:08:47PM -0700, Eric Raible wrote:\n>\n>> [Surely this has been address before,  but I wasn't able to find it...]\n>\n> There is some discussion here:\n>\n>   http://article.gmane.org/gmane.comp.version-control.git/112077\n>\n> Currently, yes. There is no way to do it internally. A patch to\n> implement it would probably be accepted, though (see the thread I\n> mentioned above for more details).\n\nSpecifically:\n\n    http://thread.gmane.org/gmane.comp.version-control.git/112077/focus=112114\n\nand its cousin\n\n    http://article.gmane.org/gmane.comp.version-control.git/112141\n"},{"id":"117245","messageId":"279b37b20906301105tf92f471g6146a083ba2e2882@mail.gmail.com","threadId":"19972","inReplyTo":"20090630040337.GA23741@sigio.peff.net","subject":"Re: RFH - git-log variant that _does_ search through diffs","fromName":"Eric Raible","fromEmail":"raible@gmail.com","sentAt":"2009-06-30T18:05:08Z","receivedAt":"2009-06-30T18:05:08Z","isPatch":false,"sender":{"key":"raible@gmail.com","avatar":null},"body":"On Mon, Jun 29, 2009 at 9:03 PM, Jeff King<peff@peff.net> wrote:\n\n> You can at least combine rev-list and diff into one command, and grep\n> like this (for 'foo'):\n>\n>  git log -z -p | perl -0ne 'print if /^[-+].*foo/m' | tr '\\0' '\\n'\n>\n> -Peff\n\nThank you, that will do very nicely as a starting point.\n\nWhat I _really_ want is the subset of all commits containing foo\nwho's oneline commit message doesn't match a given regexp.\n\nSo I'm used something like this to extract the commits of interest:\n\ngit log -z -p | perl -0ne 'print if /^[-+].*foo/m' | tr '\\0' '\\n' |\ngrep \"^commit [0-9a-f]\" | awk '{print $2}' |\nxargs -n1 git log --pretty=oneline -1 |\ngrep -v dont_want\n\nIn this specific case of wanting to ignore particular commits a loop\nover git-rev-list might yield a better solution.  But the 'git-log | perl | tr'\nsnippet is a nice idiom for day-to-day use.\n"},{"id":"117246","messageId":"279b37b20906301106h4ccbeaa5l621e427981e5c483@mail.gmail.com","threadId":"19972","inReplyTo":"7vd48m9m8i.fsf@alter.siamese.dyndns.org","subject":"Re: RFH - git-log variant that _does_ search through diffs","fromName":"Eric Raible","fromEmail":"raible@gmail.com","sentAt":"2009-06-30T18:06:50Z","receivedAt":"2009-06-30T18:06:50Z","isPatch":false,"sender":{"key":"raible@gmail.com","avatar":null},"body":"On Tue, Jun 30, 2009 at 1:09 AM, Junio C Hamano<gitster@pobox.com> wrote:\n>\n> Specifically:\n>\n>    http://thread.gmane.org/gmane.comp.version-control.git/112077/focus=112114\n>\n> and its cousin\n>\n>    http://article.gmane.org/gmane.comp.version-control.git/112141\n\nThanks - that's exactly what I was looking for but didn't find.\n"},{"id":"117249","messageId":"20090630193135.GA4460@sigio.peff.net","threadId":"19972","inReplyTo":"279b37b20906301105tf92f471g6146a083ba2e2882@mail.gmail.com","subject":"Re: RFH - git-log variant that _does_ search through diffs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-06-30T19:31:35Z","receivedAt":"2009-06-30T19:31:35Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jun 30, 2009 at 11:05:08AM -0700, Eric Raible wrote:\n\n> What I _really_ want is the subset of all commits containing foo\n> who's oneline commit message doesn't match a given regexp.\n> \n> So I'm used something like this to extract the commits of interest:\n> \n> git log -z -p | perl -0ne 'print if /^[-+].*foo/m' | tr '\\0' '\\n' |\n> grep \"^commit [0-9a-f]\" | awk '{print $2}' |\n> xargs -n1 git log --pretty=oneline -1 |\n> grep -v dont_want\n\nI think you can do this a little more simply and efficiently as:\n\n  git log -z -p --format='GREP: %s' |\n    perl -0ne 'print if /^[-+].*foo/m && !/^GREP:.*dont_want/' |\n    tr '\\0' '\\n'\n\n(though note that --format is new as of 1.6.3, I think; before that you\nhave to use \"--pretty=format:\"). Many fewer process invocations, and\nless typing, though still easy to mess up. At one point I had considered\nwriting small wrapper scripts that understood the log output so you\ncould say:\n\n  git log -z -p | filter-author $A | filter-diff $D | filter-subject $S\n\nwhich is nicely readable and Unix-y, but is really _slow_ compared to\ngit doing it all in a single process. I think a \"--grep-subject\" and a\n\"--grep-diff\" (aka \"--search\") are the only things that are missing now,\nand those would both be pretty easy to implement.\n\n-Peff\n"},{"id":"117265","messageId":"279b37b20906301422w2b895afai5edf42fcbfcc7d5a@mail.gmail.com","threadId":"19972","inReplyTo":"20090630193135.GA4460@sigio.peff.net","subject":"Re: RFH - git-log variant that _does_ search through diffs","fromName":"Eric Raible","fromEmail":"raible@gmail.com","sentAt":"2009-06-30T21:22:43Z","receivedAt":"2009-06-30T21:22:43Z","isPatch":false,"sender":{"key":"raible@gmail.com","avatar":null},"body":"On Tue, Jun 30, 2009 at 12:31 PM, Jeff King<peff@peff.net> wrote:\n>\n> I think you can do this a little more simply and efficiently as:\n>\n>  git log -z -p --format='GREP: %s' |\n>    perl -0ne 'print if /^[-+].*foo/m && !/^GREP:.*dont_want/' |\n>    tr '\\0' '\\n'\n>\n> (though note that --format is new as of 1.6.3, I think; before that you\n> have to use \"--pretty=format:\"). Many fewer process invocations, and\n> less typing, though still easy to mess up.\n\nI agree that --format leads to a much prettier solution.\nUnfortunately --format seems to turn off -z (at least in msysgit):\n\n$ git --version\ngit version 1.6.3.2.1299.gee46c\n$ git log -p > L1\n$ git log -p -z > L2\n$ diff L1 L2 | wc\n   2415    4347   62889\n$ git log -p --format=%s > L1\n$ git log -p -z --format=%s > L2\n$ diff L1 L2 | wc\n      0       0       0\n"},{"id":"117266","messageId":"279b37b20906301434r46e6973ftf5e5f68778f9fa1d@mail.gmail.com","threadId":"19972","inReplyTo":"279b37b20906301422w2b895afai5edf42fcbfcc7d5a@mail.gmail.com","subject":"Re: RFH - git-log variant that _does_ search through diffs","fromName":"Eric Raible","fromEmail":"raible@gmail.com","sentAt":"2009-06-30T21:34:45Z","receivedAt":"2009-06-30T21:34:45Z","isPatch":false,"sender":{"key":"raible@gmail.com","avatar":null},"body":"On Tue, Jun 30, 2009 at 2:22 PM, Eric Raible<raible@gmail.com> wrote:\n> On Tue, Jun 30, 2009 at 12:31 PM, Jeff King<peff@peff.net> wrote:\n>>\n>> I think you can do this a little more simply and efficiently as:\n>>\n>>  git log -z -p --format='GREP: %s' |\n>>    perl -0ne 'print if /^[-+].*foo/m && !/^GREP:.*dont_want/' |\n>>    tr '\\0' '\\n'\n>>\n>> (though note that --format is new as of 1.6.3, I think; before that you\n>> have to use \"--pretty=format:\"). Many fewer process invocations, and\n>> less typing, though still easy to mess up.\n>\n> I agree that --format leads to a much prettier solution.\n> Unfortunately --format seems to turn off -z (at least in msysgit):\n\nSorry to self-reply, but one obvious workaround is to encode the NULL\nexplicitly:\n\ngit log -z -p --format='%x00GREP: %s' | ...\n"},{"id":"117287","messageId":"20090701070252.GA7987@sigio.peff.net","threadId":"19972","inReplyTo":"279b37b20906301422w2b895afai5edf42fcbfcc7d5a@mail.gmail.com","subject":"Re: RFH - git-log variant that _does_ search through diffs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-07-01T07:02:52Z","receivedAt":"2009-07-01T07:02:52Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jun 30, 2009 at 02:22:43PM -0700, Eric Raible wrote:\n\n> Unfortunately --format seems to turn off -z (at least in msysgit):\n> \n> $ git --version\n> git version 1.6.3.2.1299.gee46c\n> $ git log -p > L1\n> $ git log -p -z > L2\n> $ diff L1 L2 | wc\n>    2415    4347   62889\n> $ git log -p --format=%s > L1\n> $ git log -p -z --format=%s > L2\n> $ diff L1 L2 | wc\n>       0       0       0\n\nUgh. I did some looking into this. It actually does work if you do:\n\n  git log -p -z --pretty=format:%s\n\nrather than\n\n  git log -p -z --pretty=tformat:%s\n\nAnd --format=%s behaves as if 'tformat' was given. And if you are not up\nto date on the difference, \"format\" implies \"separator semantics\", where\nthe NUL is placed _between_ each record. \"tformat\" implies \"terminator\nsemantics\", where the NUL is placed at the end.\n\nPlacing the separator correctly is fairly easy; when we start a new\nrecord, if we are not the first, we print the separator. Placing a\nterminator is a little trickier because of the way the code is\nstructured. I'll post my attempt in a minute; see patch 2/2 for more\ndiscussion.\n\n-Peff\n"},{"id":"117289","messageId":"20090701072628.GA8101@sigio.peff.net","threadId":"19972","inReplyTo":"20090701070252.GA7987@sigio.peff.net","subject":"[PATCH 1/2] log-tree: fix confusing comment","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-07-01T07:26:28Z","receivedAt":"2009-07-01T07:26:28Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"This comment mentions the case where use_terminator is set,\nbut this case is not handled at all by this chunk of code.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nThis comment confused me quite a bit while tracking down the issue in\n2/2.\n\n log-tree.c |    3 ++-\n 1 files changed, 2 insertions(+), 1 deletions(-)\n\ndiff --git a/log-tree.c b/log-tree.c\nindex 59d63eb..6f73c17 100644\n--- a/log-tree.c\n+++ b/log-tree.c\n@@ -321,7 +321,8 @@ void show_log(struct rev_info *opt)\n \t}\n \n \t/*\n-\t * If use_terminator is set, add a newline at the end of the entry.\n+\t * If use_terminator is set, we already handled any record termination\n+\t * at the end of the last record.\n \t * Otherwise, add a diffopt.line_termination character before all\n \t * entries but the first.  (IOW, as a separator between entries)\n \t */\n-- \n1.6.3.3.485.g0f5d4.dirty\n"},{"id":"117288","messageId":"20090701072632.GB8101@sigio.peff.net","threadId":"19972","inReplyTo":"20090701070252.GA7987@sigio.peff.net","subject":"[RFC/PATCH 2/2] fix missing NUL termination with pretty=tformat","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-07-01T07:26:33Z","receivedAt":"2009-07-01T07:26:33Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"When using \"git log -z\" with --pretty=format's \"separator\"\nsemantics, we correctly insert a NUL between each record.\nHowever, with \"--pretty=tformat\", we output no NULs at all,\nwhereas we should output one after each commit.\n\nWe can't just put a conditional in the code for the\n\"separator\" case; that code is triggered at a completely\ndifferent time: when _starting_ a new commit, and we are not\nthe first commit to be shown. As opposed to termination\nsemantics, which means we must print the terminator at the\nend of each commit.\n\nAdding to the trickiness is that we must handle two cases:\nwith and without diff output. In fact, there is already a\nspot (log-tree.c, ll. 442-445) which adds a hard-coded\nnewline as a terminator after the commit message. But we\ncan't just modify that to use the specified line terminator,\nbecause sometimes it is acting as a separator between commit\nmessage and diff, and sometimes it is acting as the\nterminator of the whole record.\n\nSimply adding another terminator after each commit has been\nshown will end up with doubled newlines for short user\nformats (like '%s'). Instead, we add the record terminator\nonly if it is not a newline, in which case the output will\nactually contain a newline followed by the terminator.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nThis one is RFC. It is missing tests, but that is because I am not\ncompletely sure what we want the output to look like. With this change,\nyou still get a newline at the end of a single-line user-formatted\nstring, like:\n\n  $ git log -z --format:%s\n  three\n  ^@two\n  ^@one\n  ^@\n\nAs explained above, it is not correct to simply turn that putchar('\\n')\ninto putchar(opt->diffopt.line_termination), since it may be followed by\nthe diff, in which case we _want_ the newline. But maybe it makes sense\nto suppress it if we have an alternate line terminator and we are not\nshowing the diff.\n\nThe case with a diff looks much better:\n\n  $ git log -z -p --format:%s\n  three\n\n  diff ...\n  ...\n  ^@two\n\n  diff ...\n  ...\n  ^@one\n\n  diff ...\n  ...\n  ^@\n\nSo I'm not sure if it is OK as-is, or if we should add that newline\nsuppression tweak.  People aren't going to be looking at \"-z\" output as\na whole, so it is really about whether somebody saying:\n\n  git log -z --pretty=tformat:\"XXX %s YYY\" | perl -0ne ...\n\nwould be surprised to find a newline after \"YYY\" in each record.\n\n log-tree.c |    8 ++++++++\n 1 files changed, 8 insertions(+), 0 deletions(-)\n\ndiff --git a/log-tree.c b/log-tree.c\nindex 6f73c17..36930ff 100644\n--- a/log-tree.c\n+++ b/log-tree.c\n@@ -557,6 +557,14 @@ int log_tree_commit(struct rev_info *opt, struct commit *commit)\n \t\tshow_log(opt);\n \t\tshown = 1;\n \t}\n+\t/*\n+\t * If we have a terminator, we want to print it here. Unless\n+\t * it is a newline, in which case we will already have just\n+\t * printed one at the end of the commit message or at the end\n+\t * of the diff, and we will end up doubling it.\n+\t */\n+\tif (opt->use_terminator && opt->diffopt.line_termination != '\\n')\n+\t\tputchar(opt->diffopt.line_termination);\n \topt->loginfo = NULL;\n \tmaybe_flush_or_die(stdout, \"stdout\");\n \treturn shown;\n-- \n1.6.3.3.485.g0f5d4.dirty\n"},{"id":"117294","messageId":"7vprcklse5.fsf@alter.siamese.dyndns.org","threadId":"19972","inReplyTo":"20090701072632.GB8101@sigio.peff.net","subject":"Re: [RFC/PATCH 2/2] fix missing NUL termination with pretty=tformat","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-07-01T08:27:46Z","receivedAt":"2009-07-01T08:27:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> This one is RFC. It is missing tests, but that is because I am not\n> completely sure what we want the output to look like. With this change,\n> you still get a newline at the end of a single-line user-formatted\n> string, like:\n>\n>   $ git log -z --format:%s\n>   three\n>   ^@two\n>   ^@one\n>   ^@\n\nI haven't thought things through, but my knee-jerk reaction is that we\nshould treat \"%s\" itself as not having a final LF in itself.  So I would\nactually expect the above to produce\n\n    three^@two^@one^@\n\nIn other words, --pretty=format:%s and --pretty=format:X-%s-Y should\nbehave identically.  Strings produced by these format specifications will\nnot have terminating LF appended (because %s itself does not end with LF,\nand in the second example you did not write any LF after your \"Y\").  For\nconsistency, I think %b should include the LF at the end on its own, but I\nmay be wrong on this last point.\n\n\"git log\" (with or without -z) would begin with the string that comes out\nof the log message part (i.e. with or without any custom format via\nformat/tformat) first.\n\nWhen run with -p, the output would be followed by a LF (with -z, or\nwithout), followed by the patch text (each line of which is terminated\nwith a LF, again with or without -z).  Without -p, this part is omitted.\n\nAfter that, a LF/NUL (depending on the use of -z) would be appended if\nthere are more records or if we are using the terminator semantics.  Then\nwe would move on to the next record.\n\nE.g.  When \"log -p\" shows a commit with a one-liner message, like this:\n\n    commit e8a39af8d471d190a749c390a0cf614cb59ec8ee\n    Author: Junio C Hamano <gitster@pobox.com>\n    Date:   Wed Jul 1 00:50:48 2009 -0700\n\n        second commit\n\n    diff --git a/one b/one\n    index e69de29..d00491f 100644\n    --- a/one\n    +++ b/one\n    @@ -0,0 +1 @@\n    +1\n\n    Side note. Conceptually I would say that canned formats (--pretty,\n    --pretty=short etc. but not --pretty=oneline) produce a string that\n    ends with \"second commit\\n\" in the above example, and the blank line\n    between the log and beginning of \"diff --git\" is coming from the rule\n    \"log is always followed by a LF (with or without -z)\".\n\n\"log -p --pretty=format:X-%s-Y\" would begin with\n\n    X-second commit-Y\n    diff --git a/one b/one\n    index e69de29..d00491f 100644\n\n\"log --pretty=format:X-%s-Y\" would become\n\n    X-second commit-Y\n    X-initial commit-Y\n\nand with -z, the latter would look\n    \n    X-second commit-Y^@X-initial commit-Y\n\nwhile \"log --pretty=tformat:X-%s-Y -z\" would be\n\n    X-second commit-Y^@X-initial commit-Y^@\n\nAll of the above was written, ignoring the memory of what the current code\nactually does, nor checking if it is easy to implement without tweaking\nthe current code structure too much.  So it may not help your RFC, but I\nat least think it is internally consistent.\n"}]}