{"thread":{"id":"53797","subject":"Git log: filtering by date and by lines do not work together","startedAt":"2020-07-03T14:14:36Z","lastAt":"2020-07-07T01:37:43Z","messageCount":4,"participants":["Мария Долгополова","René Scharfe","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"401011","messageId":"CAD6JL5SRvB_yXcqJ5HGADQaGtZAiKrtd447vtHLSagJqNGPChA@mail.gmail.com","threadId":"53797","inReplyTo":null,"subject":"Git log: filtering by date and by lines do not work together","fromName":"Мария Долгополова","fromEmail":"dolgopolovamariia@gmail.com","sentAt":"2020-07-03T14:14:23Z","receivedAt":"2020-07-03T14:14:36Z","isPatch":false,"sender":{"key":"dolgopolovamariia@gmail.com","avatar":null},"body":"Hi! I would like to report the bug.\n\nI want to get commits that change specific lines in a specific file\nand were made before a specific date.\n\nSteps:\n1) I apply filtering by date: --before=\"2016-11-18\"\nResult is in the picture FilteringByDate.png. Commit\n570778797effd69bd4536c51125b7d2e8f654c08 is presented.\n\n2) I apply filtering by lines:\n-L33,34:src/main/java/org/toradocu/translator/Subject.java\nResult is in the picture FilteringByLines.png . Commit\n570778797effd69bd4536c51125b7d2e8f654c08 is presented.\n\n3) Finally, I apply filtering by lines and by date:\n--before=\"2016-11-18\"\n-L33,34:src/main/java/org/toradocu/translator/Subject.java\nResult is in the picture FilteringByDatesAndLines.png . Commit\n570778797effd69bd4536c51125b7d2e8f654c08 is NOT presented.\n\nExpected Result: the commit is presented in filter 1 and in filter 2,\nso it must be presented in filter “1 AND 2”\n\nActual Result: the commit is NOT presented in filter “1 AND 2”\n\nEnvironment: git version 2.27.0.windows.1\n\n I beg you to provide me with a workaround. I am doing research work\non analyzing commits in open source projects. Therefore, this\nfunctionality is very important to me.\n"},{"id":"401019","messageId":"2c4a6657-1a47-cfbb-21cd-04f131d2ed49@web.de","threadId":"53797","inReplyTo":"CAD6JL5SRvB_yXcqJ5HGADQaGtZAiKrtd447vtHLSagJqNGPChA@mail.gmail.com","subject":"Re: Git log: filtering by date and by lines do not work together","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2020-07-04T12:56:23Z","receivedAt":"2020-07-04T12:56:33Z","isPatch":false,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 03.07.20 um 16:14 schrieb Мария Долгополова:\n> Hi! I would like to report the bug.\n>\n> I want to get commits that change specific lines in a specific file\n> and were made before a specific date.\n>\n> Steps:\n> 1) I apply filtering by date: --before=\"2016-11-18\"\n> Result is in the picture FilteringByDate.png. Commit\n> 570778797effd69bd4536c51125b7d2e8f654c08 is presented.\n>\n> 2) I apply filtering by lines:\n> -L33,34:src/main/java/org/toradocu/translator/Subject.java\n> Result is in the picture FilteringByLines.png . Commit\n> 570778797effd69bd4536c51125b7d2e8f654c08 is presented.\n>\n> 3) Finally, I apply filtering by lines and by date:\n> --before=\"2016-11-18\"\n> -L33,34:src/main/java/org/toradocu/translator/Subject.java\n> Result is in the picture FilteringByDatesAndLines.png . Commit\n> 570778797effd69bd4536c51125b7d2e8f654c08 is NOT presented.\n>\n> Expected Result: the commit is presented in filter 1 and in filter 2,\n> so it must be presented in filter “1 AND 2”\n>\n> Actual Result: the commit is NOT presented in filter “1 AND 2”\n>\n> Environment: git version 2.27.0.windows.1\n>\n> I beg you to provide me with a workaround. I am doing research work\n> on analyzing commits in open source projects. Therefore, this\n> functionality is very important to me.\n\nYou could do the date filtering on the output of git log, e.g.:\n\n git log -L33,34:src/main/java/org/toradocu/translator/Subject.java --date=short --color=always |\n awk -v before=2016-11-18 '\n /^[^ ]*commit/ {state=1; header=\"\"}\n /^[^ ]*Date:/ {if ($2 <= before) {state=2; printf \"%s\", header} else {state=0}}\n state == 1 {header = header $0 ORS}\n state == 2 {print}\n '\n\nThis command is a bit unwieldy, so you might want to put the AWK\nscript into a file, then you could use it like this:\n\n git log -L33,34:src/main/java/org/toradocu/translator/Subject.java --date=short --color=always |\n awk -v before=2016-11-18 -f gitlogbefore.awk\n\nRené\n"},{"id":"401020","messageId":"0c55e178-8f7c-ef84-e6eb-e50de9924ef6@web.de","threadId":"53797","inReplyTo":"CAD6JL5SRvB_yXcqJ5HGADQaGtZAiKrtd447vtHLSagJqNGPChA@mail.gmail.com","subject":"[PATCH] revision: disable min_age optimization with line-log","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2020-07-04T12:56:32Z","receivedAt":"2020-07-04T12:56:46Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"If one of the options --before, --min-age or --until is given,\nlimit_list() filters out younger commits early on. Line-log needs all\nthose commits to trace the movement of line ranges, though. Skip this\noptimization if both are used together.\n\nReported-by: Мария Долгополова \nSigned-off-by: René Scharfe \n---\nNeeds careful review -- I'm not familiar with that the line-log code and\nthe revision traversal machinery is a bit scary. AFAIU the min_age\ncheck is done again in get_commit_action(), so this patch shouldn't\ncause underage commits to be shown, but I'm not sure. Test coverage for\nthe three options is spotty. :-/\n\n revision.c | 3 ++-\n t/t4211-line-log.sh | 8 ++++++++\n 2 files changed, 10 insertions(+), 1 deletion(-)\n\ndiff --git a/revision.c b/revision.c\nindex ebb4d2a0f2..3bdc1bbf2a 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -1410,7 +1410,8 @@ static int limit_list(struct rev_info *revs)\n \t\t\t\tcontinue;\n \t\t\tbreak;\n \t\t}\n-\t\tif (revs->min_age != -1 && (commit->date > revs->min_age))\n+\t\tif (revs->min_age != -1 && (commit->date > revs->min_age) &&\n+\t\t !revs->line_level_traverse)\n \t\t\tcontinue;\n \t\tdate = commit->date;\n \t\tp = &commit_list_insert(commit, p)->next;\ndiff --git a/t/t4211-line-log.sh b/t/t4211-line-log.sh\nindex 1428eae262..e186c83250 100755\n--- a/t/t4211-line-log.sh\n+++ b/t/t4211-line-log.sh\n@@ -240,10 +240,12 @@ test_expect_success 'setup for checking line-log and parent oids' '\n \tEOF\n \tgit add file.c &&\n \ttest_tick &&\n+\tfirst_tick=$test_tick &&\n \tgit commit -m \"Add func1() and func2() in file.c\" &&\n\n \techo 1 >other-file &&\n \tgit add other-file &&\n+\ttest_tick &&\n \tgit commit -m \"Add other-file\" &&\n\n \tsed -e \"s/F1/F1 + 1/\" file.c >tmp &&\n@@ -283,4 +285,10 @@ test_expect_success 'parent oids with parent rewriting' '\n \ttest_cmp expect actual\n '\n\n+test_expect_success 'line-log with --before' '\n+\techo $root_oid >expect &&\n+\tgit log --format=%h --no-patch -L:func2:file.c --before=$first_tick >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\n--\n2.27.0\n"},{"id":"401061","messageId":"xmqqa70c85r4.fsf@gitster.c.googlers.com","threadId":"53797","inReplyTo":"0c55e178-8f7c-ef84-e6eb-e50de9924ef6@web.de","subject":"Re: [PATCH] revision: disable min_age optimization with line-log","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-07-07T01:37:35Z","receivedAt":"2020-07-07T01:37:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"René Scharfe writes:\n\n> If one of the options --before, --min-age or --until is given,\n> limit_list() filters out younger commits early on. Line-log needs all\n> those commits to trace the movement of line ranges, though. Skip this\n> optimization if both are used together.\n>\n> Reported-by: Мария Долгополова \n> Signed-off-by: René Scharfe \n> ---\n> Needs careful review -- I'm not familiar with that the line-log code and\n> the revision traversal machinery is a bit scary. AFAIU the min_age\n> check is done again in get_commit_action(), so this patch shouldn't\n> cause underage commits to be shown, but I'm not sure. Test coverage for\n> the three options is spotty. :-/\n\nI am not familiar with the line-log code, either, but anyway. This\nstarts queuing commits that are too young in the list of commits to\nbe processed, but later stages in the output callchain would decide\nthat they are too young to be shown in get_commit_action() that is\ncalled from simplify_commit(), so the overall effect of this change\nis to show these commits to the line-log machinery but filter them\nout of the final output phase (with the parent rewriting, the\nline-log machinery sees these commits in prepare_revision_walk(),\nand without it, each of these commits is shown to the line-log\nmachinery immediately before get_commit_action() decides that it is\ntoo young to be shown). And the effect you want to gain by showing\nthese commits to the line-log machinery is to ensure that the range\nof lines in the starting point is adjusted for the changes each of\nthese commits makes.\n\nWhich makes sense to me.\n\nThanks, will queue.\n\n> revision.c | 3 ++-\n> t/t4211-line-log.sh | 8 ++++++++\n> 2 files changed, 10 insertions(+), 1 deletion(-)\n>\n> diff --git a/revision.c b/revision.c\n> index ebb4d2a0f2..3bdc1bbf2a 100644\n> --- a/revision.c\n> +++ b/revision.c\n> @@ -1410,7 +1410,8 @@ static int limit_list(struct rev_info *revs)\n> \t\t\t\tcontinue;\n> \t\t\tbreak;\n> \t\t}\n> -\t\tif (revs->min_age != -1 && (commit->date > revs->min_age))\n> +\t\tif (revs->min_age != -1 && (commit->date > revs->min_age) &&\n> +\t\t !revs->line_level_traverse)\n> \t\t\tcontinue;\n> \t\tdate = commit->date;\n> \t\tp = &commit_list_insert(commit, p)->next;\n> diff --git a/t/t4211-line-log.sh b/t/t4211-line-log.sh\n> index 1428eae262..e186c83250 100755\n> --- a/t/t4211-line-log.sh\n> +++ b/t/t4211-line-log.sh\n> @@ -240,10 +240,12 @@ test_expect_success 'setup for checking line-log and parent oids' '\n> \tEOF\n> \tgit add file.c &&\n> \ttest_tick &&\n> +\tfirst_tick=$test_tick &&\n> \tgit commit -m \"Add func1() and func2() in file.c\" &&\n>\n> \techo 1 >other-file &&\n> \tgit add other-file &&\n> +\ttest_tick &&\n> \tgit commit -m \"Add other-file\" &&\n>\n> \tsed -e \"s/F1/F1 + 1/\" file.c >tmp &&\n> @@ -283,4 +285,10 @@ test_expect_success 'parent oids with parent rewriting' '\n> \ttest_cmp expect actual\n> '\n>\n> +test_expect_success 'line-log with --before' '\n> +\techo $root_oid >expect &&\n> +\tgit log --format=%h --no-patch -L:func2:file.c --before=$first_tick >actual &&\n> +\ttest_cmp expect actual\n> +'\n> +\n> test_done\n> --\n> 2.27.0\n"}]}