threads / discuss / 53797

Git log: filtering by date and by lines do not work together

Subject: Git log: filtering by date and by lines do not work together

## tl;dr

4 messages between Jul 3, 2020 and Jul 7, 2020.

replies: 3people: 3as markdown or json

Мария Долгополова· Jul 3, 2020, 14:14 UTC · lore
Hi! I would like to report the bug.

I want to get commits that change specific lines in a specific file and were made before a specific date.

Steps:
1) I apply filtering by date: --before="2016-11-18"
Result is in the picture FilteringByDate.png. Commit
570778797effd69bd4536c51125b7d2e8f654c08 is presented.
2) I apply filtering by lines:
-L33,34:src/main/java/org/toradocu/translator/Subject.java
Result is in the picture FilteringByLines.png . Commit
570778797effd69bd4536c51125b7d2e8f654c08 is presented.
3) Finally, I apply filtering by lines and by date:
--before="2016-11-18"
-L33,34:src/main/java/org/toradocu/translator/Subject.java
Result is in the picture FilteringByDatesAndLines.png . Commit
570778797effd69bd4536c51125b7d2e8f654c08 is NOT presented.

Expected Result: the commit is presented in filter 1 and in filter 2, so it must be presented in filter “1 AND 2”

Actual Result: the commit is NOT presented in filter “1 AND 2”
Environment: git version 2.27.0.windows.1
 I beg you to provide me with a workaround. I am doing research work
on analyzing commits in open source projects. Therefore, this
functionality is very important to me.
René Scharfe· Jul 4, 2020, 12:56 UTC · re: Мария Долгополова · lore

Re: Git log: filtering by date and by lines do not work together

Am 03.07.20 um 16:14 schrieb Мария Долгополова:
Show 31 quoted lines
> Hi! I would like to report the bug.
>
> I want to get commits that change specific lines in a specific file
> and were made before a specific date.
>
> Steps:
> 1) I apply filtering by date: --before="2016-11-18"
> Result is in the picture FilteringByDate.png. Commit
> 570778797effd69bd4536c51125b7d2e8f654c08 is presented.
>
> 2) I apply filtering by lines:
> -L33,34:src/main/java/org/toradocu/translator/Subject.java
> Result is in the picture FilteringByLines.png . Commit
> 570778797effd69bd4536c51125b7d2e8f654c08 is presented.
>
> 3) Finally, I apply filtering by lines and by date:
> --before="2016-11-18"
> -L33,34:src/main/java/org/toradocu/translator/Subject.java
> Result is in the picture FilteringByDatesAndLines.png . Commit
> 570778797effd69bd4536c51125b7d2e8f654c08 is NOT presented.
>
> Expected Result: the commit is presented in filter 1 and in filter 2,
> so it must be presented in filter “1 AND 2”
>
> Actual Result: the commit is NOT presented in filter “1 AND 2”
>
> Environment: git version 2.27.0.windows.1
>
>  I beg you to provide me with a workaround. I am doing research work
> on analyzing commits in open source projects. Therefore, this
> functionality is very important to me.
You could do the date filtering on the output of git log, e.g.:
  git log -L33,34:src/main/java/org/toradocu/translator/Subject.java --date=short --color=always |
  awk -v before=2016-11-18 '
    /^[^ ]*commit/ {state=1; header=""}
    /^[^ ]*Date:/ {if ($2 <= before) {state=2; printf "%s", header} else {state=0}}
    state == 1 {header = header $0 ORS}
    state == 2 {print}
  '

This command is a bit unwieldy, so you might want to put the AWK script into a file, then you could use it like this:

  git log -L33,34:src/main/java/org/toradocu/translator/Subject.java --date=short --color=always |
  awk -v before=2016-11-18 -f gitlogbefore.awk
René
René Scharfe· Jul 4, 2020, 12:56 UTC · re: Мария Долгополова · lore

[PATCH] revision: disable min_age optimization with line-log

If one of the options --before, --min-age or --until is given, limit_list() filters out younger commits early on. Line-log needs all those commits to trace the movement of line ranges, though. Skip this optimization if both are used together.

Reported-by: Мария Долгополова <dolgopolovamariia@gmail.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
Needs careful review -- I'm not familiar with that the line-log code and
the revision traversal machinery is a bit scary.  AFAIU the min_age
check is done again in get_commit_action(), so this patch shouldn't
cause underage commits to be shown, but I'm not sure.  Test coverage for
the three options is spotty. :-/
 revision.c          | 3 ++-
 t/t4211-line-log.sh | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/revision.c b/revision.c
index ebb4d2a0f2..3bdc1bbf2a 100644
--- a/revision.c
+++ b/revision.c
@@ -1410,7 +1410,8 @@ static int limit_list(struct rev_info *revs)
 				continue;
 			break;
 		}
-		if (revs->min_age != -1 && (commit->date > revs->min_age))
+		if (revs->min_age != -1 && (commit->date > revs->min_age) &&
+		    !revs->line_level_traverse)
 			continue;
 		date = commit->date;
 		p = &commit_list_insert(commit, p)->next;
diff --git a/t/t4211-line-log.sh b/t/t4211-line-log.sh
index 1428eae262..e186c83250 100755
--- a/t/t4211-line-log.sh
+++ b/t/t4211-line-log.sh
@@ -240,10 +240,12 @@ test_expect_success 'setup for checking line-log and parent oids' '
 	EOF
 	git add file.c &&
 	test_tick &&
+	first_tick=$test_tick &&
 	git commit -m "Add func1() and func2() in file.c" &&

 	echo 1 >other-file &&
 	git add other-file &&
+	test_tick &&
 	git commit -m "Add other-file" &&

 	sed -e "s/F1/F1 + 1/" file.c >tmp &&
@@ -283,4 +285,10 @@ test_expect_success 'parent oids with parent rewriting' '
 	test_cmp expect actual
 '

+test_expect_success 'line-log with --before' '
+	echo $root_oid >expect &&
+	git log --format=%h --no-patch -L:func2:file.c --before=$first_tick >actual &&
+	test_cmp expect actual
+'
+
 test_done
--
2.27.0
Junio C Hamano· Jul 7, 2020, 01:37 UTC · re: René Scharfe · lore

Re: [PATCH] revision: disable min_age optimization with line-log

René Scharfe <l.s.r@web.de> writes:
Show 13 quoted lines
> If one of the options --before, --min-age or --until is given,
> limit_list() filters out younger commits early on.  Line-log needs all
> those commits to trace the movement of line ranges, though.  Skip this
> optimization if both are used together.
>
> Reported-by: Мария Долгополова <dolgopolovamariia@gmail.com>
> Signed-off-by: René Scharfe <l.s.r@web.de>
> ---
> Needs careful review -- I'm not familiar with that the line-log code and
> the revision traversal machinery is a bit scary.  AFAIU the min_age
> check is done again in get_commit_action(), so this patch shouldn't
> cause underage commits to be shown, but I'm not sure.  Test coverage for
> the three options is spotty. :-/

I am not familiar with the line-log code, either, but anyway. This starts queuing commits that are too young in the list of commits to be processed, but later stages in the output callchain would decide that they are too young to be shown in get_commit_action() that is called from simplify_commit(), so the overall effect of this change is to show these commits to the line-log machinery but filter them out of the final output phase (with the parent rewriting, the line-log machinery sees these commits in prepare_revision_walk(), and without it, each of these commits is shown to the line-log machinery immediately before get_commit_action() decides that it is too young to be shown). And the effect you want to gain by showing these commits to the line-log machinery is to ensure that the range of lines in the starting point is adjusted for the changes each of these commits makes.

Which makes sense to me.
Thanks, will queue.
Show 48 quoted lines
>  revision.c          | 3 ++-
>  t/t4211-line-log.sh | 8 ++++++++
>  2 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/revision.c b/revision.c
> index ebb4d2a0f2..3bdc1bbf2a 100644
> --- a/revision.c
> +++ b/revision.c
> @@ -1410,7 +1410,8 @@ static int limit_list(struct rev_info *revs)
>  				continue;
>  			break;
>  		}
> -		if (revs->min_age != -1 && (commit->date > revs->min_age))
> +		if (revs->min_age != -1 && (commit->date > revs->min_age) &&
> +		    !revs->line_level_traverse)
>  			continue;
>  		date = commit->date;
>  		p = &commit_list_insert(commit, p)->next;
> diff --git a/t/t4211-line-log.sh b/t/t4211-line-log.sh
> index 1428eae262..e186c83250 100755
> --- a/t/t4211-line-log.sh
> +++ b/t/t4211-line-log.sh
> @@ -240,10 +240,12 @@ test_expect_success 'setup for checking line-log and parent oids' '
>  	EOF
>  	git add file.c &&
>  	test_tick &&
> +	first_tick=$test_tick &&
>  	git commit -m "Add func1() and func2() in file.c" &&
>
>  	echo 1 >other-file &&
>  	git add other-file &&
> +	test_tick &&
>  	git commit -m "Add other-file" &&
>
>  	sed -e "s/F1/F1 + 1/" file.c >tmp &&
> @@ -283,4 +285,10 @@ test_expect_success 'parent oids with parent rewriting' '
>  	test_cmp expect actual
>  '
>
> +test_expect_success 'line-log with --before' '
> +	echo $root_oid >expect &&
> +	git log --format=%h --no-patch -L:func2:file.c --before=$first_tick >actual &&
> +	test_cmp expect actual
> +'
> +
>  test_done
> --
> 2.27.0

← back to recent threads