threads / discuss / 29411

Finding all commits which modify a file

Subject: Finding all commits which modify a file

## tl;dr

14 messages between Jan 20, 2012 and Jan 26, 2012.

replies: 13people: 6as markdown or json

Neal Groothuis· Jan 20, 2012, 21:35 UTC · lore
Hello,

I'm trying to find /all/ commits that change a file in the repository...and its proving to be trickier than I thought. :-)

The situation that we were dealing with is this:
- Person A and person B both pull from the same central repository.
- Person A makes a change to file foo.txt and bar.txt, commits, and pushes
to the central repository.
- Person B makes a similar change to bar.txt and commits it.
- Person B does a fetch and merge.  Since both A and B made changes to
bar.txt, this requires conflicts to be resolved manually.
- B reverts A's changes to foo.txt. (If B is coming from a different
revision control system, this may happen due to confusion about how merges
are handled.)
- B commits the changes.
- B makes more changes to bar.txt, commits them, and pushes to the central
repository.
At this point, A's changes to foo.txt have been undone.
Graphically:
    A1
   /  ^
  v    \
  C1   B2<-B3
  ^    /
   \  v
    B1
B1, B2, and B3 have the same version of foo.txt as C1, A1 modifies it.

Person A discovers that his changes are missing and wants to know what happened.

git log foo.txt doesn't help; it won't even show commit A1, due to history simplification.

git log --full-history foo.txt will show commit A1. It still won't show commit B2, though, which we'd also like to show (because that's where the change to foo.txt got removed).

I would think that git log --simplify-merges foo.txt would have done what I'd wanted, but it still does not show commit B2. Based on what I'm reading in the man page, I would expect the simplification to go like this:

    A1
    | ^
    |  \
    |  B2<-B3
    |  /
    v v
    C1
(since B1 is TREESAME as C1 if we're only considering foo.txt)
    A1
    | ^
    |  \
    |  B2<-B3
    |
    v
    C1
(since C1 is an ancestor of A1)
However, the actual output only includes A1, not B2.
 - Can someone explain this, and/or
 - can someone offer a command to display all commits (including merges)
in which ANY parent is not TREESAME?
Thanks!
- Neal
Neal Kreitzinger· Jan 21, 2012, 23:16 UTC · re: Neal Groothuis · lore

Re: Finding all commits which modify a file

On 1/20/2012 3:35 PM, Neal Groothuis wrote:
Show 81 quoted lines
> Hello,
>
> I'm trying to find /all/ commits that change a file in the
> repository...and its proving to be trickier than I thought. :-)
>
> The situation that we were dealing with is this:
>
> - Person A and person B both pull from the same central repository.
>
> - Person A makes a change to file foo.txt and bar.txt, commits, and pushes
> to the central repository.
>
> - Person B makes a similar change to bar.txt and commits it.
>
> - Person B does a fetch and merge.  Since both A and B made changes to
> bar.txt, this requires conflicts to be resolved manually.
>
> - B reverts A's changes to foo.txt. (If B is coming from a different
> revision control system, this may happen due to confusion about how merges
> are handled.)
>
> - B commits the changes.
>
> - B makes more changes to bar.txt, commits them, and pushes to the central
> repository.
>
> At this point, A's changes to foo.txt have been undone.
>
> Graphically:
>
>      A1
>     /  ^
>    v    \
>    C1   B2<-B3
>    ^    /
>     \  v
>      B1
>
> B1, B2, and B3 have the same version of foo.txt as C1, A1 modifies it.
>
> Person A discovers that his changes are missing and wants to know what
> happened.
>
> git log foo.txt doesn't help; it won't even show commit A1, due to history
> simplification.
>
> git log --full-history foo.txt will show commit A1.  It still won't show
> commit B2, though, which we'd also like to show (because that's where the
> change to foo.txt got removed).
>
> I would think that git log --simplify-merges foo.txt would have done what
> I'd wanted, but it still does not show commit B2.   Based on what I'm
> reading in the man page, I would expect the simplification to go like
> this:
>
>      A1
>      | ^
>      |  \
>      |  B2<-B3
>      |  /
>      v v
>      C1
>
> (since B1 is TREESAME as C1 if we're only considering foo.txt)
>
>      A1
>      | ^
>      |  \
>      |  B2<-B3
>      |
>      v
>      C1
>
> (since C1 is an ancestor of A1)
>
> However, the actual output only includes A1, not B2.
>
>   - Can someone explain this, and/or
>   - can someone offer a command to display all commits (including merges)
> in which ANY parent is not TREESAME?
>
Does git-log --all help?

v/r, neal

Neal Groothuis· Jan 23, 2012, 16:14 UTC · re: Neal Kreitzinger · lore

Re: Finding all commits which modify a file

> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>> I'm trying to find /all/ commits that change a file in the
>> repository...and its proving to be trickier than I thought. :-)
On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
> Does git-log --all help?

I don't see how it would. The commits are all reachable from HEAD, which would seem to be the problem that --all would correct.

What I'm trying to do is find the commits in which a file differs from that same file in any of its parents.

If I'm missing something, could you provide an example of using git-log --all to accomplish this?

Santi Béjar· Jan 24, 2012, 00:58 UTC · re: Neal Groothuis · lore

Re: Finding all commits which modify a file

[Note: CC main authors of the code surrounding the patch]
On Mon, Jan 23, 2012 at 5:14 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
Show 12 quoted lines
>> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>>> I'm trying to find /all/ commits that change a file in the
>>> repository...and its proving to be trickier than I thought. :-)
>
> On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
>> Does git-log --all help?
>
> I don't see how it would.  The commits are all reachable from HEAD, which
> would seem to be the problem that --all would correct.
>
> What I'm trying to do is find the commits in which a file differs from
> that same file in any of its parents.

If you add parent rewriting (--parent, --graph or see it in gitk, with --full-history) you'll get your B2 commit as it adds commits to have a meaningful history. But I don't think this is what you are asking for.

  You could try the following patch (sorry for the whitespace damage,
also attatched):
Subject: [PATCH/RFC] revision: merging branches with different content
is interesting in --full-history
---
 revision.c                                 |    2 +-
 t/t6016-rev-list-graph-simplify-history.sh |    1 +
 2 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/revision.c b/revision.c
index 064e351..db97250 100644
--- a/revision.c
+++ b/revision.c
@@ -492,7 +492,7 @@ static void try_to_simplify_commit(struct rev_info
*revs, struct commit *commit)
                }
                die("bad tree compare for commit %s",
sha1_to_hex(commit->object.sha1));
        }
-       if (tree_changed && !tree_same)
+       if ((tree_changed && !tree_same) || (!revs->simplify_history
&& tree_changed))
                return;
        commit->object.flags |= TREESAME;
 }
diff --git a/t/t6016-rev-list-graph-simplify-history.sh
b/t/t6016-rev-list-graph-simplify-history.sh
index f7181d1..50ffcf4 100755
--- a/t/t6016-rev-list-graph-simplify-history.sh
+++ b/t/t6016-rev-list-graph-simplify-history.sh
@@ -168,6 +168,7 @@ test_expect_success '--graph --full-history
--simplify-merges -- bar.txt' '
        echo "|\\  " >> expected &&
        echo "| * $C4" >> expected &&
        echo "* | $A5" >> expected &&
+       echo "* | $A4" >> expected &&
        echo "* | $A3" >> expected &&
        echo "|/  " >> expected &&
        echo "* $A2" >> expected &&

(I could rewrite the condition but I think it is cleaner).

This patch changes the semantics of --full-history to consider all
commits with at least one modified parent as an interesting commit
(even for merges). This is almost as enabling --parent:

git $ git rev-list --full-history HEAD Makefile | wc -l
1769

$ git rev-list --full-history --parents HEAD Makefile | wc -l
6732

git $ ./git rev-list --full-history HEAD Makefile | wc -l
6052

I think that --full-history should list these extra merges as you are
asking for the full history and a merge merging two branches with
different content is an interesting event in this case (full history).
But maybe we should just add an extra flag...

HTH,
Santi


From 36a09f1a39212eb4b45268215cc9c336a7afebda Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Santi=20B=C3=A9jar?= <santi@agolina.net>
Date: Tue, 24 Jan 2012 00:46:23 +0100
Subject: [PATCH] revision: merging branches with different content is interesting in --full-history

---
 revision.c                                 |    2 +-
 t/t6016-rev-list-graph-simplify-history.sh |    1 +
 2 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/revision.c b/revision.c
index 064e351..db97250 100644
--- a/revision.c
+++ b/revision.c
@@ -492,7 +492,7 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 		}
 		die("bad tree compare for commit %s", sha1_to_hex(commit->object.sha1));
 	}
-	if (tree_changed && !tree_same)
+	if ((tree_changed && !tree_same) || (!revs->simplify_history && tree_changed))
 		return;
 	commit->object.flags |= TREESAME;
 }
diff --git a/t/t6016-rev-list-graph-simplify-history.sh b/t/t6016-rev-list-graph-simplify-history.sh
index f7181d1..50ffcf4 100755
--- a/t/t6016-rev-list-graph-simplify-history.sh
+++ b/t/t6016-rev-list-graph-simplify-history.sh
@@ -168,6 +168,7 @@ test_expect_success '--graph --full-history --simplify-merges -- bar.txt' '
 	echo "|\\  " >> expected &&
 	echo "| * $C4" >> expected &&
 	echo "* | $A5" >> expected &&
+	echo "* | $A4" >> expected &&
 	echo "* | $A3" >> expected &&
 	echo "|/  " >> expected &&
 	echo "* $A2" >> expected &&

-- 
1.7.4.rc3.8.gd2d4
Santi Béjar· Jan 24, 2012, 01:15 UTC · re: Santi Béjar · lore

Re: Finding all commits which modify a file

On Tue, Jan 24, 2012 at 1:58 AM, Santi Béjar <santi@agolina.net> wrote:
Show 19 quoted lines
> [Note: CC main authors of the code surrounding the patch]
>
> On Mon, Jan 23, 2012 at 5:14 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
>>> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>>>> I'm trying to find /all/ commits that change a file in the
>>>> repository...and its proving to be trickier than I thought. :-)
>>
>> On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
>>> Does git-log --all help?
>>
>> I don't see how it would.  The commits are all reachable from HEAD, which
>> would seem to be the problem that --all would correct.
>>
>> What I'm trying to do is find the commits in which a file differs from
>> that same file in any of its parents.
>
> If you add parent rewriting (--parent, --graph or see it in gitk, with
> --full-history) you'll get your B2 commit as it adds commits to have a
> meaningful history. But I don't think this is what you are asking for.

Note that even if it get listed, you won't get a diff for foo.txt because it is an evil merge as the result is not the expected (using the three way merge) one but is equal to one of the branches.

HTH, Santi

Linus Torvalds· Jan 24, 2012, 01:15 UTC · re: Santi Béjar · lore

Re: Finding all commits which modify a file

On Mon, Jan 23, 2012 at 4:58 PM, Santi Béjar <santi@agolina.net> wrote:
Show 10 quoted lines
>
> If you add parent rewriting (--parent, --graph or see it in gitk, with
> --full-history) you'll get your B2 commit as it adds commits to have a
> meaningful history. But I don't think this is what you are asking for.
>
>  You could try the following patch (sorry for the whitespace damage,
> also attatched):
>
> Subject: [PATCH/RFC] revision: merging branches with different content
> is interesting in --full-history
The concept seems sane.

But please check the interaction with "--simplify-merges" too, just in case. The merge simplification looks at TREESAME too, so I suspect your change may break merge simplification.

                      Linus
Santi Béjar· Jan 24, 2012, 01:36 UTC · re: Linus Torvalds · lore

Re: Finding all commits which modify a file

On Tue, Jan 24, 2012 at 2:15 AM, Linus Torvalds <torvalds@linux-foundation.org> wrote:

Show 17 quoted lines
> On Mon, Jan 23, 2012 at 4:58 PM, Santi Béjar <santi@agolina.net> wrote:
>>
>> If you add parent rewriting (--parent, --graph or see it in gitk, with
>> --full-history) you'll get your B2 commit as it adds commits to have a
>> meaningful history. But I don't think this is what you are asking for.
>>
>>  You could try the following patch (sorry for the whitespace damage,
>> also attatched):
>>
>> Subject: [PATCH/RFC] revision: merging branches with different content
>> is interesting in --full-history
>
> The concept seems sane.
>
> But please check the interaction with "--simplify-merges" too, just in
> case. The merge simplification looks at TREESAME too, so I suspect
> your change may break merge simplification.

Indeed, there is a bad interaction with --simplify-merges. If you add --simplify-merges it not only increase the number of commit but changes them :-(

$ ./git rev-list --full-history --simplify-merges HEAD Makefile >
rev-list.simp-merges
$ ./git rev-list --full-history HEAD Makefile > rev-list.new
$ diff rev-list.new rev-list.simp-merges -u | diffstat
 rev-list.simp-merges | 1841 ++++++++++++++++++++++++++-------------------------
 1 file changed, 944 insertions(+), 897 deletions(-)
Santi
Santi Béjar· Jan 24, 2012, 18:35 UTC · re: Santi Béjar · lore

Re: Finding all commits which modify a file

On Tue, Jan 24, 2012 at 2:36 AM, Santi Béjar <santi@agolina.net> wrote:
Show 30 quoted lines
> On Tue, Jan 24, 2012 at 2:15 AM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>> On Mon, Jan 23, 2012 at 4:58 PM, Santi Béjar <santi@agolina.net> wrote:
>>>
>>> If you add parent rewriting (--parent, --graph or see it in gitk, with
>>> --full-history) you'll get your B2 commit as it adds commits to have a
>>> meaningful history. But I don't think this is what you are asking for.
>>>
>>>  You could try the following patch (sorry for the whitespace damage,
>>> also attatched):
>>>
>>> Subject: [PATCH/RFC] revision: merging branches with different content
>>> is interesting in --full-history
>>
>> The concept seems sane.
>>
>> But please check the interaction with "--simplify-merges" too, just in
>> case. The merge simplification looks at TREESAME too, so I suspect
>> your change may break merge simplification.
>
> Indeed, there is a bad interaction with --simplify-merges. If you add
> --simplify-merges it not only increase the number of commit but
> changes them :-(
>
> $ ./git rev-list --full-history --simplify-merges HEAD Makefile >
> rev-list.simp-merges
> $ ./git rev-list --full-history HEAD Makefile > rev-list.new
> $ diff rev-list.new rev-list.simp-merges -u | diffstat
>  rev-list.simp-merges | 1841 ++++++++++++++++++++++++++-------------------------
>  1 file changed, 944 insertions(+), 897 deletions(-)
Ops, it even happens without my patch...

I think it is OK, it just redefines what is TREESAME, and use the new meaning in:

* If after this parent rewriting, `C'` is a root or merge commit (has
  zero or >1 parents), a boundary commit, or !TREESAME, it remains.
  Otherwise, it is replaced with its only parent.

We could keep the old meaning if --simplify-merges or we could have a flag to just change the meaning of TREESAME for merges (--with-all-interesting-merges? I'm not good at naming flags...)

Santi
Junio C Hamano· Jan 24, 2012, 01:40 UTC · re: Santi Béjar · lore

Re: Finding all commits which modify a file

Santi Béjar <santi@agolina.net> writes:
Show 13 quoted lines
> diff --git a/revision.c b/revision.c
> index 064e351..db97250 100644
> --- a/revision.c
> +++ b/revision.c
> @@ -492,7 +492,7 @@ static void try_to_simplify_commit(struct rev_info
> *revs, struct commit *commit)
>                 }
>                 die("bad tree compare for commit %s",
> sha1_to_hex(commit->object.sha1));
>         }
> -       if (tree_changed && !tree_same)
> +       if ((tree_changed && !tree_same) || (!revs->simplify_history
> && tree_changed))
Is that the same as saying this?
	if (!(tree_same && revs->simplify_history) && tree_changed)
		return;

Which reads: unless we find a parent that matches the result and we are simplifying the history, a child different from at least one parent is worth showing.

Which makes sort of sense at the conceptual level, but I am not sure if it is practically useful (the same issue with --full-history that makes complex history almost unreadable).

Neal Groothuis· Jan 24, 2012, 16:34 UTC · re: Santi Béjar · lore

Re: Finding all commits which modify a file

Show 18 quoted lines
> On Mon, Jan 23, 2012 at 5:14 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
>>> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>>>> I'm trying to find /all/ commits that change a file in the
>>>> repository...and its proving to be trickier than I thought. :-)
>>
>> On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
>>> Does git-log --all help?
>>
>> I don't see how it would.  The commits are all reachable from HEAD,
>> which
>> would seem to be the problem that --all would correct.
>>
>> What I'm trying to do is find the commits in which a file differs from
>> that same file in any of its parents.
>
> If you add parent rewriting (--parent, --graph or see it in gitk, with
> --full-history) you'll get your B2 commit as it adds commits to have a
> meaningful history. But I don't think this is what you are asking for.

Correct. If I add parent rewriting, I get all merges, even those in which the file is not changed from either parent.

Based on what's in the man page for git log about the history simplification algorithm, it seems that B2 should be included in the output when I do a git log --full-history --simplify-history foo.txt, as per the steps I noted in the original post. Is my understanding of the algorithm faulty?

Santi Béjar· Jan 24, 2012, 18:10 UTC · re: Neal Groothuis · lore

Re: Finding all commits which modify a file

[ Do not cut the CC]
On Tue, Jan 24, 2012 at 5:34 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
Show 28 quoted lines
>> On Mon, Jan 23, 2012 at 5:14 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
>>>> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>>>>> I'm trying to find /all/ commits that change a file in the
>>>>> repository...and its proving to be trickier than I thought. :-)
>>>
>>> On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
>>>> Does git-log --all help?
>>>
>>> I don't see how it would.  The commits are all reachable from HEAD,
>>> which
>>> would seem to be the problem that --all would correct.
>>>
>>> What I'm trying to do is find the commits in which a file differs from
>>> that same file in any of its parents.
>>
>> If you add parent rewriting (--parent, --graph or see it in gitk, with
>> --full-history) you'll get your B2 commit as it adds commits to have a
>> meaningful history. But I don't think this is what you are asking for.
>
> Correct.  If I add parent rewriting, I get all merges, even those in which
> the file is not changed from either parent.
>
> Based on what's in the man page for git log about the history
> simplification algorithm, it seems that B2 should be included in the
> output when I do a git log --full-history --simplify-history foo.txt, as
> per the steps I noted in the original post.  Is my understanding of the
> algorithm faulty?
>

Following your steps in the first post, B2 is excluded in the --simplify-merge phase because it is (originally) TREESAME, even if it is not in the rewritten history...

HTH, Santi

Neal Groothuis· Jan 25, 2012, 16:23 UTC · re: Santi Béjar · lore

Re: Finding all commits which modify a file

> [ Do not cut the CC]
My apologies.
Show 35 quoted lines
> On Tue, Jan 24, 2012 at 5:34 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
>>> On Mon, Jan 23, 2012 at 5:14 PM, Neal Groothuis <ngroot@lo-cal.org>
>>> wrote:
>>>>> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>>>>>> I'm trying to find /all/ commits that change a file in the
>>>>>> repository...and its proving to be trickier than I thought. :-)
>>>>
>>>> On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
>>>>> Does git-log --all help?
>>>>
>>>> I don't see how it would.  The commits are all reachable from HEAD,
>>>> which
>>>> would seem to be the problem that --all would correct.
>>>>
>>>> What I'm trying to do is find the commits in which a file differs from
>>>> that same file in any of its parents.
>>>
>>> If you add parent rewriting (--parent, --graph or see it in gitk, with
>>> --full-history) you'll get your B2 commit as it adds commits to have a
>>> meaningful history. But I don't think this is what you are asking for.
>>
>> Correct.  If I add parent rewriting, I get all merges, even those in
>> which
>> the file is not changed from either parent.
>>
>> Based on what's in the man page for git log about the history
>> simplification algorithm, it seems that B2 should be included in the
>> output when I do a git log --full-history --simplify-history foo.txt, as
>> per the steps I noted in the original post.  Is my understanding of the
>> algorithm faulty?
>>
>
> Following your steps in the first post, B2 is excluded in the
> --simplify-merge phase because it is (originally) TREESAME, even if it
> is not in the rewritten history...

Thanks, I see---labeling a commit as TREESAME happens before simplification, rather than after.

In my example, that results in a simplified history where a commit in which the contents of the specified paths change gets removed. That seems perverse; I would think the utility of a simplified history would be to track down the commits in which the contents of the specified paths change without having to consider ones in which they do not.

Is there a situation where checking for TREESAMEness before simplification is desirable and checking after would not be?

- Neal
Junio C Hamano· Jan 26, 2012, 22:42 UTC · re: Neal Groothuis · lore

Re: Finding all commits which modify a file

"Neal Groothuis" <ngroot@lo-cal.org> writes:
> Is there a situation where checking for TREESAMEness before simplification
> is desirable and checking after would not be?

When you do not want to see a side branch that does not contribute to the end result at all, obviously ;-). Outside that situation, before or after should not make a difference, I would think.

Tay Ray Chuan· Jan 22, 2012, 04:03 UTC · re: Neal Groothuis · lore

Re: Finding all commits which modify a file

On Sat, Jan 21, 2012 at 5:35 AM, Neal Groothuis <ngroot@lo-cal.org> wrote:
Show 20 quoted lines
> Hello,
>
> I'm trying to find /all/ commits that change a file in the
> repository...and its proving to be trickier than I thought. :-)
>
> The situation that we were dealing with is this:
>
> - Person A and person B both pull from the same central repository.
>
> - Person A makes a change to file foo.txt and bar.txt, commits, and pushes
> to the central repository.
>
> - Person B makes a similar change to bar.txt and commits it.
>
> - Person B does a fetch and merge.  Since both A and B made changes to
> bar.txt, this requires conflicts to be resolved manually.
>
> - B reverts A's changes to foo.txt. (If B is coming from a different
> revision control system, this may happen due to confusion about how merges
> are handled.)

How is this "revert" done? Was it done at the conflict resolution level or with a git-revert invocation?

Nonetheless, either way, A's commit would be still be present in the log history.

Show 12 quoted lines
>[snip]
> Graphically:
>
>    A1
>   /  ^
>  v    \
>  C1   B2<-B3
>  ^    /
>   \  v
>    B1
>
> B1, B2, and B3 have the same version of foo.txt as C1, A1 modifies it.

Just to clarify, is C1 the commit that both A and B both share when they first pull in the first step? And B2 is the merge?

-- 
Cheers,
Ray Chuan

← back to recent threads