threads / discuss / 25799

First/oldest entry in reflog dropped

Subject: First/oldest entry in reflog dropped

## tl;dr

8 messages between Nov 21, 2010 and Nov 25, 2010.

replies: 7people: 4as markdown or json

Martin von Zweigbergk· Nov 21, 2010, 03:11 UTC · lore
Can someone explain the behavior in the execution below?

# I expected this reflog... $ git branch tmp $ git reflog show refs/heads/tmp b60a214 refs/heads/tmp@{0}: branch: Created from master

# ... and this one as well... $ git update-ref refs/heads/tmp HEAD^ $ git reflog show refs/heads/tmp 7d1a0b8 refs/heads/tmp@{0}: b60a214 refs/heads/tmp@{1}: branch: Created from master

# ... but why is the first entry (i.e. "branch: Created from master") # dropped here? $ git update-ref refs/heads/tmp HEAD $ git reflog show refs/heads/tmp b60a214 refs/heads/tmp@{0}: 7d1a0b8 refs/heads/tmp@{1}:

If the ref is updated once more (to e.g. HEAD^^) before being moved back to HEAD, the first entry will be shown in the output.

If this is a bug, it seems to be in reflog, rather than in update-ref, because the first entry does exist in .git/logs/refs/heads/tmp.

Thanks, Martin

Jeff King· Nov 21, 2010, 05:35 UTC · re: Martin von Zweigbergk · lore

Re: First/oldest entry in reflog dropped

On Sat, Nov 20, 2010 at 10:11:34PM -0500, Martin von Zweigbergk wrote:
Show 25 quoted lines
> Can someone explain the behavior in the execution below?
> 
> # I expected this reflog...
> $ git branch tmp
> $ git reflog show refs/heads/tmp
> b60a214 refs/heads/tmp@{0}: branch: Created from master
> 
> # ... and this one as well...
> $ git update-ref refs/heads/tmp HEAD^
> $ git reflog show refs/heads/tmp
> 7d1a0b8 refs/heads/tmp@{0}:
> b60a214 refs/heads/tmp@{1}: branch: Created from master
> 
> # ... but why is the first entry (i.e. "branch: Created from master")
> # dropped here?
> $ git update-ref refs/heads/tmp HEAD
> $ git reflog show refs/heads/tmp
> b60a214 refs/heads/tmp@{0}:
> 7d1a0b8 refs/heads/tmp@{1}:
> 
> If the ref is updated once more (to e.g. HEAD^^) before being moved back
> to HEAD, the first entry will be shown in the output.
> 
> If this is a bug, it seems to be in reflog, rather than in update-ref,
> because the first entry does exist in .git/logs/refs/heads/tmp.

I think it's a bug in the reflog-walking machinery, which is sort of bolted onto the regular revision traversal machinery. When we hit b60a214 the first time, we show it and set the SHOWN flag (since the normal traversal machinery would not want to show a commit twice). When we hit it again, simplify_commit() sees that it is SHOWN and tells us to skip it.

However, the bolted-on reflog-walking machinery does have a way of handling this. While we are traversing via get_revision(), we notice that we are doing a reflog walk and call fake_reflog_parent. This function is responsible for replacing the actual parents of the commit with a fake list consisting of the previous reflog entry (so we basically pretend that the history consists of a string of commits, each one pointing to the previous reflog entry, not the actual parent).

This function _also_ clears some flags, including the SHOWN flag, in what almost seems like a tacked-on side effect. So if we hit the same commit twice, we will actually show it again. Which is what makes reflogs with repeated commits work at all.

However, there is a subtle bug: it clears the flags at the very end of the function. But through the function, if we see that there are no fake parents (because we are on the very first reflog entry), we do an early return. But we not only skip the later "set up parents" code, we also accidentally skip the "clear SHOWN flag" side-effect code.

So I believe we will always fail to show the very first reflog if it is a repeated commit.

The fix, AFAICT, is to just move the flag clearing above the early returns (patch below). But I have to admit I do not quite understand what the ADDED and SEEN flags are doing here, as this is the first time I have ever looked at the reflog-walk code. So possibly just the SHOWN flag should be unconditionally cleared.

This patch clears up your bug, and doesn't break any tests. But I'd really like to get a second opinion on the significance of those other flags, or why the flag clearing was at the bottom of the function in the first place.

-Peff
---
diff --git a/reflog-walk.c b/reflog-walk.c
index 4879615..d5d055b 100644
--- a/reflog-walk.c
+++ b/reflog-walk.c
@@ -210,44 +210,45 @@ int add_reflog_for_walk(struct reflog_walk_info *info,
 	add_commit_info(commit, commit_reflog, &info->reflogs);
 	return 0;
 }
 
 void fake_reflog_parent(struct reflog_walk_info *info, struct commit *commit)
 {
 	struct commit_info *commit_info =
 		get_commit_info(commit, &info->reflogs, 0);
 	struct commit_reflog *commit_reflog;
 	struct reflog_info *reflog;
 
+	commit->object.flags &= ~(ADDED | SEEN | SHOWN);
+
 	info->last_commit_reflog = NULL;
 	if (!commit_info)
 		return;
 
 	commit_reflog = commit_info->util;
 	if (commit_reflog->recno < 0) {
 		commit->parents = NULL;
 		return;
 	}
 
 	reflog = &commit_reflog->reflogs->items[commit_reflog->recno];
 	info->last_commit_reflog = commit_reflog;
 	commit_reflog->recno--;
 	commit_info->commit = (struct commit *)parse_object(reflog->osha1);
 	if (!commit_info->commit) {
 		commit->parents = NULL;
 		return;
 	}
 
 	commit->parents = xcalloc(sizeof(struct commit_list), 1);
 	commit->parents->item = commit_info->commit;
-	commit->object.flags &= ~(ADDED | SEEN | SHOWN);
 }
 
 void get_reflog_selector(struct strbuf *sb,
 			 struct reflog_walk_info *reflog_info,
 			 enum date_mode dmode,
 			 int shorten)
 {
 	struct commit_reflog *commit_reflog = reflog_info->last_commit_reflog;
 	struct reflog_info *info;
 	const char *printed_ref;
 
Johannes Schindelin· Nov 21, 2010, 11:36 UTC · re: Jeff King · lore

Re: First/oldest entry in reflog dropped

Hi,
On Sun, 21 Nov 2010, Jeff King wrote:
> This patch clears up your bug, and doesn't break any tests. But I'd 
> really like to get a second opinion on the significance of those other 
> flags, or why the flag clearing was at the bottom of the function in the 
> first place.

The flag clearing was at the bottom because I had the impression that something function one might want to call in that function in the future could set the flags again. Maybe a goto would be appropriate here instead of the early returns?

Ciao, Dscho

Jeff King· Nov 22, 2010, 04:42 UTC · re: Johannes Schindelin · lore

Re: First/oldest entry in reflog dropped

On Sun, Nov 21, 2010 at 12:36:21PM +0100, Johannes Schindelin wrote:
Show 11 quoted lines
> On Sun, 21 Nov 2010, Jeff King wrote:
> 
> > This patch clears up your bug, and doesn't break any tests. But I'd 
> > really like to get a second opinion on the significance of those other 
> > flags, or why the flag clearing was at the bottom of the function in the 
> > first place.
> 
> The flag clearing was at the bottom because I had the impression that 
> something function one might want to call in that function in the future 
> could set the flags again. Maybe a goto would be appropriate here instead 
> of the early returns?

That makes sense. I did a quick skim of the called code and I'm not sure any flags could be set, but even if I am right, I think it is better to be defensive.

So let's do this, which is the equivalent behavior to your gotos, but this structure makes more sense to me as a reader (and it doesn't involve goto :) ).

-- >8 --
Subject: [PATCH] reflogs: clear flags properly in corner case

The reflog-walking mechanism is based on the regular revision traversal. We just rewrite the parents of each commit in fake_reflog_parent to point to the commit in the next reflog entry instead of the real parents.

However, the regular revision traversal tries not to show the same commit twice, and so sets the SHOWN flag on each commit it shows. In a reflog, however, we may want to see the same commit more than once if it appears in the reflog multiple times (which easily happens, for example, if you do a reset to a prior state).

The fake_reflog_parent function takes care of this by clearing flags, including SHOWN. Unfortunately, it does so at the very end of the function, and it is possible to return early from the function if there is no fake parent to set up (e.g., because we are at the very first reflog entry on the branch). In such a case the flag is not cleared, and the entry is skipped by the revision traversal machinery as already shown.

You can see this by walking the log of a ref which is set to its very first commit more than once (the test below shows such a situation). In this case the reflog walk will fail to show the entry for the initial creation of the ref.

We don't want to simply move the flag-clearing to the top of the function; we want to make sure flags set during the fake-parent installation are also cleared. Instead, let's hoist the flag-clearing out of the fake_reflog_parent function entirely. It's not really about fake parents anyway, and the only caller is the get_revision machinery.

Reported-by: Martin von Zweigbergk <martin.von.zweigbergk@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
---
 reflog-walk.c          |    1 -
 revision.c             |    4 +++-
 t/t1412-reflog-loop.sh |   34 ++++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+), 2 deletions(-)
 create mode 100755 t/t1412-reflog-loop.sh
diff --git a/reflog-walk.c b/reflog-walk.c
index 4879615..5d81d39 100644
--- a/reflog-walk.c
+++ b/reflog-walk.c
@@ -239,7 +239,6 @@ void fake_reflog_parent(struct reflog_walk_info *info, struct commit *commit)
 
 	commit->parents = xcalloc(sizeof(struct commit_list), 1);
 	commit->parents->item = commit_info->commit;
-	commit->object.flags &= ~(ADDED | SEEN | SHOWN);
 }
 
 void get_reflog_selector(struct strbuf *sb,
diff --git a/revision.c b/revision.c
index b1c1890..ded8812 100644
--- a/revision.c
+++ b/revision.c
@@ -2030,8 +2030,10 @@ static struct commit *get_revision_1(struct rev_info *revs)
 		revs->commits = entry->next;
 		free(entry);
 
-		if (revs->reflog_info)
+		if (revs->reflog_info) {
 			fake_reflog_parent(revs->reflog_info, commit);
+			commit->object.flags &= ~(ADDED | SEEN | SHOWN);
+		}
 
 		/*
 		 * If we haven't done the list limiting, we need to look at
diff --git a/t/t1412-reflog-loop.sh b/t/t1412-reflog-loop.sh
new file mode 100755
index 0000000..7f519e5
--- /dev/null
+++ b/t/t1412-reflog-loop.sh
@@ -0,0 +1,34 @@
+#!/bin/sh
+
+test_description='reflog walk shows repeated commits again'
+. ./test-lib.sh
+
+test_expect_success 'setup commits' '
+	test_tick &&
+	echo content >file && git add file && git commit -m one &&
+	git tag one &&
+	echo content >>file && git add file && git commit -m two &&
+	git tag two
+'
+
+test_expect_success 'setup reflog with alternating commits' '
+	git checkout -b topic &&
+	git reset one &&
+	git reset two &&
+	git reset one &&
+	git reset two
+'
+
+test_expect_success 'reflog shows all entries' '
+	cat >expect <<-\EOF
+		topic@{0} two: updating HEAD
+		topic@{1} one: updating HEAD
+		topic@{2} two: updating HEAD
+		topic@{3} one: updating HEAD
+		topic@{4} branch: Created from HEAD
+	EOF
+	git log -g --format="%gd %gs" topic >actual &&
+	test_cmp expect actual
+'
+
+test_done
-- 
1.7.3.2.509.g15259
Junio C Hamano· Nov 24, 2010, 00:35 UTC · re: Jeff King · lore

Re: First/oldest entry in reflog dropped

Jeff King <peff@peff.net> writes:
Show 21 quoted lines
> On Sun, Nov 21, 2010 at 12:36:21PM +0100, Johannes Schindelin wrote:
>
>> On Sun, 21 Nov 2010, Jeff King wrote:
>> 
>> > This patch clears up your bug, and doesn't break any tests. But I'd 
>> > really like to get a second opinion on the significance of those other 
>> > flags, or why the flag clearing was at the bottom of the function in the 
>> > first place.
>> 
>> The flag clearing was at the bottom because I had the impression that 
>> something function one might want to call in that function in the future 
>> could set the flags again. Maybe a goto would be appropriate here instead 
>> of the early returns?
>
> That makes sense. I did a quick skim of the called code and I'm not sure
> any flags could be set, but even if I am right, I think it is better to
> be defensive.
>
> So let's do this, which is the equivalent behavior to your gotos, but
> this structure makes more sense to me as a reader (and it doesn't
> involve goto :) ).

I have a feeling that we probably should have structured the code a bit more modularly, placing some info in "rev" that tells us how to "pop" a commit from the rev->list (typically "not the ones that we have shown"), what other commits to push back into the queue (typically, "all the parents that are not interesting"), and what side effects we should cause when we do so (typically, "mark uninteresting parents"), etc., instead of the current "if we are walking reflog, here is the special codepath we take", so that the walking is more generalized when we did the reflog walking (in fact, if we did this properly we probably wouldn't be calling it "bolted on"). But for now let's refrain from doing such a rewrite.

Thanks, both.
Jeff King· Nov 25, 2010, 16:15 UTC · re: Junio C Hamano · lore

Re: First/oldest entry in reflog dropped

On Tue, Nov 23, 2010 at 04:35:52PM -0800, Junio C Hamano wrote:
Show 14 quoted lines
> > So let's do this, which is the equivalent behavior to your gotos, but
> > this structure makes more sense to me as a reader (and it doesn't
> > involve goto :) ).
> 
> I have a feeling that we probably should have structured the code a bit
> more modularly, placing some info in "rev" that tells us how to "pop" a
> commit from the rev->list (typically "not the ones that we have shown"),
> what other commits to push back into the queue (typically, "all the
> parents that are not interesting"), and what side effects we should cause
> when we do so (typically, "mark uninteresting parents"), etc., instead of
> the current "if we are walking reflog, here is the special codepath we
> take", so that the walking is more generalized when we did the reflog
> walking (in fact, if we did this properly we probably wouldn't be calling
> it "bolted on").  But for now let's refrain from doing such a rewrite.

FWIW, I agree with all of that. I don't think it's worth fixing now, but perhaps in libgit2. :)

-Peff
Martin von Zweigbergk· Nov 21, 2010, 22:57 UTC · re: Jeff King · lore

Re: First/oldest entry in reflog dropped

On Sun, Nov 21, 2010 at 12:35 AM, Jeff King <peff@peff.net> wrote:
Show 123 quoted lines
> On Sat, Nov 20, 2010 at 10:11:34PM -0500, Martin von Zweigbergk wrote:
>
>> Can someone explain the behavior in the execution below?
>>
>> # I expected this reflog...
>> $ git branch tmp
>> $ git reflog show refs/heads/tmp
>> b60a214 refs/heads/tmp@{0}: branch: Created from master
>>
>> # ... and this one as well...
>> $ git update-ref refs/heads/tmp HEAD^
>> $ git reflog show refs/heads/tmp
>> 7d1a0b8 refs/heads/tmp@{0}:
>> b60a214 refs/heads/tmp@{1}: branch: Created from master
>>
>> # ... but why is the first entry (i.e. "branch: Created from master")
>> # dropped here?
>> $ git update-ref refs/heads/tmp HEAD
>> $ git reflog show refs/heads/tmp
>> b60a214 refs/heads/tmp@{0}:
>> 7d1a0b8 refs/heads/tmp@{1}:
>>
>> If the ref is updated once more (to e.g. HEAD^^) before being moved back
>> to HEAD, the first entry will be shown in the output.
>>
>> If this is a bug, it seems to be in reflog, rather than in update-ref,
>> because the first entry does exist in .git/logs/refs/heads/tmp.
>
> I think it's a bug in the reflog-walking machinery, which is sort of
> bolted onto the regular revision traversal machinery. When we hit
> b60a214 the first time, we show it and set the SHOWN flag (since the
> normal traversal machinery would not want to show a commit twice). When
> we hit it again, simplify_commit() sees that it is SHOWN and tells us to
> skip it.
>
> However, the bolted-on reflog-walking machinery does have a way of
> handling this. While we are traversing via get_revision(), we notice
> that we are doing a reflog walk and call fake_reflog_parent. This
> function is responsible for replacing the actual parents of the commit
> with a fake list consisting of the previous reflog entry (so we
> basically pretend that the history consists of a string of commits, each
> one pointing to the previous reflog entry, not the actual parent).
>
> This function _also_ clears some flags, including the SHOWN flag, in
> what almost seems like a tacked-on side effect. So if we hit the same
> commit twice, we will actually show it again. Which is what makes
> reflogs with repeated commits work at all.
>
> However, there is a subtle bug: it clears the flags at the very end of
> the function. But through the function, if we see that there are no fake
> parents (because we are on the very first reflog entry), we do an early
> return. But we not only skip the later "set up parents" code, we also
> accidentally skip the "clear SHOWN flag" side-effect code.
>
> So I believe we will always fail to show the very first reflog if it is
> a repeated commit.
>
> The fix, AFAICT, is to just move the flag clearing above the early
> returns (patch below). But I have to admit I do not quite understand
> what the ADDED and SEEN flags are doing here, as this is the first time
> I have ever looked at the reflog-walk code. So possibly just the SHOWN
> flag should be unconditionally cleared.
>
> This patch clears up your bug, and doesn't break any tests. But I'd
> really like to get a second opinion on the significance of those other
> flags, or why the flag clearing was at the bottom of the function in the
> first place.
>
> -Peff
>
> ---
> diff --git a/reflog-walk.c b/reflog-walk.c
> index 4879615..d5d055b 100644
> --- a/reflog-walk.c
> +++ b/reflog-walk.c
> @@ -210,44 +210,45 @@ int add_reflog_for_walk(struct reflog_walk_info *info,
>        add_commit_info(commit, commit_reflog, &info->reflogs);
>        return 0;
>  }
>
>  void fake_reflog_parent(struct reflog_walk_info *info, struct commit *commit)
>  {
>        struct commit_info *commit_info =
>                get_commit_info(commit, &info->reflogs, 0);
>        struct commit_reflog *commit_reflog;
>        struct reflog_info *reflog;
>
> +       commit->object.flags &= ~(ADDED | SEEN | SHOWN);
> +
>        info->last_commit_reflog = NULL;
>        if (!commit_info)
>                return;
>
>        commit_reflog = commit_info->util;
>        if (commit_reflog->recno < 0) {
>                commit->parents = NULL;
>                return;
>        }
>
>        reflog = &commit_reflog->reflogs->items[commit_reflog->recno];
>        info->last_commit_reflog = commit_reflog;
>        commit_reflog->recno--;
>        commit_info->commit = (struct commit *)parse_object(reflog->osha1);
>        if (!commit_info->commit) {
>                commit->parents = NULL;
>                return;
>        }
>
>        commit->parents = xcalloc(sizeof(struct commit_list), 1);
>        commit->parents->item = commit_info->commit;
> -       commit->object.flags &= ~(ADDED | SEEN | SHOWN);
>  }
>
>  void get_reflog_selector(struct strbuf *sb,
>                         struct reflog_walk_info *reflog_info,
>                         enum date_mode dmode,
>                         int shorten)
>  {
>        struct commit_reflog *commit_reflog = reflog_info->last_commit_reflog;
>        struct reflog_info *info;
>        const char *printed_ref;
>
>

Good job! And yes, if the first commit is included in any cycles, it seems to be dropped. I must have been blind yesterday...

← back to recent threads