threads / discuss / 9807

rebase from ambiguous ref discards changes

Subject: rebase from ambiguous ref discards changes

## tl;dr

17 messages between Sep 6, 2007 and Sep 8, 2007.

replies: 16people: 7as markdown or json

Keith Packard· Sep 6, 2007, 21:48 UTC · lore
So, I started with a very simple repository
---*--- master
    \
     -- origin/master
From master, I did

$ git-rebase origin/master warning: refname 'master' is ambiguous. First, rewinding head to replay your work on top of it... HEAD is now at 2a8592f... Fix G33 GTT stolen mem range Fast-forwarded master to origin/master. $

Note the lack of the usual 'Applying <patch>' messages.
checking the tree, I now had
---*
    \
     -- origin/master
        master
with my patch lost.

recovering my patch (having the ID in my terminal window from the commit), I named it 'master-with-fix'

---*--- master-with-fix
    \
     -- origin/master
        master
Now the rebase from 'master-with-fix worked as expected:

$ git-rebase origin/master First, rewinding head to replay your work on top of it... HEAD is now at 2a8592f... Fix G33 GTT stolen mem range

Applying Switch to pci_device_map_range/pci_device_unmap_range APIs.
Adds trailing whitespace.
.dotest/patch:225:      
Adds trailing whitespace.
.dotest/patch:226:      if (IS_I965G(pI830)) 
Adds trailing whitespace.
.dotest/patch:446:
dev->regions[mmio_bar].size, 
Adds trailing whitespace.
.dotest/patch:449:    
warning: 4 lines add whitespace errors.
Wrote tree cd373666254d56a137d282deeb15a2ccaf8da22b
Committed: 286f5df0b62f571cbb4dbf120679d3af029b8775
$ 
And the tree looks right too:
--- origin/master --- master-with-fix
    master
       
Seems like there's something going on when 'master' is ambiguous, or
perhaps some other problem.

This is all from version 1.5.3, but I think I've seen this on 1.5.2 as well.

Git made me sad today; I'm not sure it's ever disappointed like this before.

-- 
keith.packard@intel.com
Pierre Habouzit· Sep 6, 2007, 22:37 UTC · re: Keith Packard · lore

Re: rebase from ambiguous ref discards changes

On Thu, Sep 06, 2007 at 09:48:28PM +0000, Keith Packard wrote:
> recovering my patch (having the ID in my terminal window from the
> commit), I named it 'master-with-fix'
  Note that your patch was not lost, even if you had pruned, because it
would still be accessible from the reflog of your master branch. Not
that it's supposed to soften your disappointment but well, at least it
should make you a bit less uncomfortable, maybe :)
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
Junio C Hamano· Sep 6, 2007, 23:26 UTC · re: Keith Packard · lore

Re: rebase from ambiguous ref discards changes

Keith Packard <keithp@keithp.com> writes:
Show 19 quoted lines
> So, I started with a very simple repository
>
> ---*--- master
>     \
>      -- origin/master
>
> From master, I did
>
> $ git-rebase origin/master
> warning: refname 'master' is ambiguous.
> First, rewinding head to replay your work on top of it...
> HEAD is now at 2a8592f... Fix G33 GTT stolen mem range
> Fast-forwarded master to origin/master.
> ...
> Seems like there's something going on when 'master' is ambiguous, or
> perhaps some other problem.
>
> This is all from version 1.5.3, but I think I've seen this on 1.5.2 as
> well.

We haven't touched this area for a long time (like "v1.3.0" or "since March 2006"). I wish you told us about this earlier.

I'd like to reproduce this, but I need to be sure what your "ambiguous" situation is really like. What ambiguous "master"s do you have? IOW, what does:

	git show-ref | grep master

say? I have 11 lines of output from the above and I have never seen a problem like this.

Perhaps you have ".git/master" by mistake?
Keith Packard· Sep 7, 2007, 02:58 UTC · re: Junio C Hamano · lore

Re: rebase from ambiguous ref discards changes

On Thu, 2007-09-06 at 16:26 -0700, Junio C Hamano wrote:
> We haven't touched this area for a long time (like "v1.3.0" or
> "since March 2006").  I wish you told us about this earlier.

I would have were I certain that it wasn't user-error last time. I took a few minutes this time to verify precisely what I saw and reproduce it.

Show 5 quoted lines
> I'd like to reproduce this, but I need to be sure what your
> "ambiguous" situation is really like.  What ambiguous "master"s
> do you have?  IOW, what does:
> 
> 	git show-ref | grep master

$ git show-ref | grep master 286f5df0b62f571cbb4dbf120679d3af029b8775 refs/heads/master 1feb733eb8b09a8b07b7a6987add5149c53b0157 refs/heads/master-guitar d957c6b8e1dde8e11c1db3431e0ff58c5d984880 refs/heads/master-i830 0fd3ba0518b3cde9ca0e4e2fc1854c00d8a43d5c refs/remotes/kyle/master e25f8e145f4f73b62c32389b922291fd561af9d2 refs/remotes/origin/lg3d-master 286f5df0b62f571cbb4dbf120679d3af029b8775 refs/remotes/origin/master e25f8e145f4f73b62c32389b922291fd561af9d2 refs/remotes/otc/lg3d-master 6781575f734f05547d7d5ceef4116fc157bba44d refs/remotes/otc/master

so, nothing obviously amiss here.
> Perhaps you have ".git/master" by mistake?
oops.

$ find .git -name master .git/master .git/refs/heads/master .git/refs/remotes/kyle/master .git/refs/remotes/origin/master .git/refs/remotes/otc/master .git/logs/refs/heads/master .git/logs/refs/remotes/fdo/master .git/logs/refs/remotes/kyle/master .git/logs/refs/remotes/origin/master .git/logs/refs/remotes/otc/master

So, I think that explains where the ambiguous master came from. Seems like rebase should be able to bail out before breaking things though.

-- 
keith.packard@intel.com
Pierre Habouzit· Sep 7, 2007, 06:55 UTC · re: Keith Packard · lore

Re: rebase from ambiguous ref discards changes

On Fri, Sep 07, 2007 at 02:58:18AM +0000, Keith Packard wrote:
Show 20 quoted lines
> On Thu, 2007-09-06 at 16:26 -0700, Junio C Hamano wrote:
>
> > Perhaps you have ".git/master" by mistake?
>
> oops.
>
> $ find .git -name master
> ..git/master
> ..git/refs/heads/master
> ..git/refs/remotes/kyle/master
> ..git/refs/remotes/origin/master
> ..git/refs/remotes/otc/master
> ..git/logs/refs/heads/master
> ..git/logs/refs/remotes/fdo/master
> ..git/logs/refs/remotes/kyle/master
> ..git/logs/refs/remotes/origin/master
> ..git/logs/refs/remotes/otc/master
> 
> So, I think that explains where the ambiguous master came from.  Seems
> like rebase should be able to bail out before breaking things though.
  Actually if I get this right, it didn't broke anything, it just
rebased your ".git/master" :) It just chose the wrong desambiguation for
some reason.
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
Junio C Hamano· Sep 7, 2007, 11:21 UTC · re: Keith Packard · lore

[PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

Keith Packard <keithp@keithp.com> writes:
Show 5 quoted lines
> On Thu, 2007-09-06 at 16:26 -0700, Junio C Hamano wrote:
> ...
>> Perhaps you have ".git/master" by mistake?
>
> oops.

Ok, that explains it; git is doing exactly what the user asked it to do.

              5---6 side
             /
	1---2---3---4 master

The user has this history. But he has a stray .git/master file, perhaps created by hand by mistake (it would be very interesting to find how that file got there in the first place), that points at commit "3". From a side branch, he says "git rebase master".

The first parameter to "git rebase" in a single parameter form is "the commit on which to replay the changes my current branch has". It is not limited to a branch name. IOW, it can be an arbitrary object name, and one of the rules to translate a user string that is supposed to mean an arbitrary object name goes through this table:

        "%s", "refs/%s", "refs/tags/%s", "refs/heads/%s",
        "refs/remotes/%s", "refs/remotes/%s/HEAD"

For each entry in the above table, "%s" part of the entry is replaced by the user string, and resulting string is used to see if there is such a file under .git/ directory, and the first match is used (this explanation is simplifying things a bit).

This rule has been there almost from the beginning. The first entry allowed you to have a tag A and a branch A at the same time, while giving you a way to disambiguate them by spelling them out as "refs/tags/A" and "refs/heads/A", respectively. Also the first rule covered special "ref" names such as HEAD and ORIG_HEAD.

Alas, there is one drawback of this rule. His .git/master hides refs/heads/master. So essentially his command line said "I've built a few commits since I forked from the history that leads to commit 3; I want to replay my changes on top of that commit". He meant to say 4, but said 3, and as a unfortunate consequence, commit 4 is lost.

The above explanation is solely for understanding the situation and, not meant to defend the current behaviour. I think the current behaviour is inviting mistakes and confusion.

This patch brings in a new world order by introducing a backward incompatible change. When the string the user gave us does not contain any slash, we do not apply the first entry (i.e. directly underneath .git/ without any "refs/***") unless the name consists solely of uppercase letters or an underscore, thereby ignoring .git/master. The ones we often use, such as HEAD and ORIG_HEAD are not affected by this change.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 sha1_name.c                |   35 ++++++++++++++++++++++++++++++++++
 t/t3407-rebase-confused.sh |   45 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 80 insertions(+), 0 deletions(-)
 create mode 100755 t/t3407-rebase-confused.sh
diff --git a/sha1_name.c b/sha1_name.c
index 2d727d5..1b980ca 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -239,6 +239,35 @@ static int ambiguous_path(const char *path, int len)
 	return slash;
 }
 
+static int confused_ref(const char *str)
+{
+	char ch;
+	int seen_non_uppercase = 0;
+
+	/*
+	 * People create .git/master by mistake using hand-rolled
+	 * scripts and confuse themselves utterly.  Make sure this
+	 * does not match such a path.
+	 */
+	while ((ch = *str++) != '\0') {
+		if (ch == '/')
+			return 0;
+		if (!((ch == '_') ||
+		      ('A' <= ch && ch <= 'Z') ||
+		      ('0' <= ch && ch <= '9')))
+			seen_non_uppercase = 1;
+	}
+
+	/*
+	 * str did not have any slash and we are checking when
+	 * it hangs directly underneath .git/; the only valid
+	 * cases we currently have are HEAD, ORIG_HEAD, MERGE_HEAD and
+	 * FETCH_HEAD.  If we saw any non uppercase, non underscore,
+	 * we should ignore this string.
+	 */
+	return seen_non_uppercase;
+}
+
 static const char *ref_fmt[] = {
 	"%.*s",
 	"refs/%.*s",
@@ -259,6 +288,9 @@ int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref)
 		unsigned char sha1_from_ref[20];
 		unsigned char *this_result;
 
+		if ((p == ref_fmt) && confused_ref(str))
+			continue;
+
 		this_result = refs_found ? sha1_from_ref : sha1;
 		r = resolve_ref(mkpath(*p, len, str), this_result, 1, NULL);
 		if (r) {
@@ -283,6 +315,9 @@ int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
 		char path[PATH_MAX];
 		const char *ref, *it;
 
+		if (p == ref_fmt && confused_ref(str))
+			continue;
+
 		strcpy(path, mkpath(*p, len, str));
 		ref = resolve_ref(path, hash, 0, NULL);
 		if (!ref)
diff --git a/t/t3407-rebase-confused.sh b/t/t3407-rebase-confused.sh
new file mode 100755
index 0000000..5254da6
--- /dev/null
+++ b/t/t3407-rebase-confused.sh
@@ -0,0 +1,45 @@
+#!/bin/sh
+
+test_description="Keithp's .git/master problem
+
+              5---6 side
+             /
+	1---2---3---4 master
+
+"
+
+. ./test-lib.sh
+
+build () {
+	echo "$1" >"$1" && git add "$1" && test_tick && git commit -m "$1"
+}
+
+test_expect_success setup '
+
+	build one &&
+	build two &&
+	git branch side &&
+	build three &&
+	git tag anchor &&
+	build four &&
+	git-checkout side &&
+	build five &&
+	build six &&
+	git update-ref master anchor &&
+	git tag -d anchor
+
+'
+
+test_expect_success rebase '
+
+	git rebase master
+
+'
+
+test_expect_success 'everybody is still there' '
+
+	test 6 = $(git log --pretty=oneline HEAD | wc -l)
+
+'
+
+test_done
-- 
1.5.3.1.879.g4d83f
Johannes Sixt· Sep 7, 2007, 12:36 UTC · re: Junio C Hamano · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

Junio C Hamano schrieb:
> But he has a stray .git/master file,
> perhaps created by hand by mistake (it would be very interesting
> to find how that file got there in the first place),
It is easy to get one there if, in a brave moment, you try
    git update-ref master $some_other_ref
instead of the correct
    git update-ref refs/heads/master $some_other_ref
-- Hannes
Pierre Habouzit· Sep 7, 2007, 12:42 UTC · re: Johannes Sixt · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

On Fri, Sep 07, 2007 at 12:36:15PM +0000, Johannes Sixt wrote:
Show 12 quoted lines
> Junio C Hamano schrieb:
> >But he has a stray .git/master file,
> >perhaps created by hand by mistake (it would be very interesting
> >to find how that file got there in the first place),
> 
> It is easy to get one there if, in a brave moment, you try
> 
>    git update-ref master $some_other_ref
> 
> instead of the correct
> 
>    git update-ref refs/heads/master $some_other_ref
  I was about to say the same :)
  I'd have added though that maybe update-ref should print a warning for
the references that do not match the restriction Junio added. This could
be done using the function Junio proposed un update_ref() in refs.c
  note that it's a sane thing to do anyways, I would not bet a lot of
money on what happens if you ask git to:
  git update-ref $foo $sha
  for foo in (completely random :P): index config packed-refs ...
  If a tool needs a new reference namespace, it can create a
subdirectory under refs/ so it does not really causes harm IMHO.
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
Junio C Hamano· Sep 7, 2007, 20:39 UTC · re: Pierre Habouzit · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

Pierre Habouzit <madcoder@debian.org> writes:
> I'd have added though that maybe update-ref should print a warning for
> the references that do not match the restriction Junio added. This could
> be done using the function Junio proposed un update_ref() in refs.c

I would even suggest making it into an error, even if we do not error out on the reading side (being liberal when reading but more strict when creating, that is).

That confused_ref() needs to be tightened further, by the way. It is called only when we are considering to tack the user string immediately below $GIT_DIR/ so the only valid cases are (1) the string begins with "refs/", or (2) the string is all uppercase (or underscore), especially without slash. The one in the proposed patch is not strict enough and does not enforce the former.

Pierre Habouzit· Sep 7, 2007, 21:04 UTC · re: Junio C Hamano · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

On Fri, Sep 07, 2007 at 08:39:43PM +0000, Junio C Hamano wrote:
Show 17 quoted lines
> Pierre Habouzit <madcoder@debian.org> writes:
> 
> > I'd have added though that maybe update-ref should print a warning for
> > the references that do not match the restriction Junio added. This could
> > be done using the function Junio proposed un update_ref() in refs.c
> 
> I would even suggest making it into an error, even if we do not
> error out on the reading side (being liberal when reading but
> more strict when creating, that is).
> 
> That confused_ref() needs to be tightened further, by the way.
> It is called only when we are considering to tack the user
> string immediately below $GIT_DIR/ so the only valid cases are
> (1) the string begins with "refs/", or (2) the string is all
> uppercase (or underscore), especially without slash.  The one in
> the proposed patch is not strict enough and does not enforce the
> former.
  I reckon I didn't checked what the function did in detail, just the
code layout :) And I agree an error is event better, I just don't have
enough knowledge of the scripts that used refs in git for a long time
that such a change could break. I mean, I only use git since the 1.2
(maybe even 1.3) series :)
  I'm always all for refusing dangerous layouts rather than trying too
hard to support cumbersome things that are 99% of the times issues :)
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org
Alex Riesen· Sep 8, 2007, 22:20 UTC · re: Junio C Hamano · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

Junio C Hamano, Fri, Sep 07, 2007 22:39:43 +0200:
Show 9 quoted lines
> Pierre Habouzit <madcoder@debian.org> writes:
> 
> > I'd have added though that maybe update-ref should print a warning for
> > the references that do not match the restriction Junio added. This could
> > be done using the function Junio proposed un update_ref() in refs.c
> 
> I would even suggest making it into an error, even if we do not
> error out on the reading side (being liberal when reading but
> more strict when creating, that is).
I agree (and suggest failing even on reading), but see below
> That confused_ref() needs to be tightened further, by the way.
> It is called only when we are considering to tack the user
> string immediately below $GIT_DIR/ so the only valid cases are
> (1) the string begins with "refs/",

If that will be the case git-p4-import.bat (yes, just a script of mine) will break because it has its namespace directly in $GIT_DIR (i.e. .git/p4/*) and stores there backup references. It is just a someones (ok, it is mine) script, but maybe there are others, who expect that plumbing level git-update-ref just do what its told.

> or (2) the string is all uppercase (or underscore), especially
> without slash.

I'd suggest just check for uppercase+underscore _or_ slash. It is plumbing after all.

Keith Packard· Sep 7, 2007, 16:03 UTC · re: Johannes Sixt · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

On Fri, 2007-09-07 at 14:36 +0200, Johannes Sixt wrote:
>     git update-ref master $some_other_ref

I'm sure that's how it was created; I was using update-ref on a regular basis for a few weeks with this repository before the 1.5 new world order for remotes came about.

-- 
keith.packard@intel.com
Keith Packard· Sep 7, 2007, 16:08 UTC · re: Junio C Hamano · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

On Fri, 2007-09-07 at 04:21 -0700, Junio C Hamano wrote:
Show 7 quoted lines
> This patch brings in a new world order by introducing a backward
> incompatible change.  When the string the user gave us does not
> contain any slash, we do not apply the first entry (i.e.
> directly underneath .git/ without any "refs/***") unless the
> name consists solely of uppercase letters or an underscore,
> thereby ignoring .git/master.  The ones we often use, such as
> HEAD and ORIG_HEAD are not affected by this change.

It seems to me that instead of introducing an incompatible (but probably useful) change, a sensible option would be to have the ambiguous reference be an error instead of a warning. One shouldn't be encouraged to use names in .git that conflict with stuff in refs/heads anyway.

-- 
keith.packard@intel.com
Nicolas Pitre· Sep 7, 2007, 16:29 UTC · re: Keith Packard · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

On Fri, 7 Sep 2007, Keith Packard wrote:
Show 14 quoted lines
> On Fri, 2007-09-07 at 04:21 -0700, Junio C Hamano wrote:
> 
> > This patch brings in a new world order by introducing a backward
> > incompatible change.  When the string the user gave us does not
> > contain any slash, we do not apply the first entry (i.e.
> > directly underneath .git/ without any "refs/***") unless the
> > name consists solely of uppercase letters or an underscore,
> > thereby ignoring .git/master.  The ones we often use, such as
> > HEAD and ORIG_HEAD are not affected by this change.
> 
> It seems to me that instead of introducing an incompatible (but probably
> useful) change, a sensible option would be to have the ambiguous
> reference be an error instead of a warning. One shouldn't be encouraged
> to use names in .git that conflict with stuff in refs/heads anyway.

I agree. IMHO the sensible thing to do is to always warn, and error out by default. I see no advantage for core.warnAmbiguousRefs=false other than allow the user to shoot himself in the foot someday. Instead, we should have core.allowAmbiguousRefs set to off by default.

Nicolas
Junio C Hamano· Sep 7, 2007, 20:32 UTC · re: Nicolas Pitre · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

Nicolas Pitre <nico@cam.org> writes:
Show 9 quoted lines
>> It seems to me that instead of introducing an incompatible (but probably
>> useful) change, a sensible option would be to have the ambiguous
>> reference be an error instead of a warning. One shouldn't be encouraged
>> to use names in .git that conflict with stuff in refs/heads anyway.
>
> I agree.  IMHO the sensible thing to do is to always warn, and error out 
> by default.  I see no advantage for core.warnAmbiguousRefs=false other 
> than allow the user to shoot himself in the foot someday.  Instead, we 
> should have core.allowAmbiguousRefs set to off by default.

Well, for about three weeks late November to early December 2005, we did make this an error. Since mid December 2005, we reverted that change to the original "take first match, without even attempting to detect ambiguity". I do not recall what the discussion that led to that change was about, but it could have been the issue Len had that confused "git merge" with a tag and a branch named after bugzilla bug number. In any case, this change most likely was because some people were actually using the same name and the change to make it an error hurted them.

We then reintroduced the ambiguity detection late March 2006, but only as a warning, again fearing that erroring out would break people's existing setups. I think we also rewrote examples in our documentation that said "create your own branch v2.6.13 that fork from v2.6.13 tag" to read "create your own brancy my-2.6.13..." to avoid encouraging the use of same name to people.

I think the warning has been with us for a long time and by now people know better not to confuse themselves.

So I am all for making an ambiguous refname an error in 1.5.4.

At the same time, I think it makes sense to forbid update-ref outside refs/ if the refname is not special (say, with any lowercase letters), and ignore names immediately below .git that are not all-uppercase+underscore (e.g. "FETCH_HEAD" we read, "description" we ignore).

Please make it so.
Carl Worth· Sep 7, 2007, 21:53 UTC · re: Junio C Hamano · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

On Fri, 07 Sep 2007 13:32:30 -0700, Junio C Hamano wrote:
> So I am all for making an ambiguous refname an error in 1.5.4.

If you do, then please also make it an error to create an ambiguous refname as well.

For example, look at how late the warning comes out in this case, and how changing it to an error at that point would not help anything:

	$ git branch tmp
	$ ...
	$ git tag tmp	# No warning here!
	$ git show tmp
	warning: refname 'tmp' is ambiguous.
-Carl
Junio C Hamano· Sep 7, 2007, 22:08 UTC · re: Carl Worth · lore

Re: [PATCH] HEAD, ORIG_HEAD and FETCH_HEAD are really special.

Carl Worth <cworth@cworth.org> writes:
Show 5 quoted lines
> On Fri, 07 Sep 2007 13:32:30 -0700, Junio C Hamano wrote:
>> So I am all for making an ambiguous refname an error in 1.5.4.
>
> If you do, then please also make it an error to create an ambiguous
> refname as well.
Yeah, didn't I also suggest that already?

← back to recent threads