sporadic git failures on interactive rebase

Hi,

(git version 2.2.0)

I am currently developing/testing a script for a "history surgery" on a quite big repository (~30000 commits). The script always runs against exactly the same copy of a git repository. So things should be reproducable, but sometimes i get failures for the following sequence of commands:  

$ git checkout some_branch $ GIT_SEQUENCE_EDITOR="sed -i '1s/^pick /edit /'" git rebase -i $MERGETARGET~1 $ git rm -rf some_files $ git commit --amend --no-edit -c $MERGETARGET $ git rebase --continue

(where MERGETARGET is a valid commit id in the history of some_branch)

Here is an example output where things went wrong (MERGETARGET is 6185ac39299a740dc9bc6c5906dd1f229b3f046b). The interesting parts are the messages "c4095c1: not a commit that can be picked" and "error: short SHA1 c4095c1 is ambiguous.":

. Switched to branch 'master-flat' . c4095c1: not a commit that can be picked . 80f99bd: not a commit that can be picked . Stopped at 6185ac39299a740dc9bc6c5906dd1f229b3f046b... some_comment . You can amend the commit now, with . . git commit --amend . . Once you are satisfied with your changes, run . . git rebase --continue . rm 'some_files' . [detached HEAD 56675a06316345ac121997dde2b9eddb649d0539] some_comment . Author: user <foo.bar@com.com> . Date: Wed Sep 26 09:11:17 2012 +0000 . [more info about that commit] . error: short SHA1 c4095c1 is ambiguous. . fatal: Needed a single revision . Invalid commit name: c4095c1

Now that the command failed, i checked for ambigous c4095c1. But there is only one: $ git log -1 c4095c1 . commit c4095c1f5e7c126accf93ba68e2fa72bb055f567 . ...

Just as a test: $ git log -1 c409 . error: short SHA1 c409 is ambiguous. . error: short SHA1 c409 is ambiguous.

As i said above this issue is not strictly reproducable. I have a full backup of the working dir including the .git folder from just before the command sequence above. If i rollback and execute the commands again, everything works fine. Up to now i didn't had the situation the the sequence failed twice two in a row.

I also have the backup of the directory from directly after the sequence failure. So i can still test some commands if you need more information.

Does anybody have an idea of what's going on here?

Best regards Henning

On Tue, Jan 13, 2015 at 11:54:32AM +0100, Henning Moll wrote:

Hmm. There are some instances in git where we know we are looking for an object of a particular type, and we can disambiguate a short-sha1 based on the type. And "git log" is just such a place, whereas a generic "git rev-parse" used by the git-rebase script would not be.

Try:

  git rev-list --objects --all |
  grep ^c4095c1 |
  cut -d' ' -f1 |
  git cat-file --batch-check

If my hunch is right, then you may find multiple objects, only one of which is a commit.

And if so, the solution is likely one of:

  1. Teach rev-parse a "prefer committish" option and use it in the
     appropriate spot in git-rebase.

  2. Teach whatever is generating the shortended sha1s to report a truly
     unambiguous result, not one that depends on the type. I don't think
     we _have_ a codepath to do type-dependent shortening, though, which
     is maybe an indication that this is a red herring.

It's also possible that part of the rebase sequence generates a new object that creates the ambiguity. That would be consistent with your statement that rolling back and running the commands again makes it work. I'm not sure how to defend against that, aside from adding a character or two to the shortened ids, which reduces the likelihood of a collision.

-Peff

On Wed, Jan 14, 2015 at 07:19:15AM -0500, Jeff King wrote:

Yeah, I think this cannot be it. There is a 7-character commit/blob ambiguity in git. You can find it yourself with:

  git rev-list --objects --all |
  cut -d' ' -f1 |
  sort | uniq -w 7 -D |
  git cat-file --batch-check |
  head -2

which produces:

  01319837c53050109c60e6740dfa9462327161f0 commit 649
  0131983dfbc143ce5dae77e067663bb2e7d5f126 blob 20638

And it behaves as expected. Running "git rev-parse 0131983" complains of the ambiguity, but "git log 0131983" shows the commit.

What happens if we rebase with it?

  $ git checkout 01319837
  $ git rebase -i HEAD^

will yield a todo file with the 8-character unambiguous abbreviation.

So I guess all is working as intended there. Perhaps you really were just very unlucky and an earlier step of the rebase created a conflicting sha1.

-Peff

Jeff King <peff@peff.net> writes:

That would mean 75c69766 (rebase -i: fix short SHA-1 collision, 2013-08-23) did not fix what it intended to fix, no? Is the symptom coming from pre-1.8.4.2 version of Git?

On Wed, Jan 14, 2015 at 09:12:46AM -0800, Junio C Hamano wrote:

Yeah, you're right. I didn't even remember that commit at all. On the off chance that the abbreviation code was different in that earlier version, I also checked rebasing 01319837 with an older version, but it does work fine.

So yeah, the most plausible theory to me so far is unluckiness combined with pre-1.8.4.2. That should be easy to disprove if Henning tells us his git version.

-Peff

On Wed, Jan 14, 2015 at 3:54 PM, Jeff King <peff@peff.net> wrote:

Henning mentioned it at the very top of his original problem report:

    "(git version 2.2.0)"

On Wed, Jan 14, 2015 at 4:00 PM, Eric Sunshine <sunshine@sunshineco.com> wrote:

For completeness: http://article.gmane.org/gmane.comp.version-control.git/262334

On Wed, Jan 14, 2015 at 04:00:37PM -0500, Eric Sunshine wrote:

Ah, reading comprehension. It pays off.

I'm stumped, then.

-Peff

On Wed, Jan 14, 2015 at 4:02 PM, Jeff King <peff@peff.net> wrote:

Perhaps some new code been added to git-rebase--interactive.sh since 75c69766 which neglects to invoke expand_todo_ids()?

Or, possibly some older version of git is being invoked somehow during rebase despite his "front end" use of 2.2.0?

Eric Sunshine <sunshine@sunshineco.com> writes:

Or git_sequence_editor creates conflicting objects?