threads / discuss / 39369

recovering from "unordered stage entries in index" error

Subject: recovering from "unordered stage entries in index" error

## tl;dr

17 messages between May 19, 2015 and May 26, 2015.

replies: 16people: 3as markdown or json

McHenry, Matt· May 19, 2015, 13:48 UTC · lore
	I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:

$ git svn fetch fatal: unordered stage entries in index write-tree: command returned error: 128

	'git status' shows a few untracked files but is otherwise clean.
	It looks like this check was introduced in 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary "read_index_from(): catch out of order entries when reading an index file" (first appearing in 2.2.0).
	Mailing list discussion looked like it implicated third-party tools.  I don't recall running any other tools on this repo; it doesn't do much day-to-day other than a long series of 'git svn fetch'es.  (But it's been around for a couple of years, so who knows.)
	At any rate, what can I do to recover from this situation?  I tried to locate a path with multiple index entries like this, but got no results:
$ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '
	(I originally posted on SO at http://stackoverflow.com/questions/30264826/; I'll update that with any solutions that come up here, to ease future googling.)
Junio C Hamano· May 19, 2015, 15:19 UTC · re: McHenry, Matt · lore

Re: recovering from "unordered stage entries in index" error

On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt <mmchenry@carnegielearning.com> wrote:

Show 6 quoted lines
>
>         I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:
>
> $ git svn fetch
> fatal: unordered stage entries in index
> write-tree: command returned error: 128

Are you by any chance using the split-index mode (which I think is broken in 2.3.6)?

McHenry, Matt· May 19, 2015, 16:51 UTC · re: Junio C Hamano · lore

RE: recovering from "unordered stage entries in index" error

	I had never heard of that.  Google led me to the man page for 'git update-index --split-index', and after reading it, I think this demonstrates that I'm not using that feature:

$ ls .git/*index* .git/index

Show 20 quoted lines
> -----Original Message-----
> From: jch2355@gmail.com [mailto:jch2355@gmail.com] On Behalf Of Junio C
> Hamano
> Sent: Tuesday, May 19, 2015 11:20
> To: McHenry, Matt
> Cc: git@vger.kernel.org
> Subject: Re: recovering from "unordered stage entries in index" error
> 
> On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt
> <mmchenry@carnegielearning.com> wrote:
> >
> >         I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now
> unable to run 'git svn fetch' in one of my repositories:
> >
> > $ git svn fetch
> > fatal: unordered stage entries in index
> > write-tree: command returned error: 128
> 
> Are you by any chance using the split-index mode (which I think is
> broken in 2.3.6)?
Duy Nguyen· May 21, 2015, 09:56 UTC · re: McHenry, Matt · lore

Re: recovering from "unordered stage entries in index" error

On Tue, May 19, 2015 at 8:48 PM, McHenry, Matt <mmchenry@carnegielearning.com> wrote:

Show 6 quoted lines
>
>
>         I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:
>
> $ git svn fetch
> fatal: unordered stage entries in index

This message can be improved to show what entries have this problem. But then I don't see any way to recover the index manually. ls-files will die too. Perhaps we should be gentle in this case: show warnings instead of aborting the program and internally reorder the index. I think, unless you have multiple entries with the same stage, the recovered index should run well. The broken index could be renamed to index.broken or something for later analysis, or we forbid writing the reordered index to disk.

Hmm?
Show 17 quoted lines
> write-tree: command returned error: 128
>
>         'git status' shows a few untracked files but is otherwise clean.
>
>         It looks like this check was introduced in 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary "read_index_from(): catch out of order entries when reading an index file" (first appearing in 2.2.0).
>
>         Mailing list discussion looked like it implicated third-party tools.  I don't recall running any other tools on this repo; it doesn't do much day-to-day other than a long series of 'git svn fetch'es.  (But it's been around for a couple of years, so who knows.)
>
>         At any rate, what can I do to recover from this situation?  I tried to locate a path with multiple index entries like this, but got no results:
>
> $ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '
>
>         (I originally posted on SO at http://stackoverflow.com/questions/30264826/; I'll update that with any solutions that come up here, to ease future googling.)
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-- 
Duy
McHenry, Matt· May 21, 2015, 13:19 UTC · re: Duy Nguyen · lore

RE: recovering from "unordered stage entries in index" error

> This message can be improved to show what entries have this problem.
	Yes, that would definitely be a start.  :)
> But then I don't see any way to recover the index manually. ls-files
> will die too. Perhaps we should be gentle in this case: show warnings
	Actually, ls-files succeeds on my broken index:

$ git ls-files > /dev/null $ echo $? 0

	Could I do something with 'git read-tree' to force creation of a new valid index?  I guess 'git clone' would work too, except that I have 'git svn' metadata that I'd need to preserve.
Show 41 quoted lines
> instead of aborting the program and internally reorder the index. I
> think, unless you have multiple entries with the same stage, the
> recovered index should run well. The broken index could be renamed to
> index.broken or something for later analysis, or we forbid writing the
> reordered index to disk.
> 
> Hmm?
> 
> > write-tree: command returned error: 128
> >
> >         'git status' shows a few untracked files but is otherwise clean.
> >
> >         It looks like this check was introduced in
> 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary
> "read_index_from(): catch out of order entries when reading an index file"
> (first appearing in 2.2.0).
> >
> >         Mailing list discussion looked like it implicated third-party
> tools.  I don't recall running any other tools on this repo; it doesn't do
> much day-to-day other than a long series of 'git svn fetch'es.  (But it's
> been around for a couple of years, so who knows.)
> >
> >         At any rate, what can I do to recover from this situation?  I
> tried to locate a path with multiple index entries like this, but got no
> results:
> >
> > $ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '
> >
> >         (I originally posted on SO at
> http://stackoverflow.com/questions/30264826/; I'll update that with any
> solutions that come up here, to ease future googling.)
> > --
> > To unsubscribe from this list: send the line "unsubscribe git" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 
> 
> --
> Duy
Junio C Hamano· May 21, 2015, 16:49 UTC · re: Duy Nguyen · lore

Re: recovering from "unordered stage entries in index" error

Duy Nguyen <pclouds@gmail.com> writes:
> This message can be improved to show what entries have this problem.
> But then I don't see any way to recover the index manually. ls-files
> will die too.

Isn't this failure coming from git-svn that tries to write out a tree after it prepared whatever it wants to record in its (possibly temporary) index? I have a feeling that the index held by the end user is not broken.

Duy Nguyen· May 22, 2015, 00:17 UTC · re: Junio C Hamano · lore

Re: recovering from "unordered stage entries in index" error

On Thu, May 21, 2015 at 11:49 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 10 quoted lines
> Duy Nguyen <pclouds@gmail.com> writes:
>
>> This message can be improved to show what entries have this problem.
>> But then I don't see any way to recover the index manually. ls-files
>> will die too.
>
> Isn't this failure coming from git-svn that tries to write out a
> tree after it prepared whatever it wants to record in its (possibly
> temporary) index?  I have a feeling that the index held by the end
> user is not broken.
Ahh that would explain why ls-files works. Yep.
-- 
Duy
McHenry, Matt· May 22, 2015, 18:56 UTC · re: Duy Nguyen · lore

RE: recovering from "unordered stage entries in index" error

Show 6 quoted lines
> > Isn't this failure coming from git-svn that tries to write out a
> > tree after it prepared whatever it wants to record in its (possibly
> > temporary) index?  I have a feeling that the index held by the end
> > user is not broken.
> 
> Ahh that would explain why ls-files works. Yep.
	I created a copy of this repo + wc via rsync and tried a couple of things.  'git svn rebase -l' worked fine, but didn't fix the error.  Next, reset:

$ git svn log --limit=1 | grep ^r r231655 | avuong | 2015-05-10 10:32:16 -0400 (Sun, 10 May 2015) | 2 lines

$ git svn reset -r 231655 -p r231653 = 13a7f6d6a3f3e44ed1c8523b1a63d72fc4f0ddb9 (refs/remotes/trunk)

$ git svn fetch fatal: unordered stage entries in index write-tree: command returned error: 128

	So it doesn't seem to be specific to the revision being fetched.  I could do a more drastic 'git svn reset', but as you can see I've already fetched a lot of revs, so I'd rather avoid re-fetching if possible.
	Thanks for your help so far -- any other ideas (or requests for further debugging info) are appreciated!
Duy Nguyen· May 23, 2015, 02:06 UTC · re: McHenry, Matt · lore

Re: recovering from "unordered stage entries in index" error

On Sat, May 23, 2015 at 1:56 AM, McHenry, Matt <mmchenry@carnegielearning.com> wrote:

> $ git svn fetch
> fatal: unordered stage entries in index
> write-tree: command returned error: 128

git-svn does not create the index manually. It uses update-index or read-tree to do that. While there's still a chance of bugs in update-index that produces broken index, it's probably read-tree in this case because it assumes good order from the source tree object, which is(?) generated by git-svn. And the write-tree message supports this (the code does read-tree then write-tree).

So maybe you can do "GIT_TRACE=2 git svn fetch" and post the output. I'd expect to see something like "git read-tree <sha1>" before "fatal: unorder...". You can then use git ls-tree <sha1> to examine this tree, try to sort the file list with "LANG=C sort" and compare with the original list.

-- 
Duy
McHenry, Matt· May 23, 2015, 02:47 UTC · re: Duy Nguyen · lore

RE: recovering from "unordered stage entries in index" error

Show 5 quoted lines
> So maybe you can do "GIT_TRACE=2 git svn fetch" and post the output.
> I'd expect to see something like "git read-tree <sha1>" before "fatal:
> unorder...". You can then use git ls-tree <sha1> to examine this tree,
> try to sort the file list with "LANG=C sort" and compare with the
> original list.
	There is no read-tree in the output (below).  The sha1 that is mentioned, 74332b7, is the one for the current trunk:

$ git svn log -1 --show-commit --oneline trunk r231655 | 74332b7 | CLT: changed skill from not compound to compound.

	So not surprisingly, I guess, I get basically the same results as with the ls-files commands earlier: files with Unicode chars are quoted and sort at the front:

$ git ls-tree --name-only -r 74332b7d653cde7ba3b999cc7b0adcfd9d924440 > ls-tree $ LANG=C LC_ALL=C sort ls-tree > ls-tree-sorted-lc-all $ grep -n Ninja__Beta ls-tree* ls-tree:36974:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip" ls-tree-sorted-lc-all:89:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"

	(Just sorting with LANG=C but without LC_ALL=C produced a ton of other differences, mostly around numeric vs. lexical ordering as far as I could tell.)
	I tried this same thing with my test repo, and it exhibits the same quoted-filename-sorts-to-the-top behaviour, but does not exhibit the git svn fetch write-tree error.

$ GIT_TRACE=2 git svn fetch 22:21:16.683918 git.c:557 trace: exec: 'git-svn' 'fetch' 22:21:16.683945 run-command.c:351 trace: run_command: 'git-svn' 'fetch' 02:21:16.918593 git.c:348 trace: built-in: git 'rev-parse' '--git-dir' 02:21:16.920218 git.c:348 trace: built-in: git 'rev-parse' '--show-cdup' 02:21:16.921997 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall' 02:21:16.923609 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn.repack' 02:21:16.925164 git.c:348 trace: built-in: git 'config' '--get' 'svn.repackflags' 02:21:16.926706 git.c:348 trace: built-in: git 'config' '--get' 'svn.revision' 02:21:16.928847 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout' 02:21:16.930410 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps' 02:21:16.931963 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.localtime' 02:21:16.933538 git.c:348 trace: built-in: git 'config' '--get' 'svn.includepaths' 02:21:16.935107 git.c:348 trace: built-in: git 'config' '--get' 'svn.username' 02:21:16.936675 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache' 02:21:16.940413 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.quiet' 02:21:16.942064 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor' 02:21:16.943696 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata' 02:21:16.945344 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps' 02:21:16.947607 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.parent' 02:21:16.950737 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.addauthorfrom' 02:21:16.952532 git.c:348 trace: built-in: git 'config' '--get' 'svn.authorsprog' 02:21:16.954133 git.c:348 trace: built-in: git 'config' '--get' 'svn.ignorepaths' 02:21:16.955704 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.followparent' 02:21:16.957287 git.c:348 trace: built-in: git 'config' '--get' 'svn.configdir' 02:21:16.958930 git.c:348 trace: built-in: git 'config' '--get' 'svn.authorsfile' 02:21:16.962142 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize' 02:21:16.963913 git.c:348 trace: built-in: git 'config' '--get' 'svn.ignorerefs' 02:21:16.966130 git.c:348 trace: built-in: git 'rev-parse' '--symbolic' '--all' 02:21:16.970537 git.c:348 trace: built-in: git 'config' '-l' 02:21:16.972410 git.c:348 trace: built-in: git 'config' '-l' 02:21:16.974187 git.c:348 trace: built-in: git 'config' '--bool' 'svn.useSvmProps' 02:21:16.976074 git.c:348 trace: built-in: git 'config' '-l' 02:21:17.136056 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.branches-maxRev' 02:21:17.137928 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.tags-maxRev' 02:21:17.140124 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.url' 02:21:17.142192 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.pushurl' 02:21:17.144203 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.uuid' 02:21:17.149689 git.c:348 trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '74332b7d653cde7ba3b999cc7b0adcfd9d924440..refs/remotes/trunk' '--' 02:21:17.152412 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteRoot' 02:21:17.154482 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteUUID' 02:21:17.160391 git.c:348 trace: built-in: git 'cat-file' '--batch' 02:21:17.582641 git.c:348 trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '231655' 02:21:17.585237 git.c:348 trace: built-in: git 'config' 'svn-remote.svn.tags-maxRev' '231655' 02:21:17.590152 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.usesvmprops' 02:21:17.592063 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.nometadata' 02:21:17.593997 git.c:348 trace: built-in: git 'cat-file' 'commit' '74332b7d653cde7ba3b999cc7b0adcfd9d924440' 02:21:17.596529 git.c:348 trace: built-in: git 'write-tree' fatal: unordered stage entries in index write-tree: command returned error: 128

Duy Nguyen· May 24, 2015, 09:52 UTC · re: McHenry, Matt · lore

Re: recovering from "unordered stage entries in index" error

On Sat, May 23, 2015 at 9:47 AM, McHenry, Matt <mmchenry@carnegielearning.com> wrote:

Show 7 quoted lines
>> So maybe you can do "GIT_TRACE=2 git svn fetch" and post the output.
>> I'd expect to see something like "git read-tree <sha1>" before "fatal:
>> unorder...". You can then use git ls-tree <sha1> to examine this tree,
>> try to sort the file list with "LANG=C sort" and compare with the
>> original list.
>
>         There is no read-tree in the output (below).  The sha1 that is mentioned, 74332b7, is the one for the current trunk:

Hm.. neither read-tree nor update-index is in the output. I can see git-svn closing stderr sometimes, but not sure if that's why we don't see these commands, or something else. Could you try again with GIT_TRACE=/absolute/path/to/some/where instead of GIT_TRACE=2 and post the content of /abso../some/where?

-- 
Duy
McHenry, Matt· May 26, 2015, 13:28 UTC · re: Duy Nguyen · lore

RE: recovering from "unordered stage entries in index" error

> see these commands, or something else. Could you try again with
> GIT_TRACE=/absolute/path/to/some/where instead of GIT_TRACE=2 and post
> the content of /abso../some/where?
	It looks the same as far as I can see:

$ GIT_TRACE=/tmp/git-trace git svn fetch fatal: unordered stage entries in index write-tree: command returned error: 128

$ egrep -i 'read|tree|update|index' /tmp/git-trace 13:26:08.169921 git.c:348 trace: built-in: git 'write-tree'

$ cat /tmp/git-trace 09:26:07.402735 git.c:557 trace: exec: 'git-svn' 'fetch' 09:26:07.402784 run-command.c:351 trace: run_command: 'git-svn' 'fetch' 09:26:07.688866 git.c:348 trace: built-in: git 'rev-parse' '--show-prefix' 13:26:07.691207 git.c:348 trace: built-in: git 'rev-parse' '--git-dir' 13:26:07.693228 git.c:348 trace: built-in: git 'rev-parse' '--show-cdup' 13:26:07.695594 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.quiet' 13:26:07.697279 git.c:348 trace: built-in: git 'config' '--get' 'svn.authorsprog' 13:26:07.699030 git.c:348 trace: built-in: git 'config' '--get' 'svn.ignorepaths' 13:26:07.700914 git.c:348 trace: built-in: git 'config' '--get' 'svn.authorsfile' 13:26:07.702872 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.followparent' 13:26:07.704949 git.c:348 trace: built-in: git 'config' '--get' 'svn.configdir' 13:26:07.706674 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.localtime' 13:26:07.708590 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps' 13:26:07.710669 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps' 13:26:07.712602 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache' 13:26:07.714527 git.c:348 trace: built-in: git 'config' '--get' 'svn.username' 13:26:07.716257 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize' 13:26:07.717872 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata' 13:26:07.719905 git.c:348 trace: built-in: git 'config' '--get' 'svn.ignorerefs' 13:26:07.721623 git.c:348 trace: built-in: git 'config' '--get' 'svn.includepaths' 13:26:07.723485 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout' 13:26:07.725441 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.addauthorfrom' 13:26:07.727125 git.c:348 trace: built-in: git 'config' '--get' 'svn.repackflags' 13:26:07.729179 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn.repack' 13:26:07.731143 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor' 13:26:07.733139 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.parent' 13:26:07.734911 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall' 13:26:07.736992 git.c:348 trace: built-in: git 'config' '--get' 'svn.revision' 13:26:07.739398 git.c:348 trace: built-in: git 'rev-parse' '--symbolic' '--all' 13:26:07.744514 git.c:348 trace: built-in: git 'config' '-l' 13:26:07.746770 git.c:348 trace: built-in: git 'config' '-l' 13:26:07.749108 git.c:348 trace: built-in: git 'config' '--bool' 'svn.useSvmProps' 13:26:07.751613 git.c:348 trace: built-in: git 'config' '-l' 13:26:07.907707 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.branches-maxRev' 13:26:07.910171 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.tags-maxRev' 13:26:07.912608 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.url' 13:26:07.915539 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.pushurl' 13:26:07.917825 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.uuid' 13:26:07.919620 git.c:348 trace: built-in: git 'rev-parse' '--verify' 'refs/remotes/trunk^0' 13:26:07.923752 git.c:348 trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '74332b7d653cde7ba3b999cc7b0adcfd9d924440..refs/remotes/trunk' '--' 13:26:07.926128 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteRoot' 13:26:07.928707 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteUUID' 13:26:07.933621 git.c:348 trace: built-in: git 'cat-file' '--batch' 13:26:08.150396 git.c:348 trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '231655' 13:26:08.152963 git.c:348 trace: built-in: git 'config' 'svn-remote.svn.tags-maxRev' '231655' 13:26:08.155457 git.c:348 trace: built-in: git 'rev-parse' '--verify' 'refs/remotes/trunk^0' 13:26:08.158789 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.usesvmprops' 13:26:08.160929 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.nometadata' 13:26:08.164778 git.c:348 trace: built-in: git 'cat-file' 'commit' '74332b7d653cde7ba3b999cc7b0adcfd9d924440' 13:26:08.169921 git.c:348 trace: built-in: git 'write-tree'

Duy Nguyen· May 26, 2015, 13:37 UTC · re: McHenry, Matt · lore

Re: recovering from "unordered stage entries in index" error

On Tue, May 26, 2015 at 8:28 PM, McHenry, Matt <mmchenry@carnegielearning.com> wrote:

Show 12 quoted lines
>> see these commands, or something else. Could you try again with
>> GIT_TRACE=/absolute/path/to/some/where instead of GIT_TRACE=2 and post
>> the content of /abso../some/where?
>
>         It looks the same as far as I can see:
>
> $ GIT_TRACE=/tmp/git-trace git svn fetch
> fatal: unordered stage entries in index
> write-tree: command returned error: 128
>
> $ egrep -i 'read|tree|update|index' /tmp/git-trace
> 13:26:08.169921 git.c:348               trace: built-in: git 'write-tree'
OK I give up. Can't think of how the index is written, and by whom.
-- 
Duy
Junio C Hamano· May 22, 2015, 19:24 UTC · re: McHenry, Matt · lore

Re: recovering from "unordered stage entries in index" error

The message "unordered stage entries in index" comes only when two adjacent entries in the index are in a wrong order, e.g. "test0" should come before "test1" but somehow the index records them in the other way around. Doing something like this:

$ git ls-files >Q $ LANG=C LC_ALL=C sort Q >R $ diff Q R

may tell you which entries are wrong, even though it wouldn't show who made them wrong.

(pardon top-posting, overlong lines and typos; sent from GMail web UI)

On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt <mmchenry@carnegielearning.com> wrote:

Show 22 quoted lines
>
>         I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:
>
> $ git svn fetch
> fatal: unordered stage entries in index
> write-tree: command returned error: 128
>
>         'git status' shows a few untracked files but is otherwise clean.
>
>         It looks like this check was introduced in 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary "read_index_from(): catch out of order entries when reading an index file" (first appearing in 2.2.0).
>
>         Mailing list discussion looked like it implicated third-party tools.  I don't recall running any other tools on this repo; it doesn't do much day-to-day other than a long series of 'git svn fetch'es.  (But it's been around for a couple of years, so who knows.)
>
>         At any rate, what can I do to recover from this situation?  I tried to locate a path with multiple index entries like this, but got no results:
>
> $ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '
>
>         (I originally posted on SO at http://stackoverflow.com/questions/30264826/; I'll update that with any solutions that come up here, to ease future googling.)
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
McHenry, Matt· May 22, 2015, 19:57 UTC · re: Junio C Hamano · lore

RE: recovering from "unordered stage entries in index" error

	Yes, that does turn up some interesting stuff.  It looks like the repository contains some paths with non-ASCII characters, for example this one has some en-dashes (U+2013) in its name:

$ svn ls -R svn://dev/trunk/curriculum/Fluency | grep Ninja__Beta Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 – Ninja/FT3 – Ninja__Beta.zip

$ svn ls -R svn://dev/trunk/curriculum/Fluency | grep Ninja__Beta | od -cx
0000000   H   u   r   i   x       w   o   r   k   /   s   o   u   r   c
           7548    6972    2078    6f77    6b72    732f    756f    6372
0000020   e       f   r   o   m       M   a   y       2   0   1   4   /
           2065    7266    6d6f    4d20    7961    3220    3130    2f34
0000040   F   o   r   _   A   n   e   s   h   /   0   6       D   e   l
           6f46    5f72    6e41    7365    2f68    3630    4420    6c65
0000060   i   v   e   r   a   b   l   e   s   /   P   h   a   s   e
           7669    7265    6261    656c    2f73    6850    7361    2065
0000100   2   /   F   T   3     342 200 223       N   i   n   j   a   /
           2f32    5446    2033    80e2    2093    694e    6a6e    2f61
0000120   F   T   3     342 200 223       N   i   n   j   a   _   _   B
           5446    2033    80e2    2093    694e    6a6e    5f61    425f
0000140   e   t   a   .   z   i   p  \n
           7465    2e61    697a    0a70
0000150
	In the output of 'git ls-files', those paths appear quoted (there are almost 100 of them):

$ git ls-files | grep Ninja__Beta "curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"

$ git ls-files | grep ^\" | wc -l 89

	In the diff you suggested, 'sort' puts those paths at the absolute top of the list, while plain old ls-files puts them inline with the rest of the contents of the curriculum/ subdir:

$ grep -n Ninja__Beta Q R Q:36109:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip" R:89:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"

	Also, I have the curriculum/Fluency/ directory marked as sparse-checkout:

$ cat .git/info/sparse-checkout /* !/curriculum/Fluency/ !/curriculum/Problems/lisp/ !/curriculum/Problems/lisp_es/ !/curriculum/Problems/sdk/Geometry/ !/curriculum/Problems/sdk_es/Geometry/ !/curriculum/Problems/sdk/Test-Questions/ !/curriculum/Problems/sdk_es/Test-Questions/ !/curriculum/Problems/sdk/Grammar/

	However, I tried to construct a test case that would reproduce this with a simple SVN repo containing a file created by 'touch "make-git-svn-$(echo -e '\u201c')unhappy$(echo -e '\u201d')"', but could not get it to fail.  So there may be something more subtle going on here ...
Show 57 quoted lines
> -----Original Message-----
> From: jch2355@gmail.com [mailto:jch2355@gmail.com] On Behalf Of Junio C
> Hamano
> Sent: Friday, May 22, 2015 15:25
> To: McHenry, Matt
> Cc: git@vger.kernel.org
> Subject: Re: recovering from "unordered stage entries in index" error
> 
> The message "unordered stage entries in index" comes only when
> two adjacent entries in the index are in a wrong order, e.g. "test0"
> should come before "test1" but somehow the index records them
> in the other way around. Doing something like this:
> 
> $ git ls-files >Q
> $ LANG=C LC_ALL=C sort Q >R
> $ diff Q R
> 
> may tell you which entries are wrong, even though it wouldn't show
> who made them wrong.
> 
> (pardon top-posting, overlong lines and typos; sent from GMail web UI)
> 
> On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt
> <mmchenry@carnegielearning.com> wrote:
> >
> >         I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now
> unable to run 'git svn fetch' in one of my repositories:
> >
> > $ git svn fetch
> > fatal: unordered stage entries in index
> > write-tree: command returned error: 128
> >
> >         'git status' shows a few untracked files but is otherwise clean.
> >
> >         It looks like this check was introduced in
> 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary
> "read_index_from(): catch out of order entries when reading an index file"
> (first appearing in 2.2.0).
> >
> >         Mailing list discussion looked like it implicated third-party
> tools.  I don't recall running any other tools on this repo; it doesn't do
> much day-to-day other than a long series of 'git svn fetch'es.  (But it's
> been around for a couple of years, so who knows.)
> >
> >         At any rate, what can I do to recover from this situation?  I
> tried to locate a path with multiple index entries like this, but got no
> results:
> >
> > $ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '
> >
> >         (I originally posted on SO at
> http://stackoverflow.com/questions/30264826/; I'll update that with any
> solutions that come up here, to ease future googling.)
> > --
> > To unsubscribe from this list: send the line "unsubscribe git" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
Junio C Hamano· May 23, 2015, 17:06 UTC · re: McHenry, Matt · lore

Re: recovering from "unordered stage entries in index" error

"McHenry, Matt" <mmchenry@carnegielearning.com> writes:
> 	Yes, that does turn up some interesting stuff.  It looks
> 	like the repository contains some paths with non-ASCII
> 	characters, for example this one has some en-dashes (U+2013)
> 	in its name:

Then the recipe in the message you are responding to shoulld be changed somewhat to tell Git that it is OK to show non-ASCII in verbatim.

Perhaps doing this once
    git config core.quotepath false
before asking ls-files to show the paths should be sufficient.
Thanks.
McHenry, Matt· May 23, 2015, 21:36 UTC · lore

RE: recovering from "unordered stage entries in index" error

Per Junio's email, with core.quotepath=false, there are no differences with sorting in either ls-files or the tree named in the GIT_TRACE_2 output:
$ git config --local core.quotepath false

$ git ls-tree --name-only -r 74332b7d653cde7ba3b999cc7b0adcfd9d924440 > ls-tree $ LANG=C LC_ALL=C sort ls-tree > ls-tree-sorted-lc-all $ diff -s ls-tree ls-tree-sorted-lc-all Files ls-tree and ls-tree-sorted-lc-all are identical

$ git ls-files > ls-files $ LANG=C LC_ALL=C sort ls-files > ls-files-sorted-lc-all $ diff -s ls-files ls-files-sorted-lc-all Files ls-files and ls-files-sorted-lc-all are identical

________________________________________
From: McHenry, Matt
Sent: Friday, May 22, 2015 10:47 PM
To: Duy Nguyen
Cc: Junio C Hamano; git@vger.kernel.org
Subject: RE: recovering from "unordered stage entries in index" error
Show 5 quoted lines
> So maybe you can do "GIT_TRACE=2 git svn fetch" and post the output.
> I'd expect to see something like "git read-tree <sha1>" before "fatal:
> unorder...". You can then use git ls-tree <sha1> to examine this tree,
> try to sort the file list with "LANG=C sort" and compare with the
> original list.
        There is no read-tree in the output (below).  The sha1 that is mentioned, 74332b7, is the one for the current trunk:

$ git svn log -1 --show-commit --oneline trunk r231655 | 74332b7 | CLT: changed skill from not compound to compound.

        So not surprisingly, I guess, I get basically the same results as with the ls-files commands earlier: files with Unicode chars are quoted and sort at the front:

$ git ls-tree --name-only -r 74332b7d653cde7ba3b999cc7b0adcfd9d924440 > ls-tree $ LANG=C LC_ALL=C sort ls-tree > ls-tree-sorted-lc-all $ grep -n Ninja__Beta ls-tree* ls-tree:36974:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip" ls-tree-sorted-lc-all:89:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"

        (Just sorting with LANG=C but without LC_ALL=C produced a ton of other differences, mostly around numeric vs. lexical ordering as far as I could tell.)
        I tried this same thing with my test repo, and it exhibits the same quoted-filename-sorts-to-the-top behaviour, but does not exhibit the git svn fetch write-tree error.

$ GIT_TRACE=2 git svn fetch 22:21:16.683918 git.c:557 trace: exec: 'git-svn' 'fetch' 22:21:16.683945 run-command.c:351 trace: run_command: 'git-svn' 'fetch' 02:21:16.918593 git.c:348 trace: built-in: git 'rev-parse' '--git-dir' 02:21:16.920218 git.c:348 trace: built-in: git 'rev-parse' '--show-cdup' 02:21:16.921997 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall' 02:21:16.923609 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn.repack' 02:21:16.925164 git.c:348 trace: built-in: git 'config' '--get' 'svn.repackflags' 02:21:16.926706 git.c:348 trace: built-in: git 'config' '--get' 'svn.revision' 02:21:16.928847 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout' 02:21:16.930410 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps' 02:21:16.931963 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.localtime' 02:21:16.933538 git.c:348 trace: built-in: git 'config' '--get' 'svn.includepaths' 02:21:16.935107 git.c:348 trace: built-in: git 'config' '--get' 'svn.username' 02:21:16.936675 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache' 02:21:16.940413 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.quiet' 02:21:16.942064 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor' 02:21:16.943696 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata' 02:21:16.945344 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps' 02:21:16.947607 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.parent' 02:21:16.950737 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.addauthorfrom' 02:21:16.952532 git.c:348 trace: built-in: git 'config' '--get' 'svn.authorsprog' 02:21:16.954133 git.c:348 trace: built-in: git 'config' '--get' 'svn.ignorepaths' 02:21:16.955704 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'svn.followparent' 02:21:16.957287 git.c:348 trace: built-in: git 'config' '--get' 'svn.configdir' 02:21:16.958930 git.c:348 trace: built-in: git 'config' '--get' 'svn.authorsfile' 02:21:16.962142 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize' 02:21:16.963913 git.c:348 trace: built-in: git 'config' '--get' 'svn.ignorerefs' 02:21:16.966130 git.c:348 trace: built-in: git 'rev-parse' '--symbolic' '--all' 02:21:16.970537 git.c:348 trace: built-in: git 'config' '-l' 02:21:16.972410 git.c:348 trace: built-in: git 'config' '-l' 02:21:16.974187 git.c:348 trace: built-in: git 'config' '--bool' 'svn.useSvmProps' 02:21:16.976074 git.c:348 trace: built-in: git 'config' '-l' 02:21:17.136056 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.branches-maxRev' 02:21:17.137928 git.c:348 trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.tags-maxRev' 02:21:17.140124 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.url' 02:21:17.142192 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.pushurl' 02:21:17.144203 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.uuid' 02:21:17.149689 git.c:348 trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '74332b7d653cde7ba3b999cc7b0adcfd9d924440..refs/remotes/trunk' '--' 02:21:17.152412 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteRoot' 02:21:17.154482 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteUUID' 02:21:17.160391 git.c:348 trace: built-in: git 'cat-file' '--batch' 02:21:17.582641 git.c:348 trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '231655' 02:21:17.585237 git.c:348 trace: built-in: git 'config' 'svn-remote.svn.tags-maxRev' '231655' 02:21:17.590152 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.usesvmprops' 02:21:17.592063 git.c:348 trace: built-in: git 'config' '--get' 'svn-remote.svn.nometadata' 02:21:17.593997 git.c:348 trace: built-in: git 'cat-file' 'commit' '74332b7d653cde7ba3b999cc7b0adcfd9d924440' 02:21:17.596529 git.c:348 trace: built-in: git 'write-tree' fatal: unordered stage entries in index write-tree: command returned error: 128

← back to recent threads