From: Jeff King <peff@peff.net>
Date: Thu, 06 Jun 2019 17:35:17 GMT
Subject: Re: Git self test failure on Solaris 11.3
Message-ID: <20190606173516.GA25089@sigill.intra.peff.net>
In-Reply-To: <CAPig+cS98DeSaF6pNqKCn6bAy=jjiTuc3AZpPB2cXm6hQ6vcqQ@mail.gmail.com>

On Thu, Jun 06, 2019 at 01:18:01PM -0400, Eric Sunshine wrote:

> > > not ok 12 - check normal git operations: twelve packs
> > > #
> > > #                       midx_git_two_modes "rev-list --objects --all" &&
> > > #                       midx_git_two_modes "log --raw" &&
> > > #                       midx_git_two_modes "count-objects --verbose" &&
> > > #                       midx_git_two_modes "cat-file
> > > --batch-all-objects --buffer --batch-check" &&
> > > #                       midx_git_two_modes "cat-file
> > > --batch-all-objects --buffer --batch-check --unsorted" sorted
> > > #
> [...]
> 
> Jeff Walton reported this to me privately. I'm not familiar with this
> code and don't have time presently to investigate it, so I'm
> forwarding it to the list in the hope that someone who knows the code
> intimately (Stolee) or someone who has touched this code (Peff) might
> have some idea about it. The failure is intermittent, so perhaps the
> output of some command is not stable sort-wise(?).

I can't seem to get it to fail for me, even under high load. Do you know
which of those commands is failing (i.e., what does "-v -x" say)?

If it's the rev-list or log output, it could be due to traversal order.
I notice that the commit creation doesn't increment the timestamps. This
might help:

diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index 70926b5bc0..79aa97efce 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -60,6 +60,7 @@ commit_and_list_objects () {
 	} >file_101 &&
 	git update-index --add file_101 &&
 	tree=$(git write-tree) &&
+	test_tick &&
 	commit=$(git commit-tree $tree -p HEAD</dev/null) &&
 	{
 		echo $tree &&

Though I think in each case that the results should be deterministic for
a given set of commits (and the tests here are just running over the
same set of commits with and without midx; so we should see the same
output in both cases).

The normal "cat-file" output should be sorted by sha1, so should be
stable. The unsorted output will be in whatever order we happen to hit
the packfiles, which _would_ be subject to timestamp variations.
However, Jeffrey's email mentions Git 2.21, which predates my b4a14394af
(t5319: fix bogus cat-file argument, 2019-04-05). So before that,
"cat-file --unsorted" should always be producing no output. :)

After the switch to correctly use --unordered, I could believe there
might be a timestamp issue, but the "sorted" flag there means that we
should be sorting the output before comparing.

I can't reproduce the intermittent failure either on 2.21.0, or with
v2.22.0-rc3.

-Peff