2.11.0-rc1 will not be tagged for a few days

I regret to report that I won't be able to tag 2.11-rc1 as scheduled in tinyurl.com/gitCal (I am feverish and my brain is not keeping track of things correctly) any time soon. I'll report back an updated schedule when able.

I'd appreciate that people keep discussing and exchanging patches that will not be in 2.11 final to give a head-start to topics so that people can agree on the designs and implementations in the meantime, but one thing I'd like to see when I come back is somebody tell me (when asked) what fix-up pathes that are "must to apply, or there is no point tagging 2.11-rc1 while keeping it broken" are found on the list. I am aware of only two right now ("cast enum to int to work around compiler warning", in Dscho's prepare sequencer series, and "wc -l may give leading whitespace" fix J6t pointed out in Lars's filter process series), but it is more than likely that I am missing a few more.

Thanks.  Going back to bed...

On Sun, Nov 06, 2016 at 06:32:05PM -0800, Junio C Hamano wrote:

Take your time. Even if we end up bumping the release by a whole week, I don't think it's a big deal (which seems likely given the holiday in the middle, unless you really want to release on Thanksgiving).

In addition to J6t's fix in t0021, we need mine that you queued in jk/filter-process-fix.

I think we also need to make a final decision on the indent/compaction heuristic naming. After reading Michael's [0], and the claim by you and Stefan that the "compaction" name was declared sufficiently experimental that we could later take it away, I'm inclined to follow this plan:

  1. Ship v2.11 with what is in master; i.e., both compaction and indent
     heuristics, triggerable by config or command line.

  2. Post-v2.11, retire the compaction heuristic as a failed experiment.
     Keeping it in v2.11 doesn't hurt anything (it was already
     released), and lets us take our time coming up with and cooking the
     patch.

  3. Post-v2.11, flip the default for diff.indentHeuristic to "true".
     Keep at least the command line option around indefinitely for
     experimenting (i.e., "this diff looks funny; I wonder if
     --no-indent-heuristic makes it look better").

     Config option can either stay or go at that point. I have no
     preference.

The nice thing about that plan is it punts on merging any new code to post-v2.11. :)

Another possible regression came up today in [1]. I haven't worked up a patch yet, but I'll do so in the next day or so.

I think that's where we're at now. I'll keep collecting and can give you the full list when you're back in action.

Get well.

[0] http://public-inbox.org/git/8dbbd28b-af60-5e66-ae27-d7cddca233dc@alum.mit.edu/

[1] http://public-inbox.org/git/20161108003034.apydvv3bav3s7ehq@sigill.intra.peff.net/

Am 08.11.2016 um 01:40 schrieb Jeff King:

Just to get things straight: Of my two patches, this one ("uniq -c variations")

https://public-inbox.org/git/c842e0a7-b032-e0c4-0995-f11d93c17c0a@kdbg.org/

is a bug fix in my environment, and I have a suspicion that it is also required in other less frequently tested environments (Solaris? BSD variants?)

The other one, which you most likely remember as dealing with "leading whitespace in wc -c"

https://public-inbox.org/git/b87ddffd-3de1-4481-b484-9f03a73b6ad1@kdbg.org/

is "only" an optimization. The link points at the final version.

-- Hannes

On Tue, Nov 08, 2016 at 07:25:26AM +0100, Johannes Sixt wrote:

Thanks for clarifying. I think it's probably worth applying both.

I'm collecting v2.11-rc1 topics in the "refs/heads/for-junio/" section of git://github.com/peff/git.git.

I've also got proposed merges for "master" there, though note that none of the topics has actually cooked at all in next (the fixes are trivial enough that it may be OK, though).

-Peff

Jeff King <peff@peff.net> writes:

I am not quite back up at full steam yet, but I expect I'd be more or less functioning tomorrow, so I'll fetch them from your tree and continue.

Thanks.

On Tue, Nov 08, 2016 at 10:29:07PM -0800, Junio C Hamano wrote:

It looks like Johannes prefers replacements for some of the fixes from yesterday. Just FYI that you probably do not want to take the "master" I pushed there literally, but instead just view it as a proposed list of topics.

I did not really expect you to take it literally in the first place. It's just that I found myself writing up notes like "this should be merged", and it occurred to me that I could communicate the same things by sending you a proposed history. So I'm curious if you find dissecting it via "git log" more or less convenient than a list of notes. :)

-Peff

Jeff King <peff@peff.net> writes:

Yes, I think this is a very readable form of communication.

What's especially nice in what you did is that comparing outputs of these two commands

    $ git log --no-merges --oneline master..peff/for-junio/master
    $ git log --no-merges --oneline ^pu master..peff/for-junio/master

clearly shows that you reused what I already had in 'pu' and the ones missing from 'pu' are the ones I need to check and possibly may want to sign-off myself.

I also need to note that while this is very handy format for the recipient, hence a very good way to do "pass the pumpkin" between maintainers, it is a less "open" way of communication than laying out everything on the list (cf. http://youtu.be/L8OOzaqS37s), but it is the most appropriate method in this case, I would think.

As to that change, I agree with you that I do not care too deeply about the shape of an interim step as long as the finished product uses the better one between the alternatives, but at the same time, because we are including it as a hot-fix in a released product, even though it is an interim step in the bigger picture, I want it to be using the better one, too. So I am leaning towards taking what you queued for me for the reasons you stated in the other thread [*1*].

That would probably mean Dscho needs a bit of rebase in the remaining parts of his series that adds new elements to the enum, but I am hoping that he can afford that time now in the feature freeze period. If we take the _INVALID thing instead now, the remaining series from Dscho that we would review and queue for the next cycle would need to begin with a "fixup" patch that stops doing the _INVALID thing and instead using the "compare after casting to size_t", so it is just the matter of doing the adjustment before or after the remainder of the series are posted on the list for the next release, and the smaller number of "oops that was not nice, let's fix it" patches in the published history, the better the long term health of the project, I would think.

The other fixes in your collection looked all sensible.  Thanks.

[Footnote]

*1* The convention of keeping _INVALID at the end can be arguably
    made safe from future programmer screw-ups, as long as it is
    guarded by a good comment nearby.  But the solution to cast
    would avoid having to have that potential brittleness in the
    first place.  I find the other merit of taking care of negative
    enum automatically quite attractive, too.

Jeff King <peff@peff.net> writes:

OK, I can go with that plan.  Thanks for an outline.

Junio C Hamano <gitster@pobox.com> writes:

I pushed some updates out on 'master' today. Between 'master' and 'pu' on the first-parent history there is a merge 52975d2b1f ("Merge branch 'ls/macos-update' into jch", 2016-11-10) and that matches what I expect to be in -rc1 tomorrow (modulo RelNotes and the actual version tag), unless there is a showstopper regresion reported, in which case we may want to first look into reverting the whole series that introduced the regression before considering to pile on fix-up patches.

http://tinyurl.com/gitCal has been redrawn, with -rc1 scheduled for tomorrow on 11th, -rc2 on the 17th, and final on the 23rd.

Thanks.

Junio C Hamano <gitster@pobox.com> writes:

Please make that 71d1bcb661 ("Merge branch 'ls/macos-update' into jch", 2016-11-10); among the three extra topics that is not yet in 'master', as/merge-attr-sleep topic has been updated with a better fix from Dscho.

Thanks.

Hi Junio,

On Thu, 10 Nov 2016, Junio C Hamano wrote:

Which means that t0021 is now broken also on `master` when running in Git for Windows' SDK.

To add insult to injury, t5615 is now broken, too. I do not recall it being broken in any of my CI builds in any of the integration branches before.

Ciao, Dscho

Hi,

On Fri, 11 Nov 2016, Johannes Schindelin wrote:

To be precise, it says:

	# failed 7 among 23 test(s)

Running with -i -v -x shows that the first broken test is this one:

-- snipsnap --
expecting success:
        test_config_global filter.protocol.process "rot13-filter.pl clean smudge" &&
        test_config_global filter.protocol.required true &&
        rm -rf repo &&
        mkdir repo &&
        (
                cd repo &&
                git init &&

                echo "git-stderr.log" >.gitignore &&
                echo "*.r filter=protocol" >.gitattributes &&
                git add . &&
                git commit . -m "test commit 1" &&
                git branch empty-branch &&

                cp "$TEST_ROOT/test.o" test.r &&
                cp "$TEST_ROOT/test2.o" test2.r &&
                mkdir testsubdir &&
                cp "$TEST_ROOT/test3 'sq',\$x.o" "testsubdir/test3 'sq',\$x.r" &&
                >test4-empty.r &&

                S=$(file_size test.r) &&
                S2=$(file_size test2.r) &&
                S3=$(file_size "testsubdir/test3 'sq',\$x.r") &&

                filter_git add . &&
                cat >expected.log <<-EOF &&
                        START
                        init handshake complete
                        IN: clean test.r $S [OK] -- OUT: $S . [OK]
                        IN: clean test2.r $S2 [OK] -- OUT: $S2 . [OK]
                        IN: clean test4-empty.r 0 [OK] -- OUT: 0  [OK]
                        IN: clean testsubdir/test3 'sq',\$x.r $S3 [OK] -- OUT: $S3 . [OK]
                        STOP
                EOF
                test_cmp_count expected.log rot13-filter.log &&

                filter_git commit . -m "test commit 2" &&
                cat >expected.log <<-EOF &&
                        START
                        init handshake complete
                        IN: clean test.r $S [OK] -- OUT: $S . [OK]
                        IN: clean test2.r $S2 [OK] -- OUT: $S2 . [OK]
                        IN: clean test4-empty.r 0 [OK] -- OUT: 0  [OK]
                        IN: clean testsubdir/test3 'sq',\$x.r $S3 [OK] -- OUT: $S3 . [OK]
                        IN: clean test.r $S [OK] -- OUT: $S . [OK]
                        IN: clean test2.r $S2 [OK] -- OUT: $S2 . [OK]
                        IN: clean test4-empty.r 0 [OK] -- OUT: 0  [OK]
                        IN: clean testsubdir/test3 'sq',\$x.r $S3 [OK] -- OUT: $S3 . [OK]
                        STOP
                EOF
                test_cmp_count expected.log rot13-filter.log &&

                rm -f test2.r "testsubdir/test3 'sq',\$x.r" &&

                filter_git checkout --quiet --no-progress . &&
                cat >expected.log <<-EOF &&
                        START
                        init handshake complete
                        IN: smudge test2.r $S2 [OK] -- OUT: $S2 . [OK]
                        IN: smudge testsubdir/test3 'sq',\$x.r $S3 [OK] -- OUT: $S3 . [OK]
                        STOP
                EOF
                test_cmp_exclude_clean expected.log rot13-filter.log &&

                filter_git checkout --quiet --no-progress empty-branch &&
                cat >expected.log <<-EOF &&
                        START
                        init handshake complete
                        IN: clean test.r $S [OK] -- OUT: $S . [OK]
                        STOP
                EOF
                test_cmp_exclude_clean expected.log rot13-filter.log &&

                filter_git checkout --quiet --no-progress master &&
                cat >expected.log <<-EOF &&
                        START
                        init handshake complete
                        IN: smudge test.r $S [OK] -- OUT: $S . [OK]
                        IN: smudge test2.r $S2 [OK] -- OUT: $S2 . [OK]
                        IN: smudge test4-empty.r 0 [OK] -- OUT: 0  [OK]
                        IN: smudge testsubdir/test3 'sq',\$x.r $S3 [OK] -- OUT: $S3 . [OK]
                        STOP
                EOF
                test_cmp_exclude_clean expected.log rot13-filter.log &&

                test_cmp_committed_rot13 "$TEST_ROOT/test.o" test.r &&
                test_cmp_committed_rot13 "$TEST_ROOT/test2.o" test2.r &&
                test_cmp_committed_rot13 "$TEST_ROOT/test3 'sq',\$x.o" "testsubdir/test3 'sq',\$x.r"
        )

++ test_config_global filter.protocol.process 'rot13-filter.pl clean smudge'
++ test_when_finished 'test_unconfig --global '\''filter.protocol.process'\'''
++ test 0 = 0
++ test_cleanup='{ test_unconfig --global '\''filter.protocol.process'\''
                } && (exit "$eval_ret"); eval_ret=$?; :'
++ git config --global filter.protocol.process 'rot13-filter.pl clean smudge'
++ test_config_global filter.protocol.required true
++ test_when_finished 'test_unconfig --global '\''filter.protocol.required'\'''
++ test 0 = 0
++ test_cleanup='{ test_unconfig --global '\''filter.protocol.required'\''
                } && (exit "$eval_ret"); eval_ret=$?; { test_unconfig --global '\''filter.protocol.process'\''
                } && (exit "$eval_ret"); eval_ret=$?; :'
++ git config --global filter.protocol.required true
++ rm -rf repo
++ mkdir repo
++ cd repo
++ git init
Initialized empty Git repository in C:/git-sdk-64/usr/src/git/wip2/t/trash directory.t0021-conversion/repo/.git/
++ echo git-stderr.log
++ echo '*.r filter=protocol'
++ git add .
++ git commit . -m 'test commit 1'
[master (root-commit) aa5dd37] test commit 1
 Author: A U Thor <author@example.com>
 2 files changed, 2 insertions(+)
 create mode 100644 .gitattributes
 create mode 100644 .gitignore
++ git branch empty-branch
++ cp 'C:/git-sdk-64/usr/src/git/wip2/t/trash directory.t0021-conversion/test.o' test.r
++ cp 'C:/git-sdk-64/usr/src/git/wip2/t/trash directory.t0021-conversion/test2.o' test2.r
++ mkdir testsubdir
++ cp 'C:/git-sdk-64/usr/src/git/wip2/t/trash directory.t0021-conversion/test3 '\''sq'\'',$x.o' 'testsubdir/test3 '\''sq'\'',$x.r'
+++ file_size test.r
+++ perl -e 'print -s $ARGV[0]' test.r
+++ command /usr/bin/perl -e 'print -s $ARGV[0]' test.r
+++ /usr/bin/perl -e 'print -s $ARGV[0]' test.r
++ S=57
+++ file_size test2.r
+++ perl -e 'print -s $ARGV[0]' test2.r
+++ command /usr/bin/perl -e 'print -s $ARGV[0]' test2.r
+++ /usr/bin/perl -e 'print -s $ARGV[0]' test2.r
++ S2=14
+++ file_size 'testsubdir/test3 '\''sq'\'',$x.r'
+++ perl -e 'print -s $ARGV[0]' 'testsubdir/test3 '\''sq'\'',$x.r'
+++ command /usr/bin/perl -e 'print -s $ARGV[0]' 'testsubdir/test3 '\''sq'\'',$x.r'
+++ /usr/bin/perl -e 'print -s $ARGV[0]' 'testsubdir/test3 '\''sq'\'',$x.r'
++ S3=49
++ filter_git add .
++ rm -f rot13-filter.log
++ git add .
+ test_eval_ret_=128
+ want_trace
+ test t = t
+ test t = t
+ set +x
error: last command exited with $?=128
not ok 15 - required process filter should filter data
#
#               test_config_global filter.protocol.process
#               "rot13-filter.pl clean smudge" &&
#               test_config_global filter.protocol.required true &&
#               rm -rf repo &&
#               mkdir repo &&
#               (
#                       cd repo &&
#                       git init &&
#
#                       echo "git-stderr.log" >.gitignore &&
#                       echo "*.r filter=protocol" >.gitattributes &&
#                       git add . &&
#                       git commit . -m "test commit 1" &&
#                       git branch empty-branch &&
#
#                       cp "$TEST_ROOT/test.o" test.r &&
#                       cp "$TEST_ROOT/test2.o" test2.r &&
#                       mkdir testsubdir &&
#                       cp "$TEST_ROOT/test3 'sq',\$x.o" "testsubdir/test3
#                       'sq',\$x.r" &&
#                       >test4-empty.r &&
#
#                       S=$(file_size test.r) &&
#                       S2=$(file_size test2.r) &&
#                       S3=$(file_size "testsubdir/test3 'sq',\$x.r") &&
#
#                       filter_git add . &&
#                       cat >expected.log <<-EOF &&
#                               START
#                               init handshake complete
#                               IN: clean test.r $S [OK] -- OUT: $S . [OK]
#                               IN: clean test2.r $S2 [OK] -- OUT: $S2 .
#                               [OK]
#                               IN: clean test4-empty.r 0 [OK] -- OUT: 0
#                               [OK]
#                               IN: clean testsubdir/test3 'sq',\$x.r $S3
#                               [OK] -- OUT: $S3 . [OK]
#                               STOP
#                       EOF
#                       test_cmp_count expected.log rot13-filter.log &&
#
#                       filter_git commit . -m "test commit 2" &&
#                       cat >expected.log <<-EOF &&
#                               START
#                               init handshake complete
#                               IN: clean test.r $S [OK] -- OUT: $S . [OK]
#                               IN: clean test2.r $S2 [OK] -- OUT: $S2 .
#                               [OK]
#                               IN: clean test4-empty.r 0 [OK] -- OUT: 0
#                               [OK]
#                               IN: clean testsubdir/test3 'sq',\$x.r $S3
#                               [OK] -- OUT: $S3 . [OK]
#                               IN: clean test.r $S [OK] -- OUT: $S . [OK]
#                               IN: clean test2.r $S2 [OK] -- OUT: $S2 .
#                               [OK]
#                               IN: clean test4-empty.r 0 [OK] -- OUT: 0
#                               [OK]
#                               IN: clean testsubdir/test3 'sq',\$x.r $S3
#                               [OK] -- OUT: $S3 . [OK]
#                               STOP
#                       EOF
#                       test_cmp_count expected.log rot13-filter.log &&
#
#                       rm -f test2.r "testsubdir/test3 'sq',\$x.r" &&
#
#                       filter_git checkout --quiet --no-progress . &&
#                       cat >expected.log <<-EOF &&
#                               START
#                               init handshake complete
#                               IN: smudge test2.r $S2 [OK] -- OUT: $S2 .
#                               [OK]
#                               IN: smudge testsubdir/test3 'sq',\$x.r $S3
#                               [OK] -- OUT: $S3 . [OK]
#                               STOP
#                       EOF
#                       test_cmp_exclude_clean expected.log
#                       rot13-filter.log &&
#
#                       filter_git checkout --quiet --no-progress
#                       empty-branch &&
#                       cat >expected.log <<-EOF &&
#                               START
#                               init handshake complete
#                               IN: clean test.r $S [OK] -- OUT: $S . [OK]
#                               STOP
#                       EOF
#                       test_cmp_exclude_clean expected.log
#                       rot13-filter.log &&
#
#                       filter_git checkout --quiet --no-progress master
#                       &&
#                       cat >expected.log <<-EOF &&
#                               START
#                               init handshake complete
#                               IN: smudge test.r $S [OK] -- OUT: $S .
#                               [OK]
#                               IN: smudge test2.r $S2 [OK] -- OUT: $S2 .
#                               [OK]
#                               IN: smudge test4-empty.r 0 [OK] -- OUT: 0
#                               [OK]
#                               IN: smudge testsubdir/test3 'sq',\$x.r $S3
#                               [OK] -- OUT: $S3 . [OK]
#                               STOP
#                       EOF
#                       test_cmp_exclude_clean expected.log
#                       rot13-filter.log &&
#
#                       test_cmp_committed_rot13 "$TEST_ROOT/test.o"
#                       test.r &&
#                       test_cmp_committed_rot13 "$TEST_ROOT/test2.o"
#                       test2.r &&
#                       test_cmp_committed_rot13 "$TEST_ROOT/test3
#                       'sq',\$x.o" "testsubdir/test3 'sq',\$x.r"
#               )
#

Part of the reason is that Hannes' squash got lost: http://public-inbox.org/git/d36d8b51-f2d7-a2f5-89ea-369f49556e10@kdbg.org/

@Junio: Can you apply this (see discussion with Peff linked to email above).

@Dscho: There is still one remaining new issue with t0021 ... investigating!

Confirmed on my machine.

- Lars

"17 - required process filter should be used only for "clean" operation only" seems flaky.

I don't have any more time today but I will look into it tomorrow.

- Lars

FWIW: This patch (which would be the right thing to do anyways) seems to fix 
the flakyness but I can't be sure ... it needs to run longer...

Am 11.11.2016 um 18:38 schrieb Lars Schneider:

Good to know that I am not alone. This one fails consistently for me. I dug into it a bit today, but it drives me mad. Process Monitor reports that the redirected-to file (git-stderr.log) gets marked as "Delete pending" by git.exe, but I have absolutely no clue where this is coming from. Git.exe has no business fuzzing with the file; it just has to write() something into it, if at all. When 'git checkout' ends, the file is closed, and removed due to the "Delete pending" mark. At the same time, git.exe finishes with non-zero exit code. For whatever reason.

If I remove the redirection altogether, the test passes for me consistently. The redirection isn't necessary anyway, it looks like a debugging left-over.

-- Hannes

Johannes Sixt <j6t@kdbg.org> writes:

OK, then let's have

	filter_git () {
		rm -f rot13-filter.log &&
		git "$@"
                ...

and call that -rc1; to Windows folks, this is sweeping a mysterious failure around git.exe under the rug, but for the purpose of the overall project, it is far lower priority to figure out the reason why a merely-redirected-into file is considered to be delete-pending on WIndows than have something the end user can start testing and complaining about new features and regressions.

Thanks.

Junio C Hamano <gitster@pobox.com> writes:

That is, to queue this on top of ls/filter-process and merge it down.

-- >8 --
Subject: t0021: remove debugging cruft

The redirection of the standard error stream to a temporary file is a leftover cruft during debugging. Remove it.

Besides, it is reported by folks on the Windows that the test is flaky with this redirection; somebody gets confused and this merely-redirected-to file gets marked as delete-pending by git.exe and makes it finish with a non-zero exit status when "git checkout" finishes. Windows folks may want to figure that one out, but for the purpose of this test, it shouldn't become a show-stopper.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t0021-conversion.sh | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

Am 11.11.2016 um 22:07 schrieb Junio C Hamano:

Nicely summarized! Thank you very much!

I've double-checked again that the redirection is really not used by the tests. The name does occur in a .gitignore, but I guess that is not important for the background filter process. It would be good if Lars could confirm this.

Yesterday, I wasn't sure if silencing stderr was required by the test. Junio's assessment and solution is spot on.

Thanks a lot, Lars

Lars Schneider <larsxschneider@gmail.com> writes:

Thanks for a quick and usable report followed by a quick fix.