# Regression in patch add? 22 messages from 2018-04-15 to 2018-07-11. Participants: mqudsi@neosmart.net, Martin Ågren, Phillip Wood, Oliver Joseph Ash, Junio C Hamano, Jacob Keller, Eric Sunshine, Jeff Felchner. Thread: https://gitlist.dev/t/48293 ## mqudsi@neosmart.net, 2018-04-15 12:21 Subject: Regression in patch add? Message-ID: <01010162c940b8bb-d8139971-3ee2-4cd6-bb19-35126d46753b-000000@us-west-2.amazonses.com> URL: https://gitlist.dev/e/01010162c940b8bb-d8139971-3ee2-4cd6-bb19-35126d46753b-000000%40us-west-2.amazonses.com ``` Hello all, I'm currently running the latest version of git built from `master`, and I'm running into what appears to be a regression in the behavior of the piecewise `git add -p` when applying a manually edited chunk. I first run `git add -p`, then manually edit a chunk (after hitting `s` once, if it matters). The chunk originally contains the following: ˋˋˋdiff # Manual hunk edit mode -- see bottom for a quick guide @@ -20,7 +20,7 @@ "call dein#add('Shougo/dein.vim', {'rev': 'master'}) " Add or remove your plugins here: - " call dein#add('flazz/vim-colorschemes') - call dein#add('Haron-Prime/evening_vim') + call dein#add('flazz/vim-colorschemes') + call dein#add('danilo-augusto/vim-afterglow') "core plugins that change the behavior of vim and how we use it globally ˋˋˋ Under git 2.7.4, I can edit it to the following, which is accepted without a problem: ˋˋˋdiff # Manual hunk edit mode -- see bottom for a quick guide @@ -20,7 +20,7 @@ "call dein#add('Shougo/dein.vim', {'rev': 'master'}) " Add or remove your plugins here: - " call dein#add('flazz/vim-colorschemes') - call dein#add('Haron-Prime/evening_vim') + call dein#add('flazz/vim-colorschemes') + call dein#add('Haron-Prime/evening_vim') "core plugins that change the behavior of vim and how we use it globally ˋˋˋ All I did here was remove one `+` line and manually add another (which is a variant of the second `-` line). Under git 2.17.0.252.gfe0a9ea, the same piece is opened in $VISUAL for editing (and if left unmodified applies OK), but when modified in the to the same exact value, after exiting the editor I receive the following error from git: error: patch fragment without header at line 15: @@ -25,7 +25,8 @@ I'm not sure what to make of this. Thank you, Mahmoud Al-Qudsi NeoSmart Technologies ``` ## Martin Ågren, 2018-04-15 13:59 Subject: Re: Regression in patch add? Message-ID: URL: https://gitlist.dev/e/CAN0heSqCZWR1OD4k%2B-_OBdtCjCNW-UexLo4P-C5XyBqfU6KBEA%40mail.gmail.com In-Reply-To: <01010162c940b8bb-d8139971-3ee2-4cd6-bb19-35126d46753b-000000@us-west-2.amazonses.com> ``` Hi Mahmoud On 15 April 2018 at 14:21, wrote: > I first run `git add -p`, then manually edit a chunk (after hitting `s` > once, if it matters). The chunk originally contains the following: [...] > Under git 2.7.4, I can edit it to the following, which is accepted > without a problem: > > ˋˋˋdiff > # Manual hunk edit mode -- see bottom for a quick guide > @@ -20,7 +20,7 @@ > "call dein#add('Shougo/dein.vim', {'rev': 'master'}) > > " Add or remove your plugins here: > - " call dein#add('flazz/vim-colorschemes') > - call dein#add('Haron-Prime/evening_vim') > + call dein#add('flazz/vim-colorschemes') > + call dein#add('Haron-Prime/evening_vim') > > "core plugins that change the behavior of vim and how we use it globally > ˋˋˋ > > All I did here was remove one `+` line and manually add another (which > is a variant of the second `-` line). So the line is identical (sans s/^-/+/). Interesting. > Under git 2.17.0.252.gfe0a9ea, the same piece is opened in $VISUAL for > editing (and if left unmodified applies OK), but when modified in the > to the same exact value, after exiting the editor I receive the > following error from git: > > error: patch fragment without header at line 15: @@ -25,7 +25,8 @@ I can't seem to reproduce this with some very simple testing. Are you able to share your files? Or even better, derive a minimal reproduction recipe? What happens if you do not do a "remove this line, then add it again", but instead turn that unchanged line into context? That is, you edit the hunk into something like this (but without white-space damage): ... - " call dein#add('flazz/vim-colorschemes') + call dein#add('flazz/vim-colorschemes') call dein#add('Haron-Prime/evening_vim') ... Adding Phillip to cc, since he was recently working in this area and might have an idea. Martin ``` ## Phillip Wood, 2018-04-16 10:00 Subject: Re: Regression in patch add? Message-ID: <20828345-bd21-3c45-a899-e6a97910d663@talktalk.net> URL: https://gitlist.dev/e/20828345-bd21-3c45-a899-e6a97910d663%40talktalk.net In-Reply-To: <01010162c940b8bb-d8139971-3ee2-4cd6-bb19-35126d46753b-000000@us-west-2.amazonses.com> ``` On 15/04/18 13:21, mqudsi wrote: > > Hello all, > > I'm currently running the latest version of git built from `master`, and > I'm running into what appears to be a regression in the behavior of the > piecewise `git add -p` when applying a manually edited chunk. > > I first run `git add -p`, then manually edit a chunk (after hitting `s` > once, if it matters). Thanks for mentioning that, it can matter as the code that stitches split hunks back together can't cope with edited hunks properly (though the code that checks the hunk immediately after it's been edited doesn't bother to try and stitch things back together). > The chunk originally contains the following: > > ˋˋˋdiff > # Manual hunk edit mode -- see bottom for a quick guide > @@ -20,7 +20,7 @@ > "call dein#add('Shougo/dein.vim', {'rev': 'master'}) > > " Add or remove your plugins here: > - " call dein#add('flazz/vim-colorschemes') > - call dein#add('Haron-Prime/evening_vim') > + call dein#add('flazz/vim-colorschemes') > + call dein#add('danilo-augusto/vim-afterglow') > > "core plugins that change the behavior of vim and how we use it globally > ˋˋˋ > > Under git 2.7.4, I can edit it to the following, which is accepted > without a problem: > > ˋˋˋdiff > # Manual hunk edit mode -- see bottom for a quick guide > @@ -20,7 +20,7 @@ > "call dein#add('Shougo/dein.vim', {'rev': 'master'}) > > " Add or remove your plugins here: > - " call dein#add('flazz/vim-colorschemes') > - call dein#add('Haron-Prime/evening_vim') > + call dein#add('flazz/vim-colorschemes') > + call dein#add('Haron-Prime/evening_vim') > > "core plugins that change the behavior of vim and how we use it globally > ˋˋˋ > > All I did here was remove one `+` line and manually add another (which > is a variant of the second `-` line). > > Under git 2.17.0.252.gfe0a9ea, the same piece is opened in $VISUAL for > editing (and if left unmodified applies OK), but when modified in the > to the same exact value, after exiting the editor I receive the > following error from git: > > error: patch fragment without header at line 15: @@ -25,7 +25,8 @@ I'm not quite sure what that error message is telling us, I need to spend some time understanding the code in apply.c that creates this error message. I assume that the header is coming from the next hunk which was created when you split the original hunk. If you could post the original hunk before it was split and the hunk starting at line 25 after it was split that might help. As Martin said if you could share the files or come up with a reproducible example that would really help in figuring out what is going wrong. Thanks for reporting this Phillip > I'm not sure what to make of this. > > Thank you, > > Mahmoud Al-Qudsi > NeoSmart Technologies > > ``` ## Phillip Wood, 2018-04-16 10:01 Subject: Re: Regression in patch add? Message-ID: URL: https://gitlist.dev/e/aa2599d6-6349-0c98-6a31-f68c8b8bf0f8%40talktalk.net In-Reply-To: ``` On 15/04/18 14:59, Martin Ågren wrote: > Hi Mahmoud > > On 15 April 2018 at 14:21, wrote: >> I first run `git add -p`, then manually edit a chunk (after hitting `s` >> once, if it matters). The chunk originally contains the following: > > [...] > >> Under git 2.7.4, I can edit it to the following, which is accepted >> without a problem: >> >> ˋˋˋdiff >> # Manual hunk edit mode -- see bottom for a quick guide >> @@ -20,7 +20,7 @@ >> "call dein#add('Shougo/dein.vim', {'rev': 'master'}) >> >> " Add or remove your plugins here: >> - " call dein#add('flazz/vim-colorschemes') >> - call dein#add('Haron-Prime/evening_vim') >> + call dein#add('flazz/vim-colorschemes') >> + call dein#add('Haron-Prime/evening_vim') >> >> "core plugins that change the behavior of vim and how we use it globally >> ˋˋˋ >> >> All I did here was remove one `+` line and manually add another (which >> is a variant of the second `-` line). > > So the line is identical (sans s/^-/+/). Interesting. > >> Under git 2.17.0.252.gfe0a9ea, the same piece is opened in $VISUAL for >> editing (and if left unmodified applies OK), but when modified in the >> to the same exact value, after exiting the editor I receive the >> following error from git: >> >> error: patch fragment without header at line 15: @@ -25,7 +25,8 @@ > > I can't seem to reproduce this with some very simple testing. Are you > able to share your files? Or even better, derive a minimal reproduction > recipe? > > What happens if you do not do a "remove this line, then add it again", > but instead turn that unchanged line into context? That is, you edit the > hunk into something like this (but without white-space damage): > > ... > - " call dein#add('flazz/vim-colorschemes') > + call dein#add('flazz/vim-colorschemes') > call dein#add('Haron-Prime/evening_vim') > ... That's a good idea to try > Adding Phillip to cc, since he was recently working in this area Thanks for cc-ing me > and might have an idea. I wish I did! Best Wishes Phillip > > Martin > ``` ## Oliver Joseph Ash, 2018-05-10 10:41 Subject: Re: Regression in patch add? Message-ID: <20180510104136.8653-1-oliverjash@gmail.com> URL: https://gitlist.dev/e/20180510104136.8653-1-oliverjash%40gmail.com In-Reply-To: <01010162c940b8bb-d8139971-3ee2-4cd6-bb19-35126d46753b-000000@us-west-2.amazonses.com> ``` I just ran into a similar problem: https://stackoverflow.com/questions/50258565/git-editing-hunks-fails-when-file-has-other-hunks I can reproduce on 2.17.0. The issue doesn't occur on 2.16.2, however. Is this a bug? ``` ## Martin Ågren, 2018-05-10 12:17 Subject: Re: Regression in patch add? Message-ID: URL: https://gitlist.dev/e/CAN0heSq5SyPgoEURRVHupcabVu3jX%2BtmX%2B0U-6azrJDDgfZ5Gw%40mail.gmail.com In-Reply-To: <20180510104136.8653-1-oliverjash@gmail.com> ``` On 10 May 2018 at 12:41, Oliver Joseph Ash wrote: > I just ran into a similar problem: https://stackoverflow.com/questions/50258565/git-editing-hunks-fails-when-file-has-other-hunks > > I can reproduce on 2.17.0. The issue doesn't occur on 2.16.2, however. > > Is this a bug? I would think so. Thanks for finding this thread. To keep history around, it would be nice to have your reproduction recipe on the list, not just on stackoverflow. That said, I cannot reproduce on v2.17.0 using your recipe. I suspect there is something quite interesting going on here, considering how trivial your edit is. As a shot in the dark, does your test involve unusual file systems, funny characters in filenames, ..? You are on some sort of Linux, right? The first thing to try out might be something like $ # create the initial file as before, with "bar" $ # git add, git commit ... $ # do the "change bar to bar1" everywhere $ git diff >test-patch $ git reset --hard $ # edit the *FIRST* hunk in test.patch like before (bar1 -> bar2) $ git apply --check test.patch && echo "ok..." $ git apply test.patch Does that succeed at all? $ git diff should now show bar2 in the first hunk and bar1 in the second hunk, just like your edited test.patch. If that works, it would seem that the problem is with `git add -p`, and how it is generating the patches for `git apply`. I have some ideas about how to debug from there, but ... How comfortable are you with building Git from the sources? Or with temporarily fiddling around with your Git installation? (git-add--interactive is a Perl script, so it would be possible to edit it in place to emit various debug information. That has potential for messing up royally, though.) Martin ``` ## Oliver Joseph Ash, 2018-05-10 13:15 Subject: Re: Regression in patch add? Message-ID: <20180510131502.17739-1-oliverjash@gmail.com> URL: https://gitlist.dev/e/20180510131502.17739-1-oliverjash%40gmail.com In-Reply-To: <01010162c940b8bb-d8139971-3ee2-4cd6-bb19-35126d46753b-000000@us-west-2.amazonses.com> ``` > does your test involve unusual file systems, funny characters in filenames, ..? You are on some sort of Linux, right? I'm running macOS 10.13.4. I don't have any unusual file system setup, as far as I'm aware. The filename in my test case is simply `foo`. I tried the steps you suggested: on git 2.17.0, saving the patch, editing it, and applying it, and it succeeded. > should now show bar2 in the first hunk and bar1 in the second hunk, just like your edited test.patch. That was the case, although I had to remove the `--check` flag from `git apply`. > How comfortable are you with building Git from the sources? I've never done it before, but I assume it's well documented, so I'm willing to give it a shot! Happy to try any steps to debug this! Although I'm a bit surprised no-one else can reproduce it with the same version of Git, which makes it seem less likely this could be a bug, and more likely it's something in my setup. ``` ## Oliver Joseph Ash, 2018-05-10 13:16 Subject: Re: Regression in patch add? Message-ID: <20180510131626.17859-1-oliverjash@gmail.com> URL: https://gitlist.dev/e/20180510131626.17859-1-oliverjash%40gmail.com In-Reply-To: ``` (Apologies, I accidentally sent this as a reply to the original post, instead of your email. I'm new to this!) > does your test involve unusual file systems, funny characters in filenames, ..? You are on some sort of Linux, right? I'm running macOS 10.13.4. I don't have any unusual file system setup, as far as I'm aware. The filename in my test case is simply `foo`. I tried the steps you suggested: on git 2.17.0, saving the patch, editing it, and applying it, and it succeeded. > should now show bar2 in the first hunk and bar1 in the second hunk, just like your edited test.patch. That was the case, although I had to remove the `--check` flag from `git apply`. > How comfortable are you with building Git from the sources? I've never done it before, but I assume it's well documented, so I'm willing to give it a shot! Happy to try any steps to debug this! Although I'm a bit surprised no-one else can reproduce it with the same version of Git, which makes it seem less likely this could be a bug, and more likely it's something in my setup. ``` ## Phillip Wood, 2018-05-10 13:49 Subject: Re: Regression in patch add? Message-ID: URL: https://gitlist.dev/e/be321106-2f10-e678-8237-449d2dd30fee%40talktalk.net In-Reply-To: ``` On 10/05/18 13:17, Martin Ågren wrote: > > On 10 May 2018 at 12:41, Oliver Joseph Ash wrote: >> I just ran into a similar problem: https://stackoverflow.com/questions/50258565/git-editing-hunks-fails-when-file-has-other-hunks >> >> I can reproduce on 2.17.0. The issue doesn't occur on 2.16.2, however. >> >> Is this a bug? > > I would think so. Thanks for finding this thread. To keep history > around, it would be nice to have your reproduction recipe on the list, > not just on stackoverflow. That said, I cannot reproduce on v2.17.0 > using your recipe. I suspect there is something quite interesting going > on here, considering how trivial your edit is. Thanks Oliver for posting an example that we can test, that said I can't reproduce it on Linux if the hunk is edited correctly. However if I remove the leading space from the empty line between 'baz' and 'foo' then I get the same error as you. Perhaps your editor is stripping trailing white space? If so that will lead to problems when editing diffs as the leading space is needed for apply to know that it's an empty context line. For the mailing list the hunk in question looks like @@ -1,5 +1,5 @@ foo -bar +bar1 baz foo I've tried using 'git apply --recount --cached' directly and was surprised to see that it accepts the patch with the broken context line. In 2.17.0 'add -p' no longer uses the --recount option, instead it counts the patch it's self but stops counting when it runs out of lines starting with [- +], this explains the difference from earlier versions. It seems it's not uncommon for editors to strip the space from empty context lines so maybe 'add -p' should take that into account when recounting patches. I'm about to go off line for a couple of weeks so it will probably be next month before I'm able to put a patch together (assuming Junio agrees we should support broken hunks) Best Wishes Phillip > As a shot in the dark, does your test involve unusual file systems, > funny characters in filenames, ..? You are on some sort of Linux, right? > > The first thing to try out might be something like > > $ # create the initial file as before, with "bar" > $ # git add, git commit ... > $ # do the "change bar to bar1" everywhere > $ git diff >test-patch > $ git reset --hard > $ # edit the *FIRST* hunk in test.patch like before (bar1 -> bar2) > $ git apply --check test.patch && echo "ok..." > $ git apply test.patch > > Does that succeed at all? > > $ git diff > > should now show bar2 in the first hunk and bar1 in the second hunk, > just like your edited test.patch. > > If that works, it would seem that the problem is with `git add -p`, and > how it is generating the patches for `git apply`. I have some ideas > about how to debug from there, but ... How comfortable are you with > building Git from the sources? Or with temporarily fiddling around with > your Git installation? (git-add--interactive is a Perl script, so it > would be possible to edit it in place to emit various debug > information. That has potential for messing up royally, though.) > > Martin > ``` ## Martin Ågren, 2018-05-10 13:54 Subject: Re: Regression in patch add? Message-ID: URL: https://gitlist.dev/e/CAN0heSoNP6ZjoU4x%3DtjwXxN_4oeOdrPG2LuahTPvGz0Y9WPp3w%40mail.gmail.com In-Reply-To: <20180510131626.17859-1-oliverjash@gmail.com> ``` On 10 May 2018 at 15:16, Oliver Joseph Ash wrote: > (Apologies, I accidentally sent this as a reply to the original post, instead of your email. I'm new to this!) (No worries.) ;-) >> does your test involve unusual file systems, funny characters in filenames, ..? You are on some sort of Linux, right? > > I'm running macOS 10.13.4. I don't have any unusual file system setup, as far as I'm aware. The filename in my test case is simply `foo`. I'm not too familiar with Mac, unfortunately, but let's see.. > I tried the steps you suggested: on git 2.17.0, saving the patch, editing it, and applying it, and it succeeded. > >> should now show bar2 in the first hunk and bar1 in the second hunk, just like your edited test.patch. > > That was the case, although I had to remove the `--check` flag from `git apply`. Hmm, you mean that `git apply --check test.patch` failed? With error messages? Or, you had to remove the --check flag in order for the patch to actually be applied on disk? I would guess it's the latter, but just to be clear. >> How comfortable are you with building Git from the sources? > > I've never done it before, but I assume it's well documented, so I'm willing to give it a shot! > > Happy to try any steps to debug this! Although I'm a bit surprised no-one else can reproduce it with the same version of Git, which makes it seem less likely this could be a bug, and more likely it's something in my setup. Where do the git 2.17.0 and 2.16.2 come from that you have been testing? Homebrew? Apple? (Ple So you should be able to do `git clone https://github.com/git/git.git` and read INSTALL. It might be useful to start with `git checkout v2.17.0` to make sure you're testing roughly the same thing as before. As for obtaining the dependencies, since I'm not familiar with Mac, I cannot give any good hints. I see now that Phillip has replied with a good guess. Let's hope he has managed to circle in on what's causing your problem. Martin ``` ## Oliver Joseph Ash, 2018-05-10 14:11 Subject: Re: Regression in patch add? Message-ID: <20180510141125.21677-1-oliverjash@gmail.com> URL: https://gitlist.dev/e/20180510141125.21677-1-oliverjash%40gmail.com In-Reply-To: ``` You found the problem Phillip! My editor was trimming trailing white space, which breaks the context line. I had tried to use an alternative editor to account for any editor specific behaviour, but it turns out both the editors I tested in were doing this! I suspect this change in behaviour will effect a lot of users? If so, it would be good if `git add -p` allowed for this behaviour, in the same way `git apply` does. Meanwhile, I can easily configure my editor not to do this for `*.diff` files. Thanks for your help, Phillip and Martin! Mahmoud, does this also explain your problem as per your original post? ``` ## Phillip Wood, 2018-05-10 17:58 Subject: Re: Regression in patch add? Message-ID: URL: https://gitlist.dev/e/e8aedc6b-5b3e-cfb2-be9d-971bfd9adde8%40talktalk.net In-Reply-To: <20180510141125.21677-1-oliverjash@gmail.com> ``` On 10/05/18 15:11, Oliver Joseph Ash wrote: > You found the problem Phillip! My editor was trimming trailing white space, which breaks the context line. I'm glad we found the source of the problem (and that it wasn't some obscure bug) > I had tried to use an alternative editor to account for any editor specific behaviour, but it turns out both the editors I tested in were doing this! > > I suspect this change in behaviour will effect a lot of users? If so, it would be good if `git add -p` allowed for this behaviour, in the same way `git apply` does. Yes, I think it probably makes sense to do that. Originally I didn't count empty lines as context lines in case the user accidentally added some empty lines at the end of the hunk but if 'git apply' does then I think 'git add -p' should as well > Meanwhile, I can easily configure my editor not to do this for `*.diff` files. > > Thanks for your help, Phillip and Martin! Thanks for posting an example so we could test it, it makes it much easier to track the problem down Best Wishes Phillip > Mahmoud, does this also explain your problem as per your original post? > ``` ## Junio C Hamano, 2018-05-11 02:47 Subject: Re: Regression in patch add? Message-ID: URL: https://gitlist.dev/e/xmqqzi16hpr4.fsf%40gitster-ct.c.googlers.com In-Reply-To: ``` Phillip Wood writes: > Yes, I think it probably makes sense to do that. Originally I didn't > count empty lines as context lines in case the user accidentally added > some empty lines at the end of the hunk but if 'git apply' does then I > think 'git add -p' should as well I am not sure if "adding to the tail" should be tolerated, but in any case, newer GNU diff can show an empty unaffected line as an empty line (unlike traditional unified context format in which such a line is expressed as a line with a lone SP on it), which is allowed as "implementation defined" by POSIX.1 [*1*]. Modern "git apply" knows about this. If "add -p" parses a patch, it should learn to do so, too. [Reference] *1* http://pubs.opengroup.org/onlinepubs/9699919799/utilities/diff.html > >> Meanwhile, I can easily configure my editor not to do this for `*.diff` files. >> >> Thanks for your help, Phillip and Martin! > > Thanks for posting an example so we could test it, it makes it much > easier to track the problem down > > Best Wishes > > Phillip > >> Mahmoud, does this also explain your problem as per your original post? >> ``` ## Phillip Wood, 2018-05-11 18:23 Subject: Re: Regression in patch add? Message-ID: <9a7d35c7-2889-05e4-f6f3-5706c710d327@talktalk.net> URL: https://gitlist.dev/e/9a7d35c7-2889-05e4-f6f3-5706c710d327%40talktalk.net In-Reply-To: ``` On 11/05/18 03:47, Junio C Hamano wrote: > Phillip Wood writes: > >> Yes, I think it probably makes sense to do that. Originally I didn't >> count empty lines as context lines in case the user accidentally added >> some empty lines at the end of the hunk but if 'git apply' does then I >> think 'git add -p' should as well > > I am not sure if "adding to the tail" should be tolerated, but in > any case, newer GNU diff can show an empty unaffected line as an > empty line (unlike traditional unified context format in which such > a line is expressed as a line with a lone SP on it), which is > allowed as "implementation defined" by POSIX.1 [*1*]. Modern "git > apply" knows about this. Thanks for the reference, I hadn't realized the space was optional. > If "add -p" parses a patch, it should learn to do so, too. I'm about to go off line for a while, I'll send a fix when I'm back up and running at next month (unfortunately the reroll of pw/add-p-select will have to wait until then as well) Best Wishes Phillip > > [Reference] > > *1* http://pubs.opengroup.org/onlinepubs/9699919799/utilities/diff.html > >> >>> Meanwhile, I can easily configure my editor not to do this for `*.diff` files. >>> >>> Thanks for your help, Phillip and Martin! >> >> Thanks for posting an example so we could test it, it makes it much >> easier to track the problem down >> >> Best Wishes >> >> Phillip >> >>> Mahmoud, does this also explain your problem as per your original post? >>> ``` ## Phillip Wood, 2018-06-01 17:46 Subject: [PATCH] add -p: fix counting empty context lines in edited patches Message-ID: <20180601174644.13055-1-phillip.wood@talktalk.net> URL: https://gitlist.dev/e/20180601174644.13055-1-phillip.wood%40talktalk.net In-Reply-To: <01010162c940b8bb-d8139971-3ee2-4cd6-bb19-35126d46753b-000000@us-west-2.amazonses.com> ``` From: Phillip Wood recount_edited_hunk() introduced in commit 2b8ea7f3c7 ("add -p: calculate offset delta for edited patches", 2018-03-05) required all context lines to start with a space, empty lines are not counted. This was intended to avoid any recounting problems if the user had introduced empty lines at the end when editing the patch. However this introduced a regression into 'git add -p' as it seems it is common for editors to strip the trailing whitespace from empty context lines when patches are edited thereby introducing empty lines that should be counted. 'git apply' knows how to deal with such empty lines and POSIX states that whether or not there is an space on an empty context line is implementation defined [1]. Fix the regression by counting lines consist solely of a newline as well as lines starting with a space as context lines and add a test to prevent future regressions. [1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/diff.html Reported-by: Mahmoud Al-Qudsi Reported-by: Oliver Joseph Ash Reported-by: Jeff Felchner Signed-off-by: Phillip Wood --- My apologies to everyone who was affected by this regression. git-add--interactive.perl | 2 +- t/t3701-add-interactive.sh | 43 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/git-add--interactive.perl b/git-add--interactive.perl index ab022ec073..bb6f249f03 100755 --- a/git-add--interactive.perl +++ b/git-add--interactive.perl @@ -1047,7 +1047,7 @@ sub recount_edited_hunk { $o_cnt++; } elsif ($mode eq '+') { $n_cnt++; - } elsif ($mode eq ' ') { + } elsif ($mode eq ' ' or $_ eq "\n") { $o_cnt++; $n_cnt++; } diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh index e5c66f7500..f1bb879ea4 100755 --- a/t/t3701-add-interactive.sh +++ b/t/t3701-add-interactive.sh @@ -175,6 +175,49 @@ test_expect_success 'real edit works' ' diff_cmp expected output ' +test_expect_success 'setup file' ' + test_write_lines a "" b "" c >file && + git add file && + test_write_lines a "" d "" c >file +' + +test_expect_success 'setup patch' ' + SP=" " && + NULL="" && + cat >patch <<-EOF + @@ -1,4 +1,4 @@ + a + $NULL + -b + +f + $SP + c + EOF +' + +test_expect_success 'setup expected' ' + cat >expected <<-EOF + diff --git a/file b/file + index b5dd6c9..f910ae9 100644 + --- a/file + +++ b/file + @@ -1,5 +1,5 @@ + a + $SP + -f + +d + $SP + c + EOF +' + +test_expect_success 'edit can strip spaces from empty context lines' ' + test_write_lines e n q | git add -p 2>error && + test_must_be_empty error && + git diff >output && + diff_cmp expected output +' + test_expect_success 'skip files similarly as commit -a' ' git reset && echo file >.gitignore && -- 2.17.0 ``` ## Jacob Keller, 2018-06-01 19:07 Subject: Re: [PATCH] add -p: fix counting empty context lines in edited patches Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2BxokjMcbbrH2iCt5SWMxgsstA%2BVXHOQgx4debd7Oou-RRA%40mail.gmail.com In-Reply-To: <20180601174644.13055-1-phillip.wood@talktalk.net> ``` On Fri, Jun 1, 2018 at 10:46 AM, Phillip Wood wrote: > From: Phillip Wood > > recount_edited_hunk() introduced in commit 2b8ea7f3c7 ("add -p: > calculate offset delta for edited patches", 2018-03-05) required all > context lines to start with a space, empty lines are not counted. This > was intended to avoid any recounting problems if the user had > introduced empty lines at the end when editing the patch. However this > introduced a regression into 'git add -p' as it seems it is common for > editors to strip the trailing whitespace from empty context lines when > patches are edited thereby introducing empty lines that should be > counted. 'git apply' knows how to deal with such empty lines and POSIX > states that whether or not there is an space on an empty context line > is implementation defined [1]. > > Fix the regression by counting lines consist solely of a newline as > well as lines starting with a space as context lines and add a test to > prevent future regressions. > > [1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/diff.html > > Reported-by: Mahmoud Al-Qudsi > Reported-by: Oliver Joseph Ash > Reported-by: Jeff Felchner > Signed-off-by: Phillip Wood > --- > My apologies to everyone who was affected by this regression. > Ahhh I suspect this is why my edited code in add -p was sometimes failing to apply! Thanks, Jake > git-add--interactive.perl | 2 +- > t/t3701-add-interactive.sh | 43 ++++++++++++++++++++++++++++++++++++++ > 2 files changed, 44 insertions(+), 1 deletion(-) > > diff --git a/git-add--interactive.perl b/git-add--interactive.perl > index ab022ec073..bb6f249f03 100755 > --- a/git-add--interactive.perl > +++ b/git-add--interactive.perl > @@ -1047,7 +1047,7 @@ sub recount_edited_hunk { > $o_cnt++; > } elsif ($mode eq '+') { > $n_cnt++; > - } elsif ($mode eq ' ') { > + } elsif ($mode eq ' ' or $_ eq "\n") { > $o_cnt++; > $n_cnt++; > } > diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh > index e5c66f7500..f1bb879ea4 100755 > --- a/t/t3701-add-interactive.sh > +++ b/t/t3701-add-interactive.sh > @@ -175,6 +175,49 @@ test_expect_success 'real edit works' ' > diff_cmp expected output > ' > > +test_expect_success 'setup file' ' > + test_write_lines a "" b "" c >file && > + git add file && > + test_write_lines a "" d "" c >file > +' > + > +test_expect_success 'setup patch' ' > + SP=" " && > + NULL="" && > + cat >patch <<-EOF > + @@ -1,4 +1,4 @@ > + a > + $NULL > + -b > + +f > + $SP > + c > + EOF > +' > + > +test_expect_success 'setup expected' ' > + cat >expected <<-EOF > + diff --git a/file b/file > + index b5dd6c9..f910ae9 100644 > + --- a/file > + +++ b/file > + @@ -1,5 +1,5 @@ > + a > + $SP > + -f > + +d > + $SP > + c > + EOF > +' > + > +test_expect_success 'edit can strip spaces from empty context lines' ' > + test_write_lines e n q | git add -p 2>error && > + test_must_be_empty error && > + git diff >output && > + diff_cmp expected output > +' > + > test_expect_success 'skip files similarly as commit -a' ' > git reset && > echo file >.gitignore && > -- > 2.17.0 > ``` ## Eric Sunshine, 2018-06-01 20:03 Subject: Re: [PATCH] add -p: fix counting empty context lines in edited patches Message-ID: URL: https://gitlist.dev/e/CAPig%2BcSSj2ETXfk8FYUc%2B%3DtE6bfoRuqF5Ld4kOgE4%2BDDpfL%2BBA%40mail.gmail.com In-Reply-To: <20180601174644.13055-1-phillip.wood@talktalk.net> ``` On Fri, Jun 1, 2018 at 1:46 PM, Phillip Wood wrote: > recount_edited_hunk() introduced in commit 2b8ea7f3c7 ("add -p: > calculate offset delta for edited patches", 2018-03-05) required all > context lines to start with a space, empty lines are not counted. This > was intended to avoid any recounting problems if the user had > introduced empty lines at the end when editing the patch. However this > introduced a regression into 'git add -p' as it seems it is common for > editors to strip the trailing whitespace from empty context lines when > patches are edited thereby introducing empty lines that should be > counted. 'git apply' knows how to deal with such empty lines and POSIX > states that whether or not there is an space on an empty context line > is implementation defined [1]. > > Fix the regression by counting lines consist solely of a newline as s/consist/&ing/ --or-- s/consist/that &/ > well as lines starting with a space as context lines and add a test to > prevent future regressions. > > Signed-off-by: Phillip Wood > --- > git-add--interactive.perl | 2 +- > diff --git a/git-add--interactive.perl b/git-add--interactive.perl > @@ -1047,7 +1047,7 @@ sub recount_edited_hunk { > - } elsif ($mode eq ' ') { > + } elsif ($mode eq ' ' or $_ eq "\n") { Based upon a very cursory read of parts of git-add-interactive.perl, do I understand correctly that we don't have to worry about $_ ever being "\r\n" on Windows? > diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh > @@ -175,6 +175,49 @@ test_expect_success 'real edit works' ' > +test_expect_success 'setup file' ' > + test_write_lines a "" b "" c >file && > + git add file && > + test_write_lines a "" d "" c >file > +' > + > +test_expect_success 'setup patch' ' > + SP=" " && > + NULL="" && > + cat >patch <<-EOF > + [...] > + EOF > +' > + > +test_expect_success 'setup expected' ' > + cat >expected <<-EOF > + [...] > + EOF > +' > + > +test_expect_success 'edit can strip spaces from empty context lines' ' > + test_write_lines e n q | git add -p 2>error && > + test_must_be_empty error && > + git diff >output && > + diff_cmp expected output > +' I would have expected all the setup work to be contained directly in the sole test which needs it rather than spread over three tests (two of which are composed of a single command). Not a big deal, and not worth a re-roll. ``` ## Phillip Wood, 2018-06-04 10:08 Subject: Re: [PATCH] add -p: fix counting empty context lines in edited patches Message-ID: <36f2d9e0-ba79-64d3-ffb5-d0772cafa153@talktalk.net> URL: https://gitlist.dev/e/36f2d9e0-ba79-64d3-ffb5-d0772cafa153%40talktalk.net In-Reply-To: ``` On 01/06/18 21:03, Eric Sunshine wrote: > On Fri, Jun 1, 2018 at 1:46 PM, Phillip Wood wrote: >> recount_edited_hunk() introduced in commit 2b8ea7f3c7 ("add -p: >> calculate offset delta for edited patches", 2018-03-05) required all >> context lines to start with a space, empty lines are not counted. This >> was intended to avoid any recounting problems if the user had >> introduced empty lines at the end when editing the patch. However this >> introduced a regression into 'git add -p' as it seems it is common for >> editors to strip the trailing whitespace from empty context lines when >> patches are edited thereby introducing empty lines that should be >> counted. 'git apply' knows how to deal with such empty lines and POSIX >> states that whether or not there is an space on an empty context line >> is implementation defined [1]. >> >> Fix the regression by counting lines consist solely of a newline as > > s/consist/&ing/ > --or-- > s/consist/that &/ Thanks, I'd intended to say 'that consist' >> well as lines starting with a space as context lines and add a test to >> prevent future regressions. >> >> Signed-off-by: Phillip Wood >> --- >> git-add--interactive.perl | 2 +- >> diff --git a/git-add--interactive.perl b/git-add--interactive.perl >> @@ -1047,7 +1047,7 @@ sub recount_edited_hunk { >> - } elsif ($mode eq ' ') { >> + } elsif ($mode eq ' ' or $_ eq "\n") { > > Based upon a very cursory read of parts of git-add-interactive.perl, > do I understand correctly that we don't have to worry about $_ ever > being "\r\n" on Windows? > Good question, I think the short answer no. If my understanding of the newline section of perlport [1] is correct then on Windows "\n" eq "\012" and the io layer replaces "\015\012" with "\n" when reading in 'text' mode (which I think is the default if you don't specify one when opening the file/process or with binmode()). As "\n" is only one character it would perhaps be better to test '$mode' rather than '$_' above - what do you think. [1] http://perldoc.perl.org/perlport.html#Newlines >> diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh >> @@ -175,6 +175,49 @@ test_expect_success 'real edit works' ' >> +test_expect_success 'setup file' ' >> + test_write_lines a "" b "" c >file && >> + git add file && >> + test_write_lines a "" d "" c >file >> +' >> + >> +test_expect_success 'setup patch' ' >> + SP=" " && >> + NULL="" && >> + cat >patch <<-EOF >> + [...] >> + EOF >> +' >> + >> +test_expect_success 'setup expected' ' >> + cat >expected <<-EOF >> + [...] >> + EOF >> +' >> + >> +test_expect_success 'edit can strip spaces from empty context lines' ' >> + test_write_lines e n q | git add -p 2>error && >> + test_must_be_empty error && >> + git diff >output && >> + diff_cmp expected output >> +' > > I would have expected all the setup work to be contained directly in > the sole test which needs it rather than spread over three tests (two > of which are composed of a single command). Not a big deal, and not > worth a re-roll. Good point I was torn between that and matching the existing style in that file seems to be to create a million ancillary tests to do the set-up. Thanks Phillip ``` ## Eric Sunshine, 2018-06-04 17:21 Subject: Re: [PATCH] add -p: fix counting empty context lines in edited patches Message-ID: URL: https://gitlist.dev/e/CAPig%2BcSc49B4Mn7m9XOp%2BQF3qcT06ncGGXJpssFJMNjCQ8e7Mw%40mail.gmail.com In-Reply-To: <36f2d9e0-ba79-64d3-ffb5-d0772cafa153@talktalk.net> ``` On Mon, Jun 4, 2018 at 6:08 AM, Phillip Wood wrote: > On 01/06/18 21:03, Eric Sunshine wrote: >> On Fri, Jun 1, 2018 at 1:46 PM, Phillip Wood wrote: >>> + } elsif ($mode eq ' ' or $_ eq "\n") { >> >> Based upon a very cursory read of parts of git-add-interactive.perl, >> do I understand correctly that we don't have to worry about $_ ever >> being "\r\n" on Windows? > > Good question, I think the short answer no. If my understanding of the > newline section of perlport [1] is correct then on Windows "\n" eq > "\012" and the io layer replaces "\015\012" with "\n" when reading in > 'text' mode (which I think is the default if you don't specify one when > opening the file/process or with binmode()). That was my interpretation, as well (though I didn't audit the code closely). > As "\n" is only one > character it would perhaps be better to test '$mode' rather than '$_' > above - what do you think. That could be clearer. As a reviewer, I had to spend extra brain cycles wondering why $mode was used everywhere else but $_ in just this one place. (Not that it was difficult to figure out.) >>> diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh >>> @@ -175,6 +175,49 @@ test_expect_success 'real edit works' ' >>> +test_expect_success 'setup file' ' >>> + [...] >>> +' >>> +test_expect_success 'setup patch' ' >>> + [...] >>> +' >>> +test_expect_success 'setup expected' ' >>> + [...] >>> +' >>> +test_expect_success 'edit can strip spaces from empty context lines' ' >>> + test_write_lines e n q | git add -p 2>error && >>> + test_must_be_empty error && >>> + git diff >output && >>> + diff_cmp expected output >>> +' >> >> I would have expected all the setup work to be contained directly in >> the sole test which needs it rather than spread over three tests (two >> of which are composed of a single command). Not a big deal, and not >> worth a re-roll. > > Good point I was torn between that and matching the existing style in > that file seems to be to create a million ancillary tests to do the set-up. I see what you mean. Following existing practice in the file makes sense, though breaking from that practice by bundling all the setup into the single test which uses it wouldn't hurt either. It's a judgment call (and not worrying about too much). ``` ## Phillip Wood, 2018-06-11 09:46 Subject: [PATCH v2] add -p: fix counting empty context lines in edited patches Message-ID: <20180611094602.17469-1-phillip.wood@talktalk.net> URL: https://gitlist.dev/e/20180611094602.17469-1-phillip.wood%40talktalk.net In-Reply-To: <20180601174644.13055-1-phillip.wood@talktalk.net> ``` From: Phillip Wood recount_edited_hunk() introduced in commit 2b8ea7f3c7 ("add -p: calculate offset delta for edited patches", 2018-03-05) required all context lines to start with a space, empty lines are not counted. This was intended to avoid any recounting problems if the user had introduced empty lines at the end when editing the patch. However this introduced a regression into 'git add -p' as it seems it is common for editors to strip the trailing whitespace from empty context lines when patches are edited thereby introducing empty lines that should be counted. 'git apply' knows how to deal with such empty lines and POSIX states that whether or not there is an space on an empty context line is implementation defined [1]. Fix the regression by counting lines that consist solely of a newline as well as lines starting with a space as context lines and add a test to prevent future regressions. [1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/diff.html Reported-by: Mahmoud Al-Qudsi Reported-by: Oliver Joseph Ash Reported-by: Jeff Felchner Signed-off-by: Phillip Wood --- Thanks for the feedback, the only changes since v1 are to fix the commit message to match what was in pu and to change '$_' to '$mode' in the comparison as I think that is clearer. In the end I decided to leave the tests as they are. git-add--interactive.perl | 2 +- t/t3701-add-interactive.sh | 43 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/git-add--interactive.perl b/git-add--interactive.perl index ab022ec073..8361ef45e7 100755 --- a/git-add--interactive.perl +++ b/git-add--interactive.perl @@ -1047,7 +1047,7 @@ sub recount_edited_hunk { $o_cnt++; } elsif ($mode eq '+') { $n_cnt++; - } elsif ($mode eq ' ') { + } elsif ($mode eq ' ' or $mode eq "\n") { $o_cnt++; $n_cnt++; } diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh index e5c66f7500..f1bb879ea4 100755 --- a/t/t3701-add-interactive.sh +++ b/t/t3701-add-interactive.sh @@ -175,6 +175,49 @@ test_expect_success 'real edit works' ' diff_cmp expected output ' +test_expect_success 'setup file' ' + test_write_lines a "" b "" c >file && + git add file && + test_write_lines a "" d "" c >file +' + +test_expect_success 'setup patch' ' + SP=" " && + NULL="" && + cat >patch <<-EOF + @@ -1,4 +1,4 @@ + a + $NULL + -b + +f + $SP + c + EOF +' + +test_expect_success 'setup expected' ' + cat >expected <<-EOF + diff --git a/file b/file + index b5dd6c9..f910ae9 100644 + --- a/file + +++ b/file + @@ -1,5 +1,5 @@ + a + $SP + -f + +d + $SP + c + EOF +' + +test_expect_success 'edit can strip spaces from empty context lines' ' + test_write_lines e n q | git add -p 2>error && + test_must_be_empty error && + git diff >output && + diff_cmp expected output +' + test_expect_success 'skip files similarly as commit -a' ' git reset && echo file >.gitignore && -- 2.17.0 ``` ## Jeff Felchner, 2018-07-11 20:27 Subject: Re: [PATCH v2] add -p: fix counting empty context lines in edited patches Message-ID: URL: https://gitlist.dev/e/C9B989D9-5148-4AF1-80EB-ADFAE0DB8FF8%40gmail.com In-Reply-To: <20180611094602.17469-1-phillip.wood@talktalk.net> ``` Hey all, I assumed this was going to be in 2.18, but I'm still having the same issue. What's the plan for release of this? > On 2018 Jun 11, at 4:46, Phillip Wood wrote: > > From: Phillip Wood > > recount_edited_hunk() introduced in commit 2b8ea7f3c7 ("add -p: > calculate offset delta for edited patches", 2018-03-05) required all > context lines to start with a space, empty lines are not counted. This > was intended to avoid any recounting problems if the user had > introduced empty lines at the end when editing the patch. However this > introduced a regression into 'git add -p' as it seems it is common for > editors to strip the trailing whitespace from empty context lines when > patches are edited thereby introducing empty lines that should be > counted. 'git apply' knows how to deal with such empty lines and POSIX > states that whether or not there is an space on an empty context line > is implementation defined [1]. > > Fix the regression by counting lines that consist solely of a newline > as well as lines starting with a space as context lines and add a test > to prevent future regressions. > > [1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/diff.html > > Reported-by: Mahmoud Al-Qudsi > Reported-by: Oliver Joseph Ash > Reported-by: Jeff Felchner > Signed-off-by: Phillip Wood > --- > > Thanks for the feedback, the only changes since v1 are to fix the > commit message to match what was in pu and to change '$_' to '$mode' > in the comparison as I think that is clearer. In the end I decided to > leave the tests as they are. > > git-add--interactive.perl | 2 +- > t/t3701-add-interactive.sh | 43 ++++++++++++++++++++++++++++++++++++++ > 2 files changed, 44 insertions(+), 1 deletion(-) > > diff --git a/git-add--interactive.perl b/git-add--interactive.perl > index ab022ec073..8361ef45e7 100755 > --- a/git-add--interactive.perl > +++ b/git-add--interactive.perl > @@ -1047,7 +1047,7 @@ sub recount_edited_hunk { > $o_cnt++; > } elsif ($mode eq '+') { > $n_cnt++; > - } elsif ($mode eq ' ') { > + } elsif ($mode eq ' ' or $mode eq "\n") { > $o_cnt++; > $n_cnt++; > } > diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh > index e5c66f7500..f1bb879ea4 100755 > --- a/t/t3701-add-interactive.sh > +++ b/t/t3701-add-interactive.sh > @@ -175,6 +175,49 @@ test_expect_success 'real edit works' ' > diff_cmp expected output > ' > > +test_expect_success 'setup file' ' > + test_write_lines a "" b "" c >file && > + git add file && > + test_write_lines a "" d "" c >file > +' > + > +test_expect_success 'setup patch' ' > + SP=" " && > + NULL="" && > + cat >patch <<-EOF > + @@ -1,4 +1,4 @@ > + a > + $NULL > + -b > + +f > + $SP > + c > + EOF > +' > + > +test_expect_success 'setup expected' ' > + cat >expected <<-EOF > + diff --git a/file b/file > + index b5dd6c9..f910ae9 100644 > + --- a/file > + +++ b/file > + @@ -1,5 +1,5 @@ > + a > + $SP > + -f > + +d > + $SP > + c > + EOF > +' > + > +test_expect_success 'edit can strip spaces from empty context lines' ' > + test_write_lines e n q | git add -p 2>error && > + test_must_be_empty error && > + git diff >output && > + diff_cmp expected output > +' > + > test_expect_success 'skip files similarly as commit -a' ' > git reset && > echo file >.gitignore && > -- > 2.17.0 > ``` ## Junio C Hamano, 2018-07-11 20:50 Subject: Re: [PATCH v2] add -p: fix counting empty context lines in edited patches Message-ID: URL: https://gitlist.dev/e/xmqqy3eh1oqa.fsf%40gitster-ct.c.googlers.com In-Reply-To: ``` Jeff Felchner writes: > Hey all, I assumed this was going to be in 2.18, but I'm still having the same issue. What's the plan for release of this? You assumed wrong ;-) A patch written on June 11th that is already deep into pre-release freeze, unless it is about fixing a regression during the same cycle, would never be in the release tagged on 21st. It is already a part of the 'master' branch after v2.18, so v2.19 would be the first feature release that would see it (unless we discover problems in that change and need to revert it, that is). ```