{"thread":{"id":"35317","subject":"git rm / format-patch / am fails on my file: patch does not apply","startedAt":"2013-11-11T02:39:10Z","lastAt":"2013-11-14T07:26:35Z","messageCount":10,"participants":["Ken Tanzer","Junio C Hamano","Stefan Beller","Philip Oakley"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"230376","messageId":"CAD3a31XPKsnuNE+szw7xgvaDrcxhKZ2jTDHKzTwdwtnCwFb_0w@mail.gmail.com","threadId":"35317","inReplyTo":null,"subject":"git rm / format-patch / am fails on my file: patch does not apply","fromName":"Ken Tanzer","fromEmail":"ken.tanzer@gmail.com","sentAt":"2013-11-11T02:39:10Z","receivedAt":"2013-11-11T02:39:10Z","isPatch":false,"sender":{"key":"ken.tanzer@gmail.com","avatar":null},"body":"I originally posted about this to the git-users list\n(https://groups.google.com/forum/#!topic/git-users/G5D0bldEbTo). If\nthis would normally be expected to work:\n\ngit add \ngit commit\ngit rm \ngit commit\ngit format-patch HEAD~1\ngit reset --hard HEAD~1\ngit am 0001*patch\n\nthen I'm having a problem with at least one (maybe more) file I have\nin a repository that seems like a bug. Even if I copy it to a new\nrepository, it will still fail, so it seems something about how git\nhandles the file, which \"file\" reports as:\n\nASCII text, with very long lines, with CRLF, CR, LF line terminators\n\nI've confirmed with the following test case on three machines, so it\nseems reproducible:\n\nmkdir temp_test_case\ncd temp_test_case\ngit init\n# my file. Sorry--couldn't find a saner link!\nwget -O jquery-ui-1.8.custom.min.js\nhttp://sourceforge.net/p/agency/code/ci/9358ea4dbe8e1540ec0b8bebfc7770f1bf8be0ec/tree/jquery-ui-1.8.custom.min.js?format=raw\ngit add jquery-ui-1.8.custom.min.js\ngit commit -m 'Adding jquery-ui'\ngit rm jquery-ui-1.8.custom.min.js\ngit commit -m 'Removing jquery-ui'\ngit format-patch HEAD~1\ngit reset --hard HEAD~1\ngit am 0001*\n\nThe last command reports the following:\n\nApplying: Removing jquery-ui\nerror: patch failed: jquery-ui-1.8.custom.min.js:1\nerror: jquery-ui-1.8.custom.min.js: patch does not apply\nPatch failed at 0001 Removing jquery-ui\nThe copy of the patch that failed is found in:\n /home/user/newtest2/temp_test_case/.git/rebase-apply/patch\nWhen you have resolved this problem, run \"git am --resolved\".\nIf you prefer to skip this patch, run \"git am --skip\" instead.\nTo restore the original branch and stop patching, run \"git am --abort\".\n\nLet me know if I can provide more information, or am missing something\nobvious! Thanks.\n\nKen\n"},{"id":"230413","messageId":"xmqqhabilpzx.fsf@gitster.dls.corp.google.com","threadId":"35317","inReplyTo":"CAD3a31XPKsnuNE+szw7xgvaDrcxhKZ2jTDHKzTwdwtnCwFb_0w@mail.gmail.com","subject":"Re: git rm / format-patch / am fails on my file: patch does not apply","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-11-11T19:04:02Z","receivedAt":"2013-11-11T19:04:02Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ken Tanzer writes:\n\n> ASCII text, with very long lines, with CRLF, CR, LF line terminators\n\nI am not very much surprised if such a file misbehaves, because the\n\"format-patch | am\" pipeline is designed to be used on patches that\ncan be transferred in plain-text e-mail safely. Long lines should\nprobably be OK, but mixed CRLF, CR and LF may be problematic.\n\nHaving said that...\n\n> I've confirmed with the following test case on three machines, so it\n> seems reproducible:\n>\n> mkdir temp_test_case\n> cd temp_test_case\n> git init\n> # my file. Sorry--couldn't find a saner link!\n> wget -O jquery-ui-1.8.custom.min.js\n> http://sourceforge.net/p/agency/code/ci/9358ea4dbe8e1540ec0b8bebfc7770f1bf8be0ec/tree/jquery-ui-1.8.custom.min.js?format=raw\n> git add jquery-ui-1.8.custom.min.js\n> git commit -m 'Adding jquery-ui'\n> git rm jquery-ui-1.8.custom.min.js\n> git commit -m 'Removing jquery-ui'\n> git format-patch HEAD~1\n> git reset --hard HEAD~1\n> git am 0001*\n\n... this does not break at all for me.\n"},{"id":"230416","messageId":"52812CC0.5050805@googlemail.com","threadId":"35317","inReplyTo":"xmqqhabilpzx.fsf@gitster.dls.corp.google.com","subject":"Re: git rm / format-patch / am fails on my file: patch does not apply","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-11-11T19:15:12Z","receivedAt":"2013-11-11T19:15:12Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On 11.11.2013 20:04, Junio C Hamano wrote:\n> Ken Tanzer writes:\n> \n>> ASCII text, with very long lines, with CRLF, CR, LF line terminators\n> \n> I am not very much surprised if such a file misbehaves, because the\n> \"format-patch | am\" pipeline is designed to be used on patches that\n> can be transferred in plain-text e-mail safely. Long lines should\n> probably be OK, but mixed CRLF, CR and LF may be problematic.\n> \n> Having said that...\n> \n>> I've confirmed with the following test case on three machines, so it\n>> seems reproducible:\n>>\n>> mkdir temp_test_case\n>> cd temp_test_case\n>> git init\n>> # my file. Sorry--couldn't find a saner link!\n>> wget -O jquery-ui-1.8.custom.min.js\n>> http://sourceforge.net/p/agency/code/ci/9358ea4dbe8e1540ec0b8bebfc7770f1bf8be0ec/tree/jquery-ui-1.8.custom.min.js?format=raw\n>> git add jquery-ui-1.8.custom.min.js\n>> git commit -m 'Adding jquery-ui'\n>> git rm jquery-ui-1.8.custom.min.js\n>> git commit -m 'Removing jquery-ui'\n>> git format-patch HEAD~1\n>> git reset --hard HEAD~1\n>> git am 0001*\n> \n> ... this does not break at all for me.\n> --\n\nHere it breaks, though it doesn't break when using\n\t git am --ignore-whitespace 0001*\n\nI do have this global config\n\tcore.safecrlf=warn\nregarding line endings.\n\nI was using 1.8.5.rc1.17.g0ecd94d\n\nTrying to understand the problem,\nStefan\n\n"},{"id":"230421","messageId":"xmqqd2m6logx.fsf@gitster.dls.corp.google.com","threadId":"35317","inReplyTo":"52812CC0.5050805@googlemail.com","subject":"Re: git rm / format-patch / am fails on my file: patch does not apply","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-11-11T19:37:02Z","receivedAt":"2013-11-11T19:37:02Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> I do have this global config\n> \tcore.safecrlf=warn\n> regarding line endings.\n\nOh, that sounds very suspicious. If the payload has CRLF, CR and LF\nmixed, that would immediately violate safecrlf, so failing the\napplication sounds like the right thing to do.\n\n> I was using 1.8.5.rc1.17.g0ecd94d\n>\n> Trying to understand the problem,\n\nLikewise. Thanks for chiming in.\n"},{"id":"230422","messageId":"5281336F.7070103@googlemail.com","threadId":"35317","inReplyTo":"xmqqd2m6logx.fsf@gitster.dls.corp.google.com","subject":"Re: git rm / format-patch / am fails on my file: patch does not apply","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-11-11T19:43:43Z","receivedAt":"2013-11-11T19:43:43Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On 11.11.2013 20:37, Junio C Hamano wrote:\n> Stefan Beller writes:\n> \n>> I do have this global config\n>> \tcore.safecrlf=warn\n>> regarding line endings.\n> \n> Oh, that sounds very suspicious. If the payload has CRLF, CR and LF\n> mixed, that would immediately violate safecrlf, so failing the\n> application sounds like the right thing to do.\n\nNot having read the man page, but copied this global config\nfrom some '1000 git tips in 5 minutes' years ago,\nI do expect a setting set to \"warn\" to actually not change the\nprogram flow except some inserted prints with warnings.\n\n> \n>> I was using 1.8.5.rc1.17.g0ecd94d\n>>\n>> Trying to understand the problem,\n> \n> Likewise. Thanks for chiming in.\n> \n\nLooking at the file we have a mixture of LF/CR, LF only and CR only\nending the lines. The formatted patch does have the same file endings\non the respective line of that file here.\n\nI was trying to change just one line ending to 2 lines (LF CR -> LF LF)\nwith a hex editor, so there it becomes a smaller (and more debugable )\npatch. I also tried LF -> CR and CR -> LF, but none of these small\nchanges seem to work.\n\nStefan\n\n"},{"id":"230425","messageId":"5281408E.10601@googlemail.com","threadId":"35317","inReplyTo":"5281336F.7070103@googlemail.com","subject":"Re: git rm / format-patch / am fails on my file: patch does not apply","fromName":"Stefan Beller","fromEmail":"stefanbeller@googlemail.com","sentAt":"2013-11-11T20:39:42Z","receivedAt":"2013-11-11T20:39:42Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On 11.11.2013 20:43, Stefan Beller wrote:\n> On 11.11.2013 20:37, Junio C Hamano wrote:\n>> Stefan Beller writes:\n>>\n>>> I do have this global config\n>>> \tcore.safecrlf=warn\n>>> regarding line endings.\n>>\n>> Oh, that sounds very suspicious. If the payload has CRLF, CR and LF\n>> mixed, that would immediately violate safecrlf, so failing the\n>> application sounds like the right thing to do.\n> \n> Not having read the man page, but copied this global config\n> from some '1000 git tips in 5 minutes' years ago,\n> I do expect a setting set to \"warn\" to actually not change the\n> program flow except some inserted prints with warnings.\n> \n>>\n>>> I was using 1.8.5.rc1.17.g0ecd94d\n>>>\n>>> Trying to understand the problem,\n>>\n>> Likewise. Thanks for chiming in.\n>>\n> \n> Looking at the file we have a mixture of LF/CR, LF only and CR only\n> ending the lines. The formatted patch does have the same file endings\n> on the respective line of that file here.\n> \n> I was trying to change just one line ending to 2 lines (LF CR -> LF LF)\n> with a hex editor, so there it becomes a smaller (and more debugable )\n> patch. I also tried LF -> CR and CR -> LF, but none of these small\n> changes seem to work.\n> \n> Stefan\n> \n\nJust having looked at the CRLF conversion code of git,\nand then at the file again. This file seems to be a special nasty kind,\nas the number of LFs is equal to the number of CRs in the file\n(373 of 0x0a and 0x0d each), but only 343 occurences of LF strictly\nfollowing a CR, so there are 30 CRs and 30 LFs.\n\nThis shouldn't be as problematic as my first thought was, there is\nnever a direct comparison of stats.cr to stats.lf in convert.c.\nThe CR and LF count are only compared to either the crlf count or 0.\n\nStefan\n\n\n\n\n\n"},{"id":"230545","messageId":"CAD3a31UVkNaPTCWCAbv0NwCOTE5_2A+P7-e28VRnk9Aopa6hcQ@mail.gmail.com","threadId":"35317","inReplyTo":"xmqqhabilpzx.fsf@gitster.dls.corp.google.com","subject":"Re: git rm / format-patch / am fails on my file: patch does not apply","fromName":"Ken Tanzer","fromEmail":"ken.tanzer@gmail.com","sentAt":"2013-11-13T00:26:49Z","receivedAt":"2013-11-13T00:26:49Z","isPatch":false,"sender":{"key":"ken.tanzer@gmail.com","avatar":null},"body":"> I am not very much surprised if such a file misbehaves, because the\n> \"format-patch | am\" pipeline is designed to be used on patches that\n> can be transferred in plain-text e-mail safely. Long lines should\n> probably be OK, but mixed CRLF, CR and LF may be problematic.\n\nI'm not sure I understand this comment. format-patch seems to work\nfine on binary files. So if it can handle any random collection of\nbytes, why not text files with (admittedly funky) CRs and LFs?\n\nCheers,\nKen\n\nOn Mon, Nov 11, 2013 at 11:04 AM, Junio C Hamano wrote:\n> Ken Tanzer writes:\n>\n>> ASCII text, with very long lines, with CRLF, CR, LF line terminators\n>\n> I am not very much surprised if such a file misbehaves, because the\n> \"format-patch | am\" pipeline is designed to be used on patches that\n> can be transferred in plain-text e-mail safely. Long lines should\n> probably be OK, but mixed CRLF, CR and LF may be problematic.\n>\n> Having said that...\n>\n>> I've confirmed with the following test case on three machines, so it\n>> seems reproducible:\n>>\n>> mkdir temp_test_case\n>> cd temp_test_case\n>> git init\n>> # my file. Sorry--couldn't find a saner link!\n>> wget -O jquery-ui-1.8.custom.min.js\n>> http://sourceforge.net/p/agency/code/ci/9358ea4dbe8e1540ec0b8bebfc7770f1bf8be0ec/tree/jquery-ui-1.8.custom.min.js?format=raw\n>> git add jquery-ui-1.8.custom.min.js\n>> git commit -m 'Adding jquery-ui'\n>> git rm jquery-ui-1.8.custom.min.js\n>> git commit -m 'Removing jquery-ui'\n>> git format-patch HEAD~1\n>> git reset --hard HEAD~1\n>> git am 0001*\n>\n> ... this does not break at all for me.\n\n\n\n-- \nAGENCY Software\nA data system that puts you in control\n100% Free Software\nhttp://agency-software.org/\nken.tanzer@agency-software.org\n(253) 245-3801\n\nSubscribe to the mailing list to\nlearn more about AGENCY or\nfollow the discussion.\n"},{"id":"230568","messageId":"xmqqbo1ofd1s.fsf@gitster.dls.corp.google.com","threadId":"35317","inReplyTo":"CAD3a31UVkNaPTCWCAbv0NwCOTE5_2A+P7-e28VRnk9Aopa6hcQ@mail.gmail.com","subject":"Re: git rm / format-patch / am fails on my file: patch does not apply","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-11-13T17:04:47Z","receivedAt":"2013-11-13T17:04:47Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ken Tanzer writes:\n\n>> I am not very much surprised if such a file misbehaves, because the\n>> \"format-patch | am\" pipeline is designed to be used on patches that\n>> can be transferred in plain-text e-mail safely. Long lines should\n>> probably be OK, but mixed CRLF, CR and LF may be problematic.\n>\n> I'm not sure I understand this comment. format-patch seems to work\n> fine on binary files.\n\nYes.\n\nThe problematic is when you tell it to process text files, taking\ninto account various common text breakages may be introduced to the\npayload, and line-end conversion is among the operations that the\nuser may be telling it to munge the perfectly-fine input, in an\nattempt to compensate.\n"},{"id":"230580","messageId":"5ABC4D34F41F48E39205914BC8D3DC35@PhilipOakley","threadId":"35317","inReplyTo":"xmqqbo1ofd1s.fsf@gitster.dls.corp.google.com","subject":"Re: git rm / format-patch / am fails on my file: patch does not apply","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":null,"receivedAt":"2013-11-13T23:07:54Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Junio C Hamano\" \nTo: \"Ken Tanzer\" \nSent: Wednesday, November 13, 2013 5:04 PM\n> Ken Tanzer writes:\n>\n>>> I am not very much surprised if such a file misbehaves, because the\n>>> \"format-patch | am\" pipeline is designed to be used on patches that\n>>> can be transferred in plain-text e-mail safely. Long lines should\n>>> probably be OK, but mixed CRLF, CR and LF may be problematic.\n>>\n>> I'm not sure I understand this comment. format-patch seems to work\n>> fine on binary files.\n>\n> Yes.\n>\n> The problematic is when you tell it to process text files, taking\n> into account various common text breakages may be introduced to the\n> payload, and line-end conversion is among the operations that the\n> user may be telling it to munge the perfectly-fine input, in an\n> attempt to compensate.\n>\n\n>> Stefan Beller writes:\n>>\n>>> I do have this global config\n>>> core.safecrlf=warn\n>>> regarding line endings.\n\nKen,\n\nDo you have one of these line ending conversion settings active in your \nconfig file?\n\nIt sounds like the 'git am' script may be silencing the warnings too \naggressively if that is the case.\n\nPhilip \n"},{"id":"230587","messageId":"CAD3a31X_o3cXr10ARiZjMFvG-yzj9hUUpmoLnjDXTjM4nHiD=w@mail.gmail.com","threadId":"35317","inReplyTo":"5ABC4D34F41F48E39205914BC8D3DC35@PhilipOakley","subject":"Re: git rm / format-patch / am fails on my file: patch does not apply","fromName":"Ken Tanzer","fromEmail":"ken.tanzer@gmail.com","sentAt":"2013-11-14T07:26:35Z","receivedAt":"2013-11-14T07:26:35Z","isPatch":false,"sender":{"key":"ken.tanzer@gmail.com","avatar":null},"body":"Hi Philip. I don't have any setting like that in either my\n~/.gitconfig, or in the .git/config files. I really haven't tweaked\nmy git config at all.\n\nThis is a typical .git/config file:\n\n[core]\nrepositoryformatversion = 0\nfilemode = true\nbare = false\nlogallrefupdates = true\n[remote \"origin\"]\nurl = ssh://ktanzer@git.code.sf.net/p/agency/code\nfetch = +refs/heads/*:refs/remotes/origin/*\n[branch \"master\"]\nremote = origin\nmerge = refs/heads/master\n\nMy ~/.gitconfig just has name and email settings in it.\n\nKen\n\n\nOn Wed, Nov 13, 2013 at 3:12 PM, Philip Oakley wrote:\n> From: \"Junio C Hamano\" \n> To: \"Ken Tanzer\" \n> Sent: Wednesday, November 13, 2013 5:04 PM\n>\n>> Ken Tanzer writes:\n>>\n>>>> I am not very much surprised if such a file misbehaves, because the\n>>>> \"format-patch | am\" pipeline is designed to be used on patches that\n>>>> can be transferred in plain-text e-mail safely. Long lines should\n>>>> probably be OK, but mixed CRLF, CR and LF may be problematic.\n>>>\n>>>\n>>> I'm not sure I understand this comment. format-patch seems to work\n>>> fine on binary files.\n>>\n>>\n>> Yes.\n>>\n>> The problematic is when you tell it to process text files, taking\n>> into account various common text breakages may be introduced to the\n>> payload, and line-end conversion is among the operations that the\n>> user may be telling it to munge the perfectly-fine input, in an\n>> attempt to compensate.\n>>\n>\n>>> Stefan Beller writes:\n>>>\n>>>> I do have this global config\n>>>> core.safecrlf=warn\n>>>> regarding line endings.\n>\n>\n> Ken,\n>\n> Do you have one of these line ending conversion settings active in your\n> config file?\n>\n> It sounds like the 'git am' script may be silencing the warnings too\n> aggressively if that is the case.\n>\n> Philip\n\n\n\n-- \nAGENCY Software\nA data system that puts you in control\n100% Free Software\nhttp://agency-software.org/\nken.tanzer@agency-software.org\n(253) 245-3801\n\nSubscribe to the mailing list to\nlearn more about AGENCY or\nfollow the discussion.\n"}]}