{"thread":{"id":"36061","subject":"New directory lost by git am","startedAt":"2014-03-05T02:49:07Z","lastAt":"2014-03-05T19:10:46Z","messageCount":10,"participants":["Phillip Susi","Chris Packham","Jeff King","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"236064","messageId":"531690A3.3040509@ubuntu.com","threadId":"36061","inReplyTo":null,"subject":"New directory lost by git am","fromName":"Phillip Susi","fromEmail":"psusi@ubuntu.com","sentAt":"2014-03-05T02:49:07Z","receivedAt":"2014-03-05T02:49:07Z","isPatch":false,"sender":{"key":"psusi@ubuntu.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA512\n\nI applied a patch with git am that adds a new source file to a new\ndirectory, and later noticed that file was missing from the commit.\nIt seems that git am fails to add the new file/directory to the index.\n\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.14 (GNU/Linux)\nComment: Using GnuPG with Thunderbird - http://www.enigmail.net/\n\niQEcBAEBCgAGBQJTFpCjAAoJEI5FoCIzSKrw1CsH/1E/0Wgs3RtXPLqWbwVoFy+U\nBc7dW7TBmb8EScC+3DedI4u9ryjZigjbsnBg1Y8V/gEtmUSmvt1e8CWTdvMLQpvx\nbnasL4uia/CBOg/aZkJ1iEBiHA3sUi9Es4FqoHbuGBn0bkDrA2NQvt3bCqNf6n8H\nPCeWx/qb8+F4niI0I8T5ASeqOHMxxSegHvlGezl6XZoGHa5SeLRrg7JtW3ZoWKCO\nq6GRzR6dV4FWJckfajUo34IUQNS4YA7wLpmC3PVUn3+EgF+affAEigjVWGRWdf2k\ncuaNu6hUAuD/2EHhCt6YP+ubV+FYiU86QOvmVifVpH1Apd29Fw4Kqnvyq2zJVC0=\n=hXsK\n-----END PGP SIGNATURE-----\n"},{"id":"236065","messageId":"53169549.10309@gmail.com","threadId":"36061","inReplyTo":"531690A3.3040509@ubuntu.com","subject":"Re: New directory lost by git am","fromName":"Chris Packham","fromEmail":"judge.packham@gmail.com","sentAt":"2014-03-05T03:08:57Z","receivedAt":"2014-03-05T03:08:57Z","isPatch":false,"sender":{"key":"judge.packham@gmail.com","avatar":"https://avatars.githubusercontent.com/u/155667?v=4"},"body":"Hi,\n\nOn 05/03/14 15:49, Phillip Susi wrote:\n> I applied a patch with git am that adds a new source file to a new\n> directory, and later noticed that file was missing from the commit.\n> It seems that git am fails to add the new file/directory to the index.\n> \n\nCould you provide a few more details such as your git version (git\n--version) and an example of the failure. I've tried to reproduce the\nproblem based on the description provided but everything seems to work\nas expected for me.\n\n  git --version\n    git version 1.9.0\n  mkdir test && cd test && git init\n  echo \"hello world\" >a.txt\n  git add a.txt\n  git commit -m\"Initial commit\"\n  git checkout -b temp\n  mkdir b\n  echo \"lorem ipsum\" >b/b.txt\n  git add b/b.txt\n  git commit -m\"Add b/b.txt\"\n  ls -R\n    .:\n    a.txt  b\n\n    ./b:\n    b.txt\n  git checkout master\n  git format-patch temp -1 --stdout | git am\n  ls -R\n    .:\n    a.txt  b\n\n    ./b:\n    b.txt\n"},{"id":"236066","messageId":"53169868.3010401@ubuntu.com","threadId":"36061","inReplyTo":"53169549.10309@gmail.com","subject":"Re: New directory lost by git am","fromName":"Phillip Susi","fromEmail":"psusi@ubuntu.com","sentAt":"2014-03-05T03:22:16Z","receivedAt":"2014-03-05T03:22:16Z","isPatch":false,"sender":{"key":"psusi@ubuntu.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA512\n\nOn 03/04/2014 10:08 PM, Chris Packham wrote:\n> Could you provide a few more details such as your git version (git \n> --version) and an example of the failure. I've tried to reproduce\n> the problem based on the description provided but everything seems\n> to work as expected for me.\n\nVersion 1.8.3.2.\n\n> git --version git version 1.9.0 mkdir test && cd test && git init \n> echo \"hello world\" >a.txt git add a.txt git commit -m\"Initial\n> commit\" git checkout -b temp mkdir b echo \"lorem ipsum\" >b/b.txt \n> git add b/b.txt git commit -m\"Add b/b.txt\" ls -R .: a.txt  b\n> \n> ./b: b.txt git checkout master git format-patch temp -1 --stdout |\n> git am ls -R .: a.txt  b\n> \n> ./b: b.txt\n> \n\nYou are reapplying the patch while it is already applied.  Do a reset\nHEAD~1 --hard, and git clean -x -f -d before git am.  I didn't notice\nthe missing file myself for some time because it is left in the\nworking tree, just not added to the index and included in the commit.\n\n\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.14 (GNU/Linux)\nComment: Using GnuPG with Thunderbird - http://www.enigmail.net/\n\niQEcBAEBCgAGBQJTFphoAAoJEI5FoCIzSKrwx78H/iTLvtMVb2hmn2g2YDQuJWe3\nnENrqlRNDF11YHpA9c7chxepcuP2CZaZjoXv45aCQG9Wx9XJyKPIbauhwqIIVUjR\nVYDORdtpn8u3Pf3WWyHYW+MEoupYyni4VYENVSjKnV6sLT951TuYI+4paHWat3lq\n/at9UkLy4d39hj2P/6M+voBbKWzblBZzP31lH6OY/Mno2zhh4eQChhsnZYPQ/Hfn\nREAeyB4WsLCjnPz+uEkOcWaEVVh+BwNU1UmK/tX+rzhBsaRzhDY5IIWTL9dfkD/z\nAf86IUSKdTjnMq7CTmVAmlxAfHXF0bgtlybrVY2Sdc8R/CqmWCz6USyKdUxgLIk=\n=Z3Z/\n-----END PGP SIGNATURE-----\n"},{"id":"236074","messageId":"5316DBEC.3020208@gmail.com","threadId":"36061","inReplyTo":"53169868.3010401@ubuntu.com","subject":"Re: New directory lost by git am","fromName":"Chris Packham","fromEmail":"judge.packham@gmail.com","sentAt":"2014-03-05T08:10:20Z","receivedAt":"2014-03-05T08:10:20Z","isPatch":false,"sender":{"key":"judge.packham@gmail.com","avatar":"https://avatars.githubusercontent.com/u/155667?v=4"},"body":"On 05/03/14 16:22, Phillip Susi wrote:\n> On 03/04/2014 10:08 PM, Chris Packham wrote:\n>> Could you provide a few more details such as your git version (git \n>> --version) and an example of the failure. I've tried to reproduce\n>> the problem based on the description provided but everything seems\n>> to work as expected for me.\n> \n> Version 1.8.3.2.\n> \n>> git --version git version 1.9.0 mkdir test && cd test && git init \n>> echo \"hello world\" >a.txt git add a.txt git commit -m\"Initial\n>> commit\" git checkout -b temp mkdir b echo \"lorem ipsum\" >b/b.txt \n>> git add b/b.txt git commit -m\"Add b/b.txt\" ls -R .: a.txt  b\n>>\n>> ./b: b.txt git checkout master git format-patch temp -1 --stdout |\n>> git am ls -R .: a.txt  b\n>>\n>> ./b: b.txt\n>>\n> \n> You are reapplying the patch while it is already applied.\n\nMy example is creating a commit on the \"temp\" branch then applying it to\nthe \"master\" branch using git am.\n\n> Do a reset\n> HEAD~1 --hard, and git clean -x -f -d before git am.  I didn't notice\n> the missing file myself for some time because it is left in the\n> working tree, just not added to the index and included in the commit.\n> \n\nRegardless of reproducing the issue a quick glance at the Release notes\nfor 1.8.3.3 the following sticks out:\n\nFixes since v1.8.3.2\n--------------------\n\n * \"git apply\" parsed patches that add new files, generated by programs\n   other than Git, incorrectly.  This is an old breakage in v1.7.11.\n\nDoes that sound like your problem? If you can I'd suggest updating,\nideally to the recent 1.9.0 release but if you're feeling conservative\ntry 1.8.3.4.\n"},{"id":"236088","messageId":"53173423.6050708@ubuntu.com","threadId":"36061","inReplyTo":"5316DBEC.3020208@gmail.com","subject":"Re: New directory lost by git am","fromName":"Phillip Susi","fromEmail":"psusi@ubuntu.com","sentAt":"2014-03-05T14:26:43Z","receivedAt":"2014-03-05T14:26:43Z","isPatch":false,"sender":{"key":"psusi@ubuntu.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nOn 3/5/2014 3:10 AM, Chris Packham wrote:\n> My example is creating a commit on the \"temp\" branch then applying\n> it to the \"master\" branch using git am.\n> \n>> Do a reset HEAD~1 --hard, and git clean -x -f -d before git am.\n>> I didn't notice the missing file myself for some time because it\n>> is left in the working tree, just not added to the index and\n>> included in the commit.\n>> \n\nRight... so the file is left in the directory, even though it is not\nchecked in.  A git status should show it is an unknown file, and a\nclean should remove it.\n\n> Regardless of reproducing the issue a quick glance at the Release\n> notes for 1.8.3.3 the following sticks out:\n> \n> Fixes since v1.8.3.2 --------------------\n> \n> * \"git apply\" parsed patches that add new files, generated by\n> programs other than Git, incorrectly.  This is an old breakage in\n> v1.7.11.\n> \n> Does that sound like your problem? If you can I'd suggest\n> updating, ideally to the recent 1.9.0 release but if you're feeling\n> conservative try 1.8.3.4.\n\nVaguely, except for the \"other than git\" part.  This patch was\ngenerated by git-format-patch ( I didn't think apply handled patches\nthat weren't ).\n\n\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v2.0.17 (MingW32)\nComment: Using GnuPG with Thunderbird - http://www.enigmail.net/\n\niQEcBAEBAgAGBQJTFzQiAAoJEI5FoCIzSKrwXEMH/iQdFdAApGnFCyLGA4l87d4M\nITLtyL632gHA39KnlEqtTc6TgFjQMrV3m8s9TeR6DiEI9qGvNvnP2E4JBFORZprk\nRSJoCa9qMuAYOtSEtwrzbhMZpBN7hAZeJ7txP2KwZiGXoWjr4RSawjViQHhnXQEP\nL9QMmwjWJBwZE/eklYg8W+Ov987uribGTgOL8Wx1iMME2C88VfyCWtg1ClTz3aEh\nUeyAvyLxSA+YvS4xg+nUBAxXX8bFI0g53Yjf3Lt/1/EzdO67sH7CRChR67BmANWZ\nNBoxBqenN6/qg0rfRojOnjGTtRpAJX48dgnEHulUJrixrmqBYXAbi8dNVW8KkiM=\n=tzX5\n-----END PGP SIGNATURE-----\n"},{"id":"236091","messageId":"20140305163415.GA28908@sigill.intra.peff.net","threadId":"36061","inReplyTo":"53173423.6050708@ubuntu.com","subject":"Re: New directory lost by git am","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-03-05T16:34:15Z","receivedAt":"2014-03-05T16:34:15Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Mar 05, 2014 at 09:26:43AM -0500, Phillip Susi wrote:\n\n> -----BEGIN PGP SIGNED MESSAGE-----\n> Hash: SHA1\n> \n> On 3/5/2014 3:10 AM, Chris Packham wrote:\n> > My example is creating a commit on the \"temp\" branch then applying\n> > it to the \"master\" branch using git am.\n> > \n> >> Do a reset HEAD~1 --hard, and git clean -x -f -d before git am.\n> >> I didn't notice the missing file myself for some time because it\n> >> is left in the working tree, just not added to the index and\n> >> included in the commit.\n> >> \n> \n> Right... so the file is left in the directory, even though it is not\n> checked in.  A git status should show it is an unknown file, and a\n> clean should remove it.\n\nI don't think those steps are necessary for Chris's example. When he\nswitches back to the master branch, git removes the subdirectory (the\nfile is tracked in \"temp\" but not \"master\", so we remove it when\nswitching branches, and then the directory is empty, so we clean it up,\ntoo). You can verify with an extra \"ls\" after the checkout but before\nthe \"am\".\n\n> > * \"git apply\" parsed patches that add new files, generated by\n> > programs other than Git, incorrectly.  This is an old breakage in\n> > v1.7.11.\n> > \n> > Does that sound like your problem? If you can I'd suggest\n> > updating, ideally to the recent 1.9.0 release but if you're feeling\n> > conservative try 1.8.3.4.\n> \n> Vaguely, except for the \"other than git\" part.  This patch was\n> generated by git-format-patch ( I didn't think apply handled patches\n> that weren't ).\n\nI can't get Chris's script to fail on any version of git. Can you show\nus an example of a patch that does not behave (or better yet, a\nreproduction recipe to generate the patch with \"format-patch\")?\n\n-Peff\n"},{"id":"236092","messageId":"53175510.7020000@ubuntu.com","threadId":"36061","inReplyTo":"20140305163415.GA28908@sigill.intra.peff.net","subject":"Re: New directory lost by git am","fromName":"Phillip Susi","fromEmail":"psusi@ubuntu.com","sentAt":"2014-03-05T16:47:12Z","receivedAt":"2014-03-05T16:47:12Z","isPatch":false,"sender":{"key":"psusi@ubuntu.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nOn 3/5/2014 11:34 AM, Jeff King wrote:\n> I don't think those steps are necessary for Chris's example. When\n> he switches back to the master branch, git removes the subdirectory\n> (the file is tracked in \"temp\" but not \"master\", so we remove it\n> when switching branches, and then the directory is empty, so we\n> clean it up, too). You can verify with an extra \"ls\" after the\n> checkout but before the \"am\".\n\nRight.\n\n>>> * \"git apply\" parsed patches that add new files, generated by \n>>> programs other than Git, incorrectly.  This is an old breakage\n>>> in v1.7.11.\n>>> \n>>> Does that sound like your problem? If you can I'd suggest \n>>> updating, ideally to the recent 1.9.0 release but if you're\n>>> feeling conservative try 1.8.3.4.\n>> \n>> Vaguely, except for the \"other than git\" part.  This patch was \n>> generated by git-format-patch ( I didn't think apply handled\n>> patches that weren't ).\n> \n> I can't get Chris's script to fail on any version of git. Can you\n> show us an example of a patch that does not behave (or better yet,\n> a reproduction recipe to generate the patch with \"format-patch\")?\n\nAHA!  It requires a conflict.  There were simple conflicts in the NEWS\nfile so I applied the patch with git am --reject and fixed up the\nNEWS, and ran git am --resolved.  The git am --reject fails to add the\nnew directory to the index.\n\n\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v2.0.17 (MingW32)\nComment: Using GnuPG with Thunderbird - http://www.enigmail.net/\n\niQEcBAEBAgAGBQJTF1UOAAoJEI5FoCIzSKrwTD4H/35pUf8DFsbwPIVVQi+8I8e3\n5NMHwQrHK3TPbZigVPBgVfwRCtOAxX656BPhninfhix99HWs00W5zGaFDwkymRNp\n87EeU3LVcIjapqijszw9AqwBLvfm9uzXEus964hShCJVOmKBezQfl6Mvcrkn5Na1\nUchJLkRzEoi6VUyUso8FH0xpL7JyjF08H19dtvXoUbrvrXYuN1Ys3UMBHXVEVdi+\n5O924lo4+psgdjGZ3HUpclYRbKO0LS5IVMCxFRw5Q+EfARJQ7NXzv/csRXIKyms7\nroCQqmQnnem71GHx6SQaepnY5pKuEnmmDaqXbCOqZdpyfo1CB7SFJDq/VXrbLyw=\n=zS2r\n-----END PGP SIGNATURE-----\n"},{"id":"236093","messageId":"20140305171334.GA31252@sigill.intra.peff.net","threadId":"36061","inReplyTo":"53175510.7020000@ubuntu.com","subject":"Re: New directory lost by git am","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-03-05T17:13:34Z","receivedAt":"2014-03-05T17:13:34Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Mar 05, 2014 at 11:47:12AM -0500, Phillip Susi wrote:\n\n> > I can't get Chris's script to fail on any version of git. Can you\n> > show us an example of a patch that does not behave (or better yet,\n> > a reproduction recipe to generate the patch with \"format-patch\")?\n> \n> AHA!  It requires a conflict.  There were simple conflicts in the NEWS\n> file so I applied the patch with git am --reject and fixed up the\n> NEWS, and ran git am --resolved.  The git am --reject fails to add the\n> new directory to the index.\n\nThanks, I can reproduce here. I do not think it has anything to do with\nbeing in a subdirectory; any new file does not get added to the index.\nIn fact, I do not think we update the index at all with \"--reject\". For\nexample, try this:\n\n    git init repo &&\n    cd repo &&\n\n    echo base >conflict &&\n    echo base >modified &&\n    git add . &&\n    git commit -m base &&\n\n    echo master >conflict &&\n    git add . &&\n    git commit -m master &&\n\n    git checkout -b other HEAD^ &&\n    echo other >conflict &&\n    echo other >modified &&\n    echo other >new &&\n    git add . &&\n    git commit -m other &&\n\n    git checkout master &&\n    git format-patch other -1 --stdout >patch &&\n    git am --reject patch\n\nRunning \"git status -s\" shows:\n\n   M modified\n   ?? conflict.rej\n   ?? new\n   ?? patch\n\nWe apply the changes to \"modified\" and \"new\" to the working tree, but we\ndo not stage anything in the index. I suspect this is because our\ninvocation of \"apply --index\" (which is what is doing the real work with\n\"--reject\" here) bails before touching the index. In theory it should be\nable to update the index for files that applied cleanly and leave the\nother ones alone.\n\nBut I have not thought hard about it, so maybe there is a good reason\nnot to (it is a little weird just because the resulting index is a\npartial application of the patch).  The \"am -3\" path does what you want\nhere, but it is much simpler: it knows it can represent the 3-way\nconflict in the index. So the index represents the complete state of the\npatch application at the end, including conflicts.\n\n-Peff\n"},{"id":"236105","messageId":"53176D07.3020206@ubuntu.com","threadId":"36061","inReplyTo":"20140305171334.GA31252@sigill.intra.peff.net","subject":"Re: New directory lost by git am","fromName":"Phillip Susi","fromEmail":"psusi@ubuntu.com","sentAt":"2014-03-05T18:29:27Z","receivedAt":"2014-03-05T18:29:27Z","isPatch":false,"sender":{"key":"psusi@ubuntu.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nOn 3/5/2014 12:13 PM, Jeff King wrote:\n> We apply the changes to \"modified\" and \"new\" to the working tree,\n> but we do not stage anything in the index. I suspect this is\n> because our invocation of \"apply --index\" (which is what is doing\n> the real work with \"--reject\" here) bails before touching the\n> index. In theory it should be able to update the index for files\n> that applied cleanly and leave the other ones alone.\n\nYikes, that's really bad.\n\n> But I have not thought hard about it, so maybe there is a good\n> reason not to (it is a little weird just because the resulting\n> index is a partial application of the patch).  The \"am -3\" path\n> does what you want here, but it is much simpler: it knows it can\n> represent the 3-way conflict in the index. So the index represents\n> the complete state of the patch application at the end, including\n> conflicts.\n\nyes, but -3 fails if it can't find the parent blobs.\n\n\n\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v2.0.17 (MingW32)\nComment: Using GnuPG with Thunderbird - http://www.enigmail.net/\n\niQEcBAEBAgAGBQJTF20HAAoJEI5FoCIzSKrwFfQIAJfPmplu7zskercvjnuZGCle\nccTzK0rYtrwQn/78Vrbc6kqcrQvbvtrqUMN4/ILJ5xeaO80Gzzz8pchBPNN8khhY\nVBQiWUOrKzBH1vszveneva+gFUrLIWk2KI6T8lGTnYulvRVe38WRAwr/8qEClPX6\nhUnYChM17WE+KTV39ezA6ww9ZAyOX7EHq87PJp5nVgBB4HkmkDmccfxYTFNB4FGg\nPPqun8g0Fyytd+Qrsk2M5L6NsPUXi32wIt8EWcyPwU6QrQgKbuWK7QlVkcPPzecM\neHifKm8V1V0VKudm3S8jbaUDG2KnLIdMveXu/e9Hn7+YgDQh9zM1m7f+NVJDvjk=\n=NAe9\n-----END PGP SIGNATURE-----\n"},{"id":"236115","messageId":"xmqqtxbch361.fsf@gitster.dls.corp.google.com","threadId":"36061","inReplyTo":"20140305171334.GA31252@sigill.intra.peff.net","subject":"Re: New directory lost by git am","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-03-05T19:10:46Z","receivedAt":"2014-03-05T19:10:46Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> But I have not thought hard about it, so maybe there is a good reason\n> not to (it is a little weird just because the resulting index is a\n> partial application of the patch).\n\nOriginally \".rej\" was a deliberate attempt to be \"not very Git but\nmore like 'patch'\", so I wouldn't be surprised if the combination\nbetween \"--reject\" and \"--index\" did not work, in the sense that we\ndid not add such a partial change to the index.\n\nI do not offhand think of a reason to forbid the combination,\nthough, as long as we make sure that \"git apply --index --reject\"\nstill exits with failure to prevent a partial application to be\nauto-committed.\n"}]}