{"thread":{"id":"11351","subject":"bug with .gitattributes diff and embedded NUL","startedAt":"2007-12-19T05:07:27Z","lastAt":"2007-12-19T14:19:16Z","messageCount":3,"participants":["Eric Blake","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"63714","messageId":"4768A70F.8020904@byu.net","threadId":"11351","inReplyTo":null,"subject":"bug with .gitattributes diff and embedded NUL","fromName":"Eric Blake","fromEmail":"ebb9@byu.net","sentAt":"2007-12-19T05:07:27Z","receivedAt":"2007-12-19T05:07:27Z","isPatch":false,"sender":{"key":"eblake@redhat.com","avatar":"https://avatars.githubusercontent.com/u/32933908?v=4"},"body":"The man page for gitattributes suggests that one use of the diff attribute\nis to force text-based diffs, even when the file contains embedded NULs.\nYet actually trying this falls flat, since git generates invalid diffs\nwhen encountering embedded NUL.\n\nHere is a pared-down example showing the bug.  I based it off of what I\nwas trying to do in the git repository for GNU m4, where I was working on\nadding testcases to make m4 seamlessly handle embedded NUL.\n\n$ git --version\ngit version 1.5.4.rc0.67.gf9c5\n$ mkdir sample\n$ cd sample\n$ git init\nInitialized empty Git repository in .git/\n$ printf 'context with\\0embedded nul\\nbefore\\nmore context\\n' > file\n$ git add file\n$ git commit -m initial\nCreated initial commit 25108f6: initial\n 1 files changed, 0 insertions(+), 0 deletions(-)\n create mode 100644 file\n$ git tag t1\n$ sed -i 's/before/after/' file\n$ git add file\n$ git commit -m later\nCreated commit a5ee10c: later\n 1 files changed, 0 insertions(+), 0 deletions(-)\n$ git tag t2\n$ git reset --hard t1\nHEAD is now at 25108f6... initial\n$ sed -i 's/more/some &/' file\n$ echo 'file diff' > .gitattributes\n$ git add .gitattributes file\n$ git commit -m 'switch file to text mode'\nCreated commit 0f2a9b2: switch file to text mode\n 2 files changed, 2 insertions(+), 1 deletions(-)\n create mode 100644 .gitattributes\n$ git tag t3\n$ git reset --hard t2\nHEAD is now at a5ee10c... later\n$ git rebase --onto t3 t1\nFirst, rewinding head to replay your work on top of it...\nHEAD is now at 0f2a9b2... switch file to text mode\nApplying later\nerror: patch failed: file:1\nerror: file: patch does not apply\nUsing index info to reconstruct a base tree...\nerror: patch failed: file:1\nerror: file: patch does not apply\nDid you hand edit your patch?\nIt does not apply to blobs recorded in its index.\nCannot fall back to three-way merge.\nPatch failed at 0001.\n\nWhen you have resolved this problem run \"git rebase --continue\".\nIf you would prefer to skip this patch, instead run \"git rebase --skip\".\nTo restore the original branch and stop rebasing run \"git rebase --abort\".\n\nLooking closer at .dotest/0001, the diff is indeed invalid, containing the\nsingle line\n\n context with-before\n\ninstead of the desired two lines\n\n context with^@embedded nul\n-before\n\nI'm also wondering if using GIT_EXTERNAL_DIFF would provide a way to work\naround this bug in git diff, since GNU diffutils doesn't seem to have any\nproblems when doing text-mode diffs with embedded NUL:\n\n$ diff -u -a <(git show t1:file) <(git show t2:file) | cat -v\n--- /dev/fd/63\t2006-11-30 17:00:00.000000000 -0700\n+++ /dev/fd/62\t2006-11-30 17:00:00.000000000 -0700\n@@ -1,3 +1,3 @@\n context with^@embedded nul\n-before\n+after\n more context\n\n-- \nDon't work too hard, make some time for fun as well!\n\nEric Blake             ebb9@byu.net\n"},{"id":"63715","messageId":"7vve6vw5kg.fsf@gitster.siamese.dyndns.org","threadId":"11351","inReplyTo":"4768A70F.8020904@byu.net","subject":"Re: bug with .gitattributes diff and embedded NUL","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-12-19T06:40:15Z","receivedAt":"2007-12-19T06:40:15Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Blake <ebb9@byu.net> writes:\n\n> Looking closer at .dotest/0001, the diff is indeed invalid, containing the\n> single line\n\nThat is not a bug in diff but actually in bug in non-merge form of\nrebase, which reuses \"format patch as if we are sending it out in\ne-mail, and apply that on top of a different commit as if we are\naccepting that e-mail\" machinery.  That dataflow loses NUL.  Fixing\nrebase not to reuse the e-mailed patch dataflow is on my to-do list, but\nhas been slipping.\n\nSo, no, I do not think GIT_EXTERNAL_DIFF would help, as the problem is\nnot diff generation.\n\nI think you can use \"rebase -m\" to work this issue around.\n"},{"id":"63767","messageId":"47692864.3090300@byu.net","threadId":"11351","inReplyTo":"7vve6vw5kg.fsf@gitster.siamese.dyndns.org","subject":"Re: bug with .gitattributes diff and embedded NUL","fromName":"Eric Blake","fromEmail":"ebb9@byu.net","sentAt":"2007-12-19T14:19:16Z","receivedAt":"2007-12-19T14:19:16Z","isPatch":false,"sender":{"key":"eblake@redhat.com","avatar":"https://avatars.githubusercontent.com/u/32933908?v=4"},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nAccording to Junio C Hamano on 12/18/2007 11:40 PM:\n> Eric Blake <ebb9@byu.net> writes:\n> \n>> Looking closer at .dotest/0001, the diff is indeed invalid, containing the\n>> single line\n> \n> That dataflow loses NUL.  Fixing\n> rebase not to reuse the e-mailed patch dataflow is on my to-do list, but\n> has been slipping.\n> \n> I think you can use \"rebase -m\" to work this issue around.\n\nIndeed, after learning about git-rebase -m and git-mergetool, adding the\nmerge option alongside the diff option for the file in question, and\nteaching emacs that it should use emerge-diff-options \"-a\" so that it can\nhandle embedded NUL, I was finally able to use git-mergetool to do a sane\nmerge of the differences in the mostly-text file.  Thanks!\n\nBut it raised some additional issues:\n\nIs there a way to specify custom merge tools, rather than the current\nhard-coded list valid for merge.tool?  For example, what if I prefer\nemacs' ediff driver over its emerge driver?\n\nUsing merge.tool of emerge leaves junk around when the merge is aborted.\nBefore I edited emacs' emerge-diff-options, the subsidiary diff3 was\nfailing because it treated the file to merge as binary even though git was\ntrying to treat it as text.  As a result, emerge left behind all of its\ncommand-line argument files (file.{BASE,LOCAL,REMOTE}.pid), and git did\nnot clean any of them up.\n\n- --\nDon't work too hard, make some time for fun as well!\n\nEric Blake             ebb9@byu.net\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.5 (Cygwin)\nComment: Public key at home.comcast.net/~ericblake/eblake.gpg\nComment: Using GnuPG with Mozilla - http://enigmail.mozdev.org\n\niD8DBQFHaShk84KuGfSFAYARAkDtAJ0XhhhbeJZJS6td3/aeOlxd7iuoawCgiVY6\n3s+aMxRO1vjKf/EqncKW2jE=\n=GrcF\n-----END PGP SIGNATURE-----\n"}]}