{"thread":{"id":"64430","subject":"[RFC PATCH] diff: add option to report binary files in raw diffs","startedAt":"2025-11-04T02:15:14Z","lastAt":"2025-11-07T17:26:55Z","messageCount":12,"participants":["Justin Tobler","Junio C Hamano","Ben Knoble","Torsten Bögershausen"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"530171","messageId":"20251104021455.379807-1-jltobler@gmail.com","threadId":"64430","inReplyTo":null,"subject":"[RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-11-04T02:14:55Z","receivedAt":"2025-11-04T02:15:14Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"When generating patch diff output, if either side of a filepair is\ndetected as binary, Git omits the diff content and instead prints a\n\"Binary files differ\" message. From this message it is known that at\nleast one of the files in the pair is considered binary, but not exactly\nwhich ones.\n\nAdd a --report-binary-files diff option that, when enabled, extends the\nraw diff output format to explicitly indicate for each file whether it\nwas considered binary or not.\n\nSigned-off-by: Justin Tobler \n---\n\nGreetings,\n\nI have a usecase where I would like to know exactly which files in a\ndiff pair are considered binary by Git when computing diffs. When\ncomputing patch diff output, Git already omits filepair diffs where at\nleast one side is considered binary and prints a \"binary files differ\"\nmessage instead. From this message we cannot discern exactly which files\nwere considered binary by Git though.\n\nIn this patch, the raw diff format is extended with a\n`--report-binary-files` option to explicitly specify which files in the\ndiff pair were considered binary. The output in this form looks\nsomething like this:\n\n $ git diff-tree --abbrev=8 --report-binary-files HEAD~ HEAD\n :100644 100644 a1961526 e231acb1 bt M\tfoo\n :100644 100644 31eedd5c 402a70d7 bb M\tbar\n\nWith this format, there is a new column before the status that specifies\nthe binary status for each file. 'b' indicates binary and 't' is used\notherwise.\n\nIn an earlier iteration of this patch, I originally extended the patch\noutput \"binary files differ\" message to indicate the binary status for\neach file in the diff pair, but felt it wasn't the best place to do so\nsince I also want it to be machine friendly. So I ended up extending the\nraw diff format instead.\n\nI'm not entirely sure the current implementation is most ideal format\nhere so I'm very open to feedback. :)\n\n-Justin\n\n---\n Documentation/diff-format.adoc | 12 ++++++++++++\n Documentation/diff-options.adoc | 4 ++++\n diff.c | 9 +++++++++\n diff.h | 6 ++++++\n t/t4012-diff-binary.sh | 29 +++++++++++++++++++++++++++++\n 5 files changed, 60 insertions(+)\n\ndiff --git a/Documentation/diff-format.adoc b/Documentation/diff-format.adoc\nindex 9f7e988241..74c0a064ad 100644\n--- a/Documentation/diff-format.adoc\n+++ b/Documentation/diff-format.adoc\n@@ -83,6 +83,18 @@ quoted as explained for the configuration variable `core.quotePath`\n (see linkgit:git-config[1]). Using `-z` the filename is output\n verbatim and the line is terminated by a NUL byte.\n \n+With the `--report-binary-files` option, a new column is added prior to the\n+status indicating for each file if Git considered it binary or not. If\n+considered binary, a file is denoted with `b`. Otherwise, `t` is used. This\n+column is followed by a space character. Combined diffs do not report binary\n+file info.\n+\n+Example:\n+\n+------------------------------------------------\n+:100644 100644 5be4a4a cc95eb0 bt M file.c\n+------------------------------------------------\n+\n diff format for merges\n ----------------------\n \ndiff --git a/Documentation/diff-options.adoc b/Documentation/diff-options.adoc\nindex ae31520f7f..54eb48c067 100644\n--- a/Documentation/diff-options.adoc\n+++ b/Documentation/diff-options.adoc\n@@ -544,6 +544,10 @@ ifndef::git-format-patch[]\n \tImplies `--patch`.\n endif::git-format-patch[]\n \n+`--report-binary-files`::\n+\tAdds a column to raw diff output to report for each file in the pair\n+\twhether it was considered binary by Git.\n+\n `--abbrev[=]`::\n \tInstead of showing the full 40-byte hexadecimal object\n \tname in diff-raw format output and diff-tree header\ndiff --git a/diff.c b/diff.c\nindex a1961526c0..e231acb1a9 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -5747,6 +5747,8 @@ struct option *add_diff_options(const struct option *opts,\n \t\tOPT_CALLBACK_F(0, \"binary\", options, NULL,\n \t\t\t N_(\"output a binary diff that can be applied\"),\n \t\t\t PARSE_OPT_NONEG | PARSE_OPT_NOARG, diff_opt_binary),\n+\t\tOPT_BOOL(0, \"report-binary-files\", &options->report_binary_files,\n+\t\t\t N_(\"report if pre- and post-image blobs are binary\")),\n \t\tOPT_BOOL(0, \"full-index\", &options->flags.full_index,\n \t\t\t N_(\"show full pre- and post-image object names on the \\\"index\\\" lines\")),\n \t\tOPT_COLOR_FLAG(0, \"color\", &options->use_color,\n@@ -6111,6 +6113,13 @@ static void diff_flush_raw(struct diff_filepair *p, struct diff_options *opt)\n \t\tfprintf(opt->file, \"%s \",\n \t\t\tdiff_aligned_abbrev(&p->two->oid, opt->abbrev));\n \t}\n+\n+\tif (opt->report_binary_files) {\n+\t\tchar one = diff_filespec_is_binary(opt->repo, p->one) ? 'b' : 't';\n+\t\tchar two = diff_filespec_is_binary(opt->repo, p->two) ? 'b' : 't';\n+\t\tfprintf(opt->file, \"%c%c \", one, two);\n+\t}\n+\n \tif (p->score) {\n \t\tfprintf(opt->file, \"%c%03d%c\", p->status, similarity_index(p),\n \t\t\tinter_name_termination);\ndiff --git a/diff.h b/diff.h\nindex 31eedd5c0c..402a70d7ad 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -369,6 +369,12 @@ struct diff_options {\n \t */\n \tint skip_resolving_statuses;\n \n+\t/*\n+\t * When generating raw diff output, report for each file whether it was\n+\t * considered binary.\n+\t */\n+\tint report_binary_files;\n+\n \t/* Callback which allows tweaking the options in diff_setup_done(). */\n \tvoid (*set_default)(struct diff_options *);\n \ndiff --git a/t/t4012-diff-binary.sh b/t/t4012-diff-binary.sh\nindex d1d30ac2a9..e026e1d3a4 100755\n--- a/t/t4012-diff-binary.sh\n+++ b/t/t4012-diff-binary.sh\n@@ -130,4 +130,33 @@ test_expect_success 'diff --stat with binary files and big change count' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success SHA1 'diff --report-binary-files' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\n+\t\techo foo >foo &&\n+\t\tprintf \"\\0bar\\0\" >bar &&\n+\t\techo baz >baz &&\n+\t\tgit add foo bar baz &&\n+\t\tgit commit -m foo &&\n+\n+\t\tprintf \"\\0foo\\0\" >foo &&\n+\t\tprintf \"\\0bar2\\0\" >bar &&\n+\t\techo baz2 >baz &&\n+\t\tgit commit -am \"binary foo\" &&\n+\n+\t\tcat >expect <<-\\EOF &&\n+\t\t:100644 100644 e02d9a3a8aeb904ccc3bb9ed0600f2e963ba1a10 884a24af772a87733e911a3491c0ab576d34c06c bb M\tbar\n+\t\t:100644 100644 76018072e09c5d31c8c6e3113b8aa0fe625195ca 3414c84ca6b7ca9cbbe40dd44f4d0715c1464f6e tt M\tbaz\n+\t\t:100644 100644 257cc5642cb1a054f08cc83f2d943e56fd3ebe99 a60073ceafeca287824d7b9ac3eebef233b72fce tb M\tfoo\n+\t\tEOF\n+\n+\t\tgit diff-tree --report-binary-files HEAD~ HEAD >out &&\n+\n+\t\ttest_cmp expect out\n+\t)\n+'\n+\n test_done\n\nbase-commit: 7f278e958afbf9b7e0727631b4c26dcfa1c63d6e\n-- \n2.51.0.193.g4975ec3473b\n\n"},{"id":"530172","messageId":"xmqqa512sfcj.fsf@gitster.g","threadId":"64430","inReplyTo":"20251104021455.379807-1-jltobler@gmail.com","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-11-04T02:26:20Z","receivedAt":"2025-11-04T02:26:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Justin Tobler writes:\n\n> I have a usecase where I would like to know exactly which files in a\n> diff pair are considered binary by Git when computing diffs. When\n> computing patch diff output, Git already omits filepair diffs where at\n> least one side is considered binary and prints a \"binary files differ\"\n> message instead. From this message we cannot discern exactly which files\n> were considered binary by Git though.\n\nI have a usecase where I would like to know exactly which side of a\ndiff filepair ends in an incomplete line in a concise format.\n\nShould we add yet another column to the raw output to indicate who\nis complete and who is incomplete?\n\nWhere does it lead us and when will it stop?\n\nIOW, yuck ;-).\n\n> In this patch, the raw diff format is extended with a\n> `--report-binary-files` option to explicitly specify which files in the\n> diff pair were considered binary. The output in this form looks\n> something like this:\n>\n> $ git diff-tree --abbrev=8 --report-binary-files HEAD~ HEAD\n> :100644 100644 a1961526 e231acb1 bt M\tfoo\n> :100644 100644 31eedd5c 402a70d7 bb M\tbar\n>\n> With this format, there is a new column before the status that specifies\n> the binary status for each file. 'b' indicates binary and 't' is used\n> otherwise.\n\nHow will would this extend beyond 2-way diffs, I wonder. \nShould \n\n $ git show -c --report-binary \n\nshow [bt]{3} instead of [bt]{2} before the change status letter?\n"},{"id":"530173","messageId":"xmqqzf92quen.fsf@gitster.g","threadId":"64430","inReplyTo":"xmqqa512sfcj.fsf@gitster.g","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-11-04T04:44:00Z","receivedAt":"2025-11-04T04:44:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Justin Tobler writes:\n>\n>> I have a usecase where I would like to know exactly which files in a\n>> diff pair are considered binary by Git when computing diffs. When\n>> computing patch diff output, Git already omits filepair diffs where at\n>> least one side is considered binary and prints a \"binary files differ\"\n>> message instead. From this message we cannot discern exactly which files\n>> were considered binary by Git though.\n>\n> I have a usecase where I would like to know exactly which side of a\n> diff filepair ends in an incomplete line in a concise format.\n>\n> Should we add yet another column to the raw output to indicate who\n> is complete and who is incomplete?\n>\n> Where does it lead us and when will it stop?\n>\n> IOW, yuck ;-).\n\nMy point being that it will be a huge mistake to do this only by\nsingling a trait that is not so special as if it is very special,\nonly because you have been thinking about it too long (the \"ends in\nan incomplete line\" trait is what has been on my mind for the past\nfew days, \"this side is binary\" may be what you've been thinking\nabout). There are many other things people would want to learn\nconcisely in machine readable format, like \"where did the file stop\nusing CRLF line endings and swithced to LF line endings\", that are\nequally plausible as the question you are asking, or the question I\nwould be asking \"which commit lost the final newline?\"\n\nPerhaps an extensible command line option syntax like\n\n $ git log --raw-extended=binary,incomplete,crlf,...\n\nis in order, and the presense of these options would add \"tt,ic,cl\"\nsomewhere in the output to signal that both sides are text, preimage\nends in an incomplete line but not postimage, and preimage uses crlf\nbut postimage uses lf, or something?\n\nExtending beyond 2-way diff is still something we would need to\nthink about, I guess, but the only thing we need to do may be to\nallow N-letter tuples instead of limiting ourselves to 2-letter\npairs, perhaps?\n\n>> In this patch, the raw diff format is extended with a\n>> `--report-binary-files` option to explicitly specify which files in the\n>> diff pair were considered binary. The output in this form looks\n>> something like this:\n>>\n>> $ git diff-tree --abbrev=8 --report-binary-files HEAD~ HEAD\n>> :100644 100644 a1961526 e231acb1 bt M\tfoo\n>> :100644 100644 31eedd5c 402a70d7 bb M\tbar\n>>\n>> With this format, there is a new column before the status that specifies\n>> the binary status for each file. 'b' indicates binary and 't' is used\n>> otherwise.\n>\n> How will would this extend beyond 2-way diffs, I wonder. \n> Should \n>\n> $ git show -c --report-binary \n>\n> show [bt]{3} instead of [bt]{2} before the change status letter?\n"},{"id":"530230","messageId":"nxl3sgs3h2psylifnbwcjmubdfmfhj53jjun5nsa4aqq5robh7@quga463ajvbu","threadId":"64430","inReplyTo":"xmqqzf92quen.fsf@gitster.g","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-11-05T00:17:05Z","receivedAt":"2025-11-05T00:17:09Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/11/03 08:44PM, Junio C Hamano wrote:\n> Junio C Hamano writes:\n> \n> > Justin Tobler writes:\n> >\n> >> I have a usecase where I would like to know exactly which files in a\n> >> diff pair are considered binary by Git when computing diffs. When\n> >> computing patch diff output, Git already omits filepair diffs where at\n> >> least one side is considered binary and prints a \"binary files differ\"\n> >> message instead. From this message we cannot discern exactly which files\n> >> were considered binary by Git though.\n> >\n> > I have a usecase where I would like to know exactly which side of a\n> > diff filepair ends in an incomplete line in a concise format.\n> >\n> > Should we add yet another column to the raw output to indicate who\n> > is complete and who is incomplete?\n> >\n> > Where does it lead us and when will it stop?\n> >\n> > IOW, yuck ;-).\n> \n> My point being that it will be a huge mistake to do this only by\n> singling a trait that is not so special as if it is very special,\n> only because you have been thinking about it too long (the \"ends in\n> an incomplete line\" trait is what has been on my mind for the past\n> few days, \"this side is binary\" may be what you've been thinking\n> about). There are many other things people would want to learn\n> concisely in machine readable format, like \"where did the file stop\n> using CRLF line endings and swithced to LF line endings\", that are\n> equally plausible as the question you are asking, or the question I\n> would be asking \"which commit lost the final newline?\"\n\nCompletely fair. Having a bunch specific options for special info we\nwant to add to the raw diff format would get messy quickly and is not\nvery extensible.\n\n> Perhaps an extensible command line option syntax like\n> \n> $ git log --raw-extended=binary,incomplete,crlf,...\n\nI quite like this and agree it would be better to have a single\nextensible option.\n\n> is in order, and the presense of these options would add \"tt,ic,cl\"\n> somewhere in the output to signal that both sides are text, preimage\n> ends in an incomplete line but not postimage, and preimage uses crlf\n> but postimage uses lf, or something?\n\nMaybe the output should be something like:\n\n binary=tt,incomplete=ic,crlf=cl\n\nor something along those lines. That way we could freely extend in the\nfuture without having to worry about a specific order. If we think all\nof the raw diff extension modes would only report with yes/no for each\nfile we could just do:\n\n binary=yn,incomplete=yy,crlf=nn\n\nbut maybe we should be more flexible and leave it up to the mode to\ndecide what its values can be?\n\nAlso, maybe this info could be on a newline following each raw diff\nentry? Something like:\n\n :100644 100644 a1961526 e231acb1 M\tfoo\n binary=yy\n :100644 100644 31eedd5c 402a70d7 M\tbar\n binary=nn\n\n> Extending beyond 2-way diff is still something we would need to\n> think about, I guess, but the only thing we need to do may be to\n> allow N-letter tuples instead of limiting ourselves to 2-letter\n> pairs, perhaps?\n\nYa, for combined diffs I think we could just add another letter for each\nsource? Something like:\n\n $ git diff-tree -c --raw-extended=binary \n ::100644 100644 100644 f38991c02a 2defd2d465 54f409c249 MM foo\n binary=yyy\n\nI think it would be reasonable to expect that each extension mode\n(binary, incomplete, crlf, etc) would want to check the commit and each\nof its sources.\n\nThanks for the feedback :)\n\n-Justin\n"},{"id":"530237","messageId":"xmqqtsz8nbwv.fsf@gitster.g","threadId":"64430","inReplyTo":"nxl3sgs3h2psylifnbwcjmubdfmfhj53jjun5nsa4aqq5robh7@quga463ajvbu","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-11-05T08:04:00Z","receivedAt":"2025-11-05T08:04:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Justin Tobler writes:\n\n> Maybe the output should be something like:\n>\n> binary=tt,incomplete=ic,crlf=cl\n>\n> or something along those lines. That way we could freely extend in the\n> future without having to worry about a specific order. If we think all\n> of the raw diff extension modes would only report with yes/no for each\n> file we could just do:\n>\n> binary=yn,incomplete=yy,crlf=nn\n>\n> but maybe we should be more flexible and leave it up to the mode to\n> decide what its values can be?\n>\n> Also, maybe this info could be on a newline following each raw diff\n> entry? Something like:\n>\n> :100644 100644 a1961526 e231acb1 M\tfoo\n> binary=yy\n> :100644 100644 31eedd5c 402a70d7 M\tbar\n> binary=nn\n\nI know these are parse-able, but quite honestly, both sounds\nsomewhat backwards, if you meant to make this easier to parse by\nsimple scripts. Scripts do not mind their input line wider than 80\ncolumns, but it is cumbersome if they have to take each pair of\nlines and combine them to process. And repeated keywords like\nbinary= etc., do not look like it is less error prone for scripts to\nparse them out, either. So, I dunno.\n"},{"id":"530239","messageId":"D80AE9D4-EE8D-4CAE-9212-2A592F45AB90@gmail.com","threadId":"64430","inReplyTo":"nxl3sgs3h2psylifnbwcjmubdfmfhj53jjun5nsa4aqq5robh7@quga463ajvbu","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2025-11-05T12:14:12Z","receivedAt":"2025-11-05T12:14:24Z","isPatch":true,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"\n> Le 4 nov. 2025 à 19:17, Justin Tobler a écrit :\n> \n> On 25/11/03 08:44PM, Junio C Hamano wrote:\n>> Junio C Hamano writes:\n>> \n>>> Justin Tobler writes:\n>>> \n>>>> I have a usecase where I would like to know exactly which files in a\n>>>> diff pair are considered binary by Git when computing diffs. When\n>>>> computing patch diff output, Git already omits filepair diffs where at\n>>>> least one side is considered binary and prints a \"binary files differ\"\n>>>> message instead. From this message we cannot discern exactly which files\n>>>> were considered binary by Git though.\n>>> \n>>> I have a usecase where I would like to know exactly which side of a\n>>> diff filepair ends in an incomplete line in a concise format.\n>>> \n>>> Should we add yet another column to the raw output to indicate who\n>>> is complete and who is incomplete?\n>>> \n>>> Where does it lead us and when will it stop?\n>>> \n>>> IOW, yuck ;-).\n>> \n>> My point being that it will be a huge mistake to do this only by\n>> singling a trait that is not so special as if it is very special,\n>> only because you have been thinking about it too long (the \"ends in\n>> an incomplete line\" trait is what has been on my mind for the past\n>> few days, \"this side is binary\" may be what you've been thinking\n>> about). There are many other things people would want to learn\n>> concisely in machine readable format, like \"where did the file stop\n>> using CRLF line endings and swithced to LF line endings\", that are\n>> equally plausible as the question you are asking, or the question I\n>> would be asking \"which commit lost the final newline?\"\n> \n> Completely fair. Having a bunch specific options for special info we\n> want to add to the raw diff format would get messy quickly and is not\n> very extensible.\n> \n>> Perhaps an extensible command line option syntax like\n>> \n>> $ git log --raw-extended=binary,incomplete,crlf,...\n> \n> I quite like this and agree it would be better to have a single\n> extensible option.\n> \n>> is in order, and the presense of these options would add \"tt,ic,cl\"\n>> somewhere in the output to signal that both sides are text, preimage\n>> ends in an incomplete line but not postimage, and preimage uses crlf\n>> but postimage uses lf, or something?\n> \n> Maybe the output should be something like:\n> \n> binary=tt,incomplete=ic,crlf=cl\n> \n> or something along those lines. That way we could freely extend in the\n> future without having to worry about a specific order. If we think all\n> of the raw diff extension modes would only report with yes/no for each\n> file we could just do:\n> \n> binary=yn,incomplete=yy,crlf=nn\n> \n> but maybe we should be more flexible and leave it up to the mode to\n> decide what its values can be?\n> \n> Also, maybe this info could be on a newline following each raw diff\n> entry? Something like:\n> \n> :100644 100644 a1961526 e231acb1 M foo\n> binary=yy\n> :100644 100644 31eedd5c 402a70d7 M bar\n> binary=nn\n> \n\nWhether combined or separate, self-documenting output is nice. Separate might be easier for line-oriented tools? Having to split on commas and loop looking for keywords seems like more work than just processing a line at a time. Idk."},{"id":"530342","messageId":"z5mkdl67vx47na5critwtyz7cacvhs6iixy46ia3svdoq7xui5@ckpfnjw3dkoq","threadId":"64430","inReplyTo":"xmqqtsz8nbwv.fsf@gitster.g","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-11-06T21:42:49Z","receivedAt":"2025-11-06T21:42:53Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/11/05 12:04AM, Junio C Hamano wrote:\n> Justin Tobler writes:\n> \n> > Maybe the output should be something like:\n> >\n> > binary=tt,incomplete=ic,crlf=cl\n> >\n> > or something along those lines. That way we could freely extend in the\n> > future without having to worry about a specific order. If we think all\n> > of the raw diff extension modes would only report with yes/no for each\n> > file we could just do:\n> >\n> > binary=yn,incomplete=yy,crlf=nn\n> >\n> > but maybe we should be more flexible and leave it up to the mode to\n> > decide what its values can be?\n> >\n> > Also, maybe this info could be on a newline following each raw diff\n> > entry? Something like:\n> >\n> > :100644 100644 a1961526 e231acb1 M\tfoo\n> > binary=yy\n> > :100644 100644 31eedd5c 402a70d7 M\tbar\n> > binary=nn\n> \n> I know these are parse-able, but quite honestly, both sounds\n> somewhat backwards, if you meant to make this easier to parse by\n> simple scripts. Scripts do not mind their input line wider than 80\n> columns, but it is cumbersome if they have to take each pair of\n> lines and combine them to process. And repeated keywords like\n> binary= etc., do not look like it is less error prone for scripts to\n> parse them out, either. So, I dunno.\n\nPersonally, while keywords like \"binary=\" add a bit of complexity to the\noutput, I do like the idea of having the output be self-documenting that\nway the parsers can avoid being aware of the inputted arguments or some\npredetermined output order. I do agree though that spreading the output\nacross multiple lines doesn't really help us much as it probably doesn't\nmatter whether we split on a comma or a newline. It's probably simpler\njust to keep it all on a single line.\n\nCurrently the output in the next version will look like:\n\n :100644 100644 a1961526 e231acb1 binary=yy M\tfoo\n :100644 100644 31eedd5c 402a70d7 binary=nn M\tbar\n\nThanks,\n-Justin\n"},{"id":"530343","messageId":"id5zny6t4cr5eaow4ttvdmaksywazvgraq56m2x3wviw3rij7v@fhjfhva5favl","threadId":"64430","inReplyTo":"D80AE9D4-EE8D-4CAE-9212-2A592F45AB90@gmail.com","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-11-06T21:52:02Z","receivedAt":"2025-11-06T21:52:03Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/11/05 07:14AM, Ben Knoble wrote:\n> \n> > Le 4 nov. 2025 à 19:17, Justin Tobler a écrit :\n> > Also, maybe this info could be on a newline following each raw diff\n> > entry? Something like:\n> > \n> > :100644 100644 a1961526 e231acb1 M foo\n> > binary=yy\n> > :100644 100644 31eedd5c 402a70d7 M bar\n> > binary=nn\n> > \n> \n> Whether combined or separate, self-documenting output is nice. Separate might be easier for line-oriented tools? Having to split on commas and loop looking for keywords seems like more work than just processing a line at a time. Idk.\n\nYa, I'm also a bit torn on whether a single line or multiple lines would\nbe best. I'm currently leaning back towards using just a single line as\nI do think it is somewhat nice that each diff entry itself is on its own\nline. Ultimately from the perspective of the parser, I don't think it\nshould matter too much whether its splits raw-extended output on a\ncommand or newline.\n\nIn the next version, I'm thinking the output will look something like:\n\n $ git diff-tree --raw-extended=binary,crlf ...\n :100644 100644 a1961526 e231acb1 binary=yy,crlf=nn M foo\n :100644 100644 31eedd5c 402a70d7 binary=nn,crlf=yy M bar\n\nThanks,\n-Justin\n"},{"id":"530359","messageId":"20251107083015.GA18670@tb-raspi4","threadId":"64430","inReplyTo":"z5mkdl67vx47na5critwtyz7cacvhs6iixy46ia3svdoq7xui5@ckpfnjw3dkoq","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2025-11-07T08:30:15Z","receivedAt":"2025-11-07T08:30:20Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Thu, Nov 06, 2025 at 03:42:49PM -0600, Justin Tobler wrote:\n> On 25/11/05 12:04AM, Junio C Hamano wrote:\n> > Justin Tobler writes:\n> > \n[snip]\n> Currently the output in the next version will look like:\n> \n> :100644 100644 a1961526 e231acb1 binary=yy M\tfoo\n> :100644 100644 31eedd5c 402a70d7 binary=nn M\tbar\n>\n\nI think that is a good solutution ;-)\nWhen I once developped the\ngit ls-files --eol option someone (Junio ?) convinced my to\nuse a TAB as a seperator.\nIn this case just before the filename:\n\ngit ls-file --eol | xxd \n00000000: 692f 6c66 2020 2020 772f 6c66 2020 2020 i/lf w/lf \n00000010: 6174 7472 2f20 2020 2020 2020 2020 2020 attr/ \n00000020: 2020 2020 2020 092e 6369 7272 7573 2e79 ..cirrus.y\n ^^\n00000030: 6d6c 0a ml.\n\nThis makes the output both human readable and machine parsable:\nAll info is before the TAB here. (And may be parsed again in a second\nround, if needed).\nThoughts ?\n"},{"id":"530383","messageId":"xmqq7bw1hllz.fsf@gitster.g","threadId":"64430","inReplyTo":"20251107083015.GA18670@tb-raspi4","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-11-07T16:07:52Z","receivedAt":"2025-11-07T16:07:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen writes:\n\n> git ls-files --eol option someone (Junio ?) convinced my to\n> use a TAB as a seperator.\n> In this case just before the filename:\n>\n> git ls-file --eol | xxd \n> 00000000: 692f 6c66 2020 2020 772f 6c66 2020 2020 i/lf w/lf \n> 00000010: 6174 7472 2f20 2020 2020 2020 2020 2020 attr/ \n> 00000020: 2020 2020 2020 092e 6369 7272 7573 2e79 ..cirrus.y\n> ^^\n> 00000030: 6d6c 0a ml.\n>\n> This makes the output both human readable and machine parsable:\n> All info is before the TAB here. (And may be parsed again in a second\n> round, if needed).\n> Thoughts ?\n\nThis brings up another interesting question: which command should\nlearn these new classifications.\n\nThe original desire \"when I diff A and B, I cannot tell which one of\nA or B had binary when I see 'binary file differs'\" almost suggests\nto me that 'diff' is a wrong place and rather they wanted to know \"I\nhave A; now who is binary in there?\" Or \"when I diff A and B with\npathspec P, I cannot tell...\" is probably a wrong question to ask,\nand the question may be \"I have A; now who is binary in that tree\nwithin pathspec P?\"\n\nIOW, \"git show\" or \"git log\", when showing a commit C in the\nhistory, would give \"that one is a binary\" information as if it is\nan attribute of the change between commit C and its parents, if you\ntuck this new logic into the \"diff\" machinery. I am not sure if\nthat is what we really want. If we do so in \"ls-tree\" and allow\n\"git log\" to show characteristics of each tree it encounters while\ntraversing the history, on the other hand, \"that one is a binary\"\nwould truly become an attribute of an entry in a tree, and it is not\naffected by what is in the trees of the commits that are adjacent to\nthe commit in the history.\n"},{"id":"530386","messageId":"ikzwvvyyhuhvr7picunl3r4zem4cn566zpjxpmh6u4oq6ncswa@cdfokimobtms","threadId":"64430","inReplyTo":"20251107083015.GA18670@tb-raspi4","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2025-11-07T17:16:35Z","receivedAt":"2025-11-07T17:16:40Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 25/11/07 09:30AM, Torsten Bögershausen wrote:\n> On Thu, Nov 06, 2025 at 03:42:49PM -0600, Justin Tobler wrote:\n> > On 25/11/05 12:04AM, Junio C Hamano wrote:\n> > > Justin Tobler writes:\n> > Currently the output in the next version will look like:\n> > \n> > :100644 100644 a1961526 e231acb1 binary=yy M\tfoo\n> > :100644 100644 31eedd5c 402a70d7 binary=nn M\tbar\n> >\n> \n> I think that is a good solutution ;-)\n> When I once developped the\n> git ls-files --eol option someone (Junio ?) convinced my to\n> use a TAB as a seperator.\n> In this case just before the filename:\n> \n> git ls-file --eol | xxd \n> 00000000: 692f 6c66 2020 2020 772f 6c66 2020 2020 i/lf w/lf \n> 00000010: 6174 7472 2f20 2020 2020 2020 2020 2020 attr/ \n> 00000020: 2020 2020 2020 092e 6369 7272 7573 2e79 ..cirrus.y\n> ^^\n> 00000030: 6d6c 0a ml.\n> \n> This makes the output both human readable and machine parsable:\n> All info is before the TAB here. (And may be parsed again in a second\n> round, if needed).\n> Thoughts ?\n\nSo the raw diff format for a normal diff pair is as follows:\n\n :SPSPSPSP[score]TAB[TAB]LF\n\nWhen the `-z` option is used, tab and LF are replaced with a NUL byte.\nSo we do already use a tab to delimit between the score/paths. If we\nwanted to drop avoid using comma to delimit between extended raw diff\noutput we could use a space instead and use TAB to indicate the end.\nMaybe something like:\n\n :100644 100644 a1961526 e231acb1 binary=yy crlf=nn M foo\n :100644 100644 31eedd5c 402a70d7 binary=nn crlf=yy M bar\n\nor we could maybe move the extended info towards the start of the line\nand leave the remaining bits the same:\n\n :binary=yy crlf=nn 100644 100644 a1961526 e231acb1 M foo\n :binary=nn crlf=yy 100644 100644 31eedd5c 402a70d7 M bar\n\nWith either of these formats, the expectation would be parsers continue\nreading space delimited \"key=value\" pairs until they encounter a tab. I\ndo think this latter format looks a bit nicer and I don't think it would\nmeaningfully impact the complexity of the parser. Ultimately, I don't\nfeel super strongly one way or the other though. I may go with this last\nformat in the next version since it does look a little nicer IMO. I'm\nstill very much interested in folks thoughts here though. :)\n\nThanks,\n-Justin\n"},{"id":"530387","messageId":"xmqqjz01g3du.fsf@gitster.g","threadId":"64430","inReplyTo":"ikzwvvyyhuhvr7picunl3r4zem4cn566zpjxpmh6u4oq6ncswa@cdfokimobtms","subject":"Re: [RFC PATCH] diff: add option to report binary files in raw diffs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-11-07T17:26:53Z","receivedAt":"2025-11-07T17:26:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Justin Tobler writes:\n\n> or we could maybe move the extended info towards the start of the line\n> and leave the remaining bits the same:\n>\n> :binary=yy crlf=nn 100644 100644 a1961526 e231acb1 M foo\n> :binary=nn crlf=yy 100644 100644 31eedd5c 402a70d7 M bar\n>\n> With either of these formats, the expectation would be parsers continue\n> reading space delimited \"key=value\" pairs until they encounter a tab. I\n> do think this latter format looks a bit nicer and I don't think it would\n> meaningfully impact the complexity of the parser. Ultimately, I don't\n> feel super strongly one way or the other though. I may go with this last\n> format in the next version since it does look a little nicer IMO. I'm\n> still very much interested in folks thoughts here though. :)\n\nWith this are your parsers/readers still using the output fields\nthat appear in the --raw output? Do they still want the mode bits,\nor object names in preimage and postimage? Do they need to even\nlook at \"M\" anymore, as a new file or a removed file would certainly\nhave only a single sign for these additional traits like binary as\nsuch a filepair has only one side by definition?\n\nIOW, I am not sure if it is wise to shoehorn the new pieces of\ninformation into the --raw format. Existing parsers would not be\nable to grok the above at all (they do not even see the fields they\nrecognise at the beginning of lines which is where they recognise\nthem as such), so I do not see any good reason to even pretend this\nto be some extension to an existing --raw format.\n\n"}]}