git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 3/7] builtin: patch-id: fix patch-id with binary diffs

From
Jerry Zhang via GitGitGadget <gitgitgadget@gmail.com>
Date
Oct 14, 2022, 08:56 UTC
Message-ID
<2164212892712930cb34223499bb3e03bf2c2392.1665737804.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1359.v3.git.1665737804.gitgitgadget@gmail.com>
From: Jerry Zhang <Jerry@skydio.com>

"git patch-id" currently doesn't produce correct output if the incoming diff has any binary files. Add logic to get_one_patchid to handle the different possible styles of binary diff. This attempts to keep resulting patch-ids identical to what would be produced by the counterpart logic in diff.c, that is it produces the id by hashing the a and b oids in succession.

In general we handle binary diffs by first caching the object ids from the "index" line and using those if we then find an indication that the diff is binary.

The input could contain patches generated with "git diff --binary". This currently breaks the parse logic and results in multiple patch-ids output for a single commit. Here we have to skip the contents of the patch itself since those do not go into the patch id. --binary implies --full-index so the object ids are always available.

When the diff is generated with --full-index there is no patch content to skip over.

When a diff is generated without --full-index or --binary, it will contain abbreviated object ids. This will still result in a sufficiently unique patch-id when hashed, but does not match internal patch id output. We'll call this ok for now as we already need specialized arguments to diff in order to match internal patch id (namely -U3).

Signed-off-by: Jerry Zhang <Jerry@skydio.com>
---
 builtin/patch-id.c  | 36 ++++++++++++++++++++++++++++++++++--
 t/t4204-patch-id.sh | 29 ++++++++++++++++++++++++++++-
 2 files changed, 62 insertions(+), 3 deletions(-)
diff --git a/builtin/patch-id.c b/builtin/patch-id.c
index 881fcf32732..e7a31123142 100644
--- a/builtin/patch-id.c
+++ b/builtin/patch-id.c
@@ -61,6 +61,8 @@ static int get_one_patchid(struct object_id *next_oid, struct object_id *result,
 {
 	int patchlen = 0, found_next = 0;
 	int before = -1, after = -1;
+	int diff_is_binary = 0;
+	char pre_oid_str[GIT_MAX_HEXSZ + 1], post_oid_str[GIT_MAX_HEXSZ + 1];
 	git_hash_ctx ctx;
 
 	the_hash_algo->init_fn(&ctx);
@@ -88,14 +90,44 @@ static int get_one_patchid(struct object_id *next_oid, struct object_id *result,
 
 		/* Parsing diff header?  */
 		if (before == -1) {
-			if (starts_with(line, "index "))
+			if (starts_with(line, "GIT binary patch") ||
+			    starts_with(line, "Binary files")) {
+				diff_is_binary = 1;
+				before = 0;
+				the_hash_algo->update_fn(&ctx, pre_oid_str,
+							 strlen(pre_oid_str));
+				the_hash_algo->update_fn(&ctx, post_oid_str,
+							 strlen(post_oid_str));
+				if (stable)
+					flush_one_hunk(result, &ctx);
 				continue;
-			else if (starts_with(line, "--- "))
+			} else if (skip_prefix(line, "index ", &p)) {
+				char *oid1_end = strstr(line, "..");
+				char *oid2_end = NULL;
+				if (oid1_end)
+					oid2_end = strstr(oid1_end, " ");
+				if (!oid2_end)
+					oid2_end = line + strlen(line) - 1;
+				if (oid1_end != NULL && oid2_end != NULL) {
+					*oid1_end = *oid2_end = '\0';
+					strlcpy(pre_oid_str, p, GIT_MAX_HEXSZ + 1);
+					strlcpy(post_oid_str, oid1_end + 2, GIT_MAX_HEXSZ + 1);
+				}
+				continue;
+			} else if (starts_with(line, "--- "))
 				before = after = 1;
 			else if (!isalpha(line[0]))
 				break;
 		}
 
+		if (diff_is_binary) {
+			if (starts_with(line, "diff ")) {
+				diff_is_binary = 0;
+				before = -1;
+			}
+			continue;
+		}
+
 		/* Looking for a valid hunk header?  */
 		if (before == 0 && after == 0) {
 			if (starts_with(line, "@@ -")) {
diff --git a/t/t4204-patch-id.sh b/t/t4204-patch-id.sh
index a730c0db985..cdc5191aa8d 100755
--- a/t/t4204-patch-id.sh
+++ b/t/t4204-patch-id.sh
@@ -42,7 +42,7 @@ calc_patch_id () {
 }
 
 get_top_diff () {
-	git log -p -1 "$@" -O bar-then-foo --
+	git log -p -1 "$@" -O bar-then-foo --full-index --
 }
 
 get_patch_id () {
@@ -61,6 +61,33 @@ test_expect_success 'patch-id detects inequality' '
 	get_patch_id notsame &&
 	! test_cmp patch-id_main patch-id_notsame
 '
+test_expect_success 'patch-id detects equality binary' '
+	cat >.gitattributes <<-\EOF &&
+	foo binary
+	bar binary
+	EOF
+	get_patch_id main &&
+	get_patch_id same &&
+	git log -p -1 --binary main >top-diff.output &&
+	calc_patch_id <top-diff.output main_binpatch &&
+	git log -p -1 --binary same >top-diff.output &&
+	calc_patch_id <top-diff.output same_binpatch &&
+	test_cmp patch-id_main patch-id_main_binpatch &&
+	test_cmp patch-id_same patch-id_same_binpatch &&
+	test_cmp patch-id_main patch-id_same &&
+	test_when_finished "rm .gitattributes"
+'
+
+test_expect_success 'patch-id detects inequality binary' '
+	cat >.gitattributes <<-\EOF &&
+	foo binary
+	bar binary
+	EOF
+	get_patch_id main &&
+	get_patch_id notsame &&
+	! test_cmp patch-id_main patch-id_notsame &&
+	test_when_finished "rm .gitattributes"
+'
 
 test_expect_success 'patch-id supports git-format-patch output' '
 	get_patch_id main &&
-- 
gitgitgadget
Previous: Jerry Zhang via GitGitGadgetNext: Junio C Hamano
Message 10 of 46 in “update internal patch-id to use "stable" algorithm”
  1. 0/2 update internal patch-id to use "stable" algorithmJerry Zhang via GitGitGadget, Sep 20, 2022
  2. 2/2 patch-id: use stable patch-id for rebasesJerry Zhang via GitGitGadget, Sep 20, 2022
  3. 1/2 patch-id: fix stable patch id for binary / header-onlyJerry Zhang via GitGitGadget, Sep 20, 2022
  4. 0/2 update internal patch-id to use "stable" algorithmJerry Zhang via GitGitGadget, Sep 20, 2022
  5. 1/2 patch-id: fix stable patch id for binary / header-onlyJerry Zhang via GitGitGadget, Sep 20, 2022
  6. 2/2 patch-id: use stable patch-id for rebasesJerry Zhang via GitGitGadget, Sep 20, 2022
  7. 0/7 patch-id fixes and improvementsJerry Zhang via GitGitGadget, Oct 14, 2022
  8. 1/7 patch-id: fix stable patch id for binary / header-onlyJerry Zhang via GitGitGadget, Oct 14, 2022
  9. 2/7 patch-id: use stable patch-id for rebasesJerry Zhang via GitGitGadget, Oct 14, 2022
  10. 3/7 builtin: patch-id: fix patch-id with binary diffsJerry Zhang via GitGitGadget, Oct 14, 2022
  11. Junio C HamanoOct 14, 2022
  12. Jerry ZhangOct 14, 2022
  13. Junio C HamanoOct 14, 2022
  14. Jerry ZhangOct 14, 2022
  15. Junio C HamanoOct 17, 2022
  16. 7/7 documentation: format-patch: clarify requirements for patch-ids to matchJerry Zhang via GitGitGadget, Oct 14, 2022
  17. Junio C HamanoOct 17, 2022
  18. Jerry ZhangOct 18, 2022
  19. Junio C HamanoOct 19, 2022
  20. 4/7 patch-id: fix patch-id for mode changesJerry Zhang via GitGitGadget, Oct 14, 2022
  21. Junio C HamanoOct 14, 2022
  22. 6/7 builtin: patch-id: remove unused diff-tree prefixJerry Zhang via GitGitGadget, Oct 14, 2022
  23. Junio C HamanoOct 14, 2022
  24. 5/7 builtin: patch-id: add --include-whitespace as a command modeJerry Zhang via GitGitGadget, Oct 14, 2022
  25. Junio C HamanoOct 14, 2022
  26. Jerry ZhangOct 14, 2022
  27. Junio C HamanoOct 17, 2022
  28. Jerry ZhangOct 18, 2022
  29. 0/6 patch-id fixes and improvementsJerry Zhang via GitGitGadget, Oct 20, 2022
  30. 1/6 patch-id: fix stable patch id for binary / header-onlyJerry Zhang via GitGitGadget, Oct 20, 2022
  31. 2/6 patch-id: use stable patch-id for rebasesJerry Zhang via GitGitGadget, Oct 20, 2022
  32. 3/6 builtin: patch-id: fix patch-id with binary diffsJerry Zhang via GitGitGadget, Oct 20, 2022
  33. 4/6 patch-id: fix patch-id for mode changesJerry Zhang via GitGitGadget, Oct 20, 2022
  34. 5/6 builtin: patch-id: add --verbatim as a command modeJerry Zhang via GitGitGadget, Oct 20, 2022
  35. 6/6 builtin: patch-id: remove unused diff-tree prefixJerry Zhang via GitGitGadget, Oct 20, 2022
  36. Junio C HamanoOct 21, 2022
  37. 0/6 patch-id fixes and improvementsJerry Zhang via GitGitGadget, Oct 24, 2022
  38. 4/6 patch-id: fix patch-id for mode changesJerry Zhang via GitGitGadget, Oct 24, 2022
  39. 5/6 builtin: patch-id: add --verbatim as a command modeJerry Zhang via GitGitGadget, Oct 24, 2022
  40. 1/6 patch-id: fix stable patch id for binary / header-onlyJerry Zhang via GitGitGadget, Oct 24, 2022
  41. 2/6 patch-id: use stable patch-id for rebasesJerry Zhang via GitGitGadget, Oct 24, 2022
  42. 3/6 builtin: patch-id: fix patch-id with binary diffsJerry Zhang via GitGitGadget, Oct 24, 2022
  43. 6/6 builtin: patch-id: remove unused diff-tree prefixJerry Zhang via GitGitGadget, Oct 24, 2022
  44. Junio C HamanoOct 24, 2022
  45. Junio C HamanoSep 21, 2022
  46. Jerry ZhangSep 21, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.