git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 04/20] merge-ort: use histogram diff

From
Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>
Date
Dec 13, 2020, 08:04 UTC
Message-ID
<38b4f9cf78c885e958158a3960dd74715dc22d97.1607846667.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.923.v3.git.git.1607846667.gitgitgadget@gmail.com>
From: Elijah Newren <newren@gmail.com>

In my cursory investigation, histogram diffs are about 2% slower than Myers diffs. Others have probably done more detailed benchmarks. But, in short, histogram diffs have been around for years and in a number of cases provide obviously better looking diffs where Myers diffs are unintelligible but the performance hit has kept them from becoming the default.

However, there are real merge bugs we know about that have triggered on git.git and linux.git, which I don't have a clue how to address without the additional information that I believe is provided by histogram diffs. See the following:

https://lore.kernel.org/git/20190816184051.GB13894@sigill.intra.peff.net/ https://lore.kernel.org/git/CABPp-BHvJHpSJT7sdFwfNcPn_sOXwJi3=o14qjZS3M8Rzcxe2A@mail.gmail.com/ https://lore.kernel.org/git/CABPp-BGtez4qjbtFT1hQoREfcJPmk9MzjhY5eEq1QhXT23tFOw@mail.gmail.com/

I don't like mismerges. I really don't like silent mismerges. While I am sometimes willing to make performance and correctness tradeoff, I'm much more interested in correctness in general. I want to fix the above bugs. I have not yet started doing so, but I believe histogram diff at least gives me an angle. Unfortunately, I can't rely on using the information from histogram diff unless it's in use. And it hasn't been used because of a few percentage performance hit.

In testcases I have looked at, merge-ort is _much_ faster than merge-recursive for non-trivial merges/rebases/cherry-picks. As such, this is a golden opportunity to switch out the underlying diff algorithm (at least the one used by the merge machinery; git-diff and git-log are separate questions); doing so will allow me to get additional data and improved diffs, and I believe it will help me fix the above bugs at some point in the future.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 4 ++++
 1 file changed, 4 insertions(+)
diff --git a/merge-ort.c b/merge-ort.c
index fb07c8f2b30..85942cfa7c7 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -21,6 +21,7 @@
 #include "diffcore.h"
 #include "strmap.h"
 #include "tree.h"
+#include "xdiff-interface.h"
 
 struct merge_options_internal {
 	/*
@@ -245,6 +246,9 @@ static void merge_start(struct merge_options *opt, struct merge_result *result)
 
 	assert(opt->priv == NULL);
 
+	/* Default to histogram diff.  Actually, just hardcode it...for now. */
+	opt->xdl_opts = DIFF_WITH_ALG(opt, HISTOGRAM_DIFF);
+
 	/* Initialization of opt->priv, our internal merge data */
 	opt->priv = xcalloc(1, sizeof(*opt->priv));
 
-- 
gitgitgadget
Previous: Elijah Newren via GitGitGadgetNext: Elijah Newren via GitGitGadget
Message 50 of 70 in “fundamentals of merge-ort implementation”
  1. 00/20 fundamentals of merge-ort implementationElijah Newren via GitGitGadget, Nov 29, 2020
  2. 03/20 merge-ort: port merge_start() from merge-recursiveElijah Newren via GitGitGadget, Nov 29, 2020
  3. 02/20 merge-ort: add some high-level algorithm structureElijah Newren via GitGitGadget, Nov 29, 2020
  4. 04/20 merge-ort: use histogram diffElijah Newren via GitGitGadget, Nov 29, 2020
  5. 01/20 merge-ort: setup basic internal data structuresElijah Newren via GitGitGadget, Nov 29, 2020
  6. 07/20 merge-ort: avoid repeating fill_tree_descriptor() on the same treeElijah Newren via GitGitGadget, Nov 29, 2020
  7. 05/20 merge-ort: add an err() function similar to one from merge-recursiveElijah Newren via GitGitGadget, Nov 29, 2020
  8. Ævar Arnfjörð BjarmasonNov 29, 2020
  9. Elijah NewrenNov 30, 2020
  10. Ævar Arnfjörð BjarmasonNov 29, 2020
  11. 10/20 merge-ort: avoid recursing into identical treesElijah Newren via GitGitGadget, Nov 29, 2020
  12. 08/20 merge-ort: compute a few more useful fields for collect_merge_infoElijah Newren via GitGitGadget, Nov 29, 2020
  13. 06/20 merge-ort: implement a very basic collect_merge_info()Elijah Newren via GitGitGadget, Nov 29, 2020
  14. 11/20 merge-ort: add a preliminary simple process_entries() implementationElijah Newren via GitGitGadget, Nov 29, 2020
  15. 12/20 merge-ort: have process_entries operate in a defined orderElijah Newren via GitGitGadget, Nov 29, 2020
  16. 14/20 merge-ort: step 2 of tree writing -- function to create tree objectElijah Newren via GitGitGadget, Nov 29, 2020
  17. 09/20 merge-ort: record stage and auxiliary info for every pathElijah Newren via GitGitGadget, Nov 29, 2020
  18. 15/20 merge-ort: step 3 of tree writing -- handling subdirectories as we goElijah Newren via GitGitGadget, Nov 29, 2020
  19. 16/20 merge-ort: basic outline for merge_switch_to_result()Elijah Newren via GitGitGadget, Nov 29, 2020
  20. 13/20 merge-ort: step 1 of tree writing -- record basenames, modes, and oidsElijah Newren via GitGitGadget, Nov 29, 2020
  21. 19/20 merge-ort: add implementation of record_conflicted_index_entries()Elijah Newren via GitGitGadget, Nov 29, 2020
  22. Ævar Arnfjörð BjarmasonNov 29, 2020
  23. 18/20 tree: enable cmp_cache_name_compare() to be used elsewhereElijah Newren via GitGitGadget, Nov 29, 2020
  24. 17/20 merge-ort: add implementation of checkout()Elijah Newren via GitGitGadget, Nov 29, 2020
  25. 20/20 merge-ort: free data structures in merge_finalize()Elijah Newren via GitGitGadget, Nov 29, 2020
  26. Elijah NewrenNov 29, 2020
  27. 00/20 fundamentals of merge-ort implementationElijah Newren via GitGitGadget, Dec 4, 2020
  28. 02/20 merge-ort: add some high-level algorithm structureElijah Newren via GitGitGadget, Dec 4, 2020
  29. 03/20 merge-ort: port merge_start() from merge-recursiveElijah Newren via GitGitGadget, Dec 4, 2020
  30. 01/20 merge-ort: setup basic internal data structuresElijah Newren via GitGitGadget, Dec 4, 2020
  31. 05/20 merge-ort: add an err() function similar to one from merge-recursiveElijah Newren via GitGitGadget, Dec 4, 2020
  32. 04/20 merge-ort: use histogram diffElijah Newren via GitGitGadget, Dec 4, 2020
  33. 15/20 merge-ort: step 3 of tree writing -- handling subdirectories as we goElijah Newren via GitGitGadget, Dec 4, 2020
  34. 14/20 merge-ort: step 2 of tree writing -- function to create tree objectElijah Newren via GitGitGadget, Dec 4, 2020
  35. 07/20 merge-ort: avoid repeating fill_tree_descriptor() on the same treeElijah Newren via GitGitGadget, Dec 4, 2020
  36. 09/20 merge-ort: record stage and auxiliary info for every pathElijah Newren via GitGitGadget, Dec 4, 2020
  37. 11/20 merge-ort: add a preliminary simple process_entries() implementationElijah Newren via GitGitGadget, Dec 4, 2020
  38. 10/20 merge-ort: avoid recursing into identical treesElijah Newren via GitGitGadget, Dec 4, 2020
  39. 08/20 merge-ort: compute a few more useful fields for collect_merge_infoElijah Newren via GitGitGadget, Dec 4, 2020
  40. 19/20 merge-ort: add implementation of record_conflicted_index_entries()Elijah Newren via GitGitGadget, Dec 4, 2020
  41. 18/20 tree: enable cmp_cache_name_compare() to be used elsewhereElijah Newren via GitGitGadget, Dec 4, 2020
  42. 12/20 merge-ort: have process_entries operate in a defined orderElijah Newren via GitGitGadget, Dec 4, 2020
  43. 13/20 merge-ort: step 1 of tree writing -- record basenames, modes, and oidsElijah Newren via GitGitGadget, Dec 4, 2020
  44. 06/20 merge-ort: implement a very basic collect_merge_info()Elijah Newren via GitGitGadget, Dec 4, 2020
  45. 17/20 merge-ort: add implementation of checkout()Elijah Newren via GitGitGadget, Dec 4, 2020
  46. 16/20 merge-ort: basic outline for merge_switch_to_result()Elijah Newren via GitGitGadget, Dec 4, 2020
  47. 20/20 merge-ort: free data structures in merge_finalize()Elijah Newren via GitGitGadget, Dec 4, 2020
  48. 00/20 fundamentals of merge-ort implementationElijah Newren via GitGitGadget, Dec 13, 2020
  49. 01/20 merge-ort: setup basic internal data structuresElijah Newren via GitGitGadget, Dec 13, 2020
  50. 04/20 merge-ort: use histogram diffElijah Newren via GitGitGadget, Dec 13, 2020
  51. 03/20 merge-ort: port merge_start() from merge-recursiveElijah Newren via GitGitGadget, Dec 13, 2020
  52. 02/20 merge-ort: add some high-level algorithm structureElijah Newren via GitGitGadget, Dec 13, 2020
  53. 08/20 merge-ort: compute a few more useful fields for collect_merge_infoElijah Newren via GitGitGadget, Dec 13, 2020
  54. 06/20 merge-ort: implement a very basic collect_merge_info()Elijah Newren via GitGitGadget, Dec 13, 2020
  55. 07/20 merge-ort: avoid repeating fill_tree_descriptor() on the same treeElijah Newren via GitGitGadget, Dec 13, 2020
  56. 05/20 merge-ort: add an err() function similar to one from merge-recursiveElijah Newren via GitGitGadget, Dec 13, 2020
  57. 10/20 merge-ort: avoid recursing into identical treesElijah Newren via GitGitGadget, Dec 13, 2020
  58. 14/20 merge-ort: step 2 of tree writing -- function to create tree objectElijah Newren via GitGitGadget, Dec 13, 2020
  59. 12/20 merge-ort: have process_entries operate in a defined orderElijah Newren via GitGitGadget, Dec 13, 2020
  60. 16/20 merge-ort: basic outline for merge_switch_to_result()Elijah Newren via GitGitGadget, Dec 13, 2020
  61. 09/20 merge-ort: record stage and auxiliary info for every pathElijah Newren via GitGitGadget, Dec 13, 2020
  62. 18/20 tree: enable cmp_cache_name_compare() to be used elsewhereElijah Newren via GitGitGadget, Dec 13, 2020
  63. 15/20 merge-ort: step 3 of tree writing -- handling subdirectories as we goElijah Newren via GitGitGadget, Dec 13, 2020
  64. 17/20 merge-ort: add implementation of checkout()Elijah Newren via GitGitGadget, Dec 13, 2020
  65. 19/20 merge-ort: add implementation of record_conflicted_index_entries()Elijah Newren via GitGitGadget, Dec 13, 2020
  66. 20/20 merge-ort: free data structures in merge_finalize()Elijah Newren via GitGitGadget, Dec 13, 2020
  67. 13/20 merge-ort: step 1 of tree writing -- record basenames, modes, and oidsElijah Newren via GitGitGadget, Dec 13, 2020
  68. 11/20 merge-ort: add a preliminary simple process_entries() implementationElijah Newren via GitGitGadget, Dec 13, 2020
  69. Felipe ContrerasDec 14, 2020
  70. Elijah NewrenDec 14, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.