git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 6/7] read-tree: make two-way merge sparse-aware

From
Victoria Dye via GitGitGadget <gitgitgadget@gmail.com>
Date
Feb 23, 2022, 18:25 UTC
Message-ID
<9fdcab038b2962b7f954363e32d04591476cf219.1645640717.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1157.git.1645640717.gitgitgadget@gmail.com>
From: Victoria Dye <vdye@github.com>

Enable two-way merge with 'git read-tree' without expanding the sparse index. When in a sparse index, a two-way merge will trivially succeed as long as there are not changes to the same sparse directory in multiple trees (i.e., sparse directory-level "edit-edit" conflicts). If there are such conflicts, the merge will fail despite the possibility that individual files could merge cleanly.

In order to resolve these "edit-edit" conflicts, "conflicted" sparse directories are - rather than rejected - merged by traversing their associated trees by OID. For each child of the sparse directory:

1. Files are merged as normal (see Documentation/git-read-tree.txt for
   details).
2. Subdirectories are treated as sparse directories and merged in
   'twoway_merge'. If there are no conflicts, they are merged according to
   the rules in Documentation/git-read-tree.txt; otherwise, the subdirectory
   is recursively traversed and merged.

This process allows sparse directories to be individually merged at the necessary depth *without* expanding a full index.

Signed-off-by: Victoria Dye <vdye@github.com>
---
 builtin/read-tree.c                      |  5 --
 t/t1092-sparse-checkout-compatibility.sh |  3 +-
 unpack-trees.c                           | 75 ++++++++++++++++++++++++
 3 files changed, 77 insertions(+), 6 deletions(-)
diff --git a/builtin/read-tree.c b/builtin/read-tree.c
index a7b7f822281..5a421de2629 100644
--- a/builtin/read-tree.c
+++ b/builtin/read-tree.c
@@ -225,11 +225,6 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)
 			opts.fn = opts.prefix ? bind_merge : oneway_merge;
 			break;
 		case 2:
-			/*
-			 * TODO: update twoway_merge to handle edit/edit conflicts in
-			 * sparse directories.
-			 */
-			ensure_full_index(&the_index);
 			opts.fn = twoway_merge;
 			opts.initial_checkout = is_cache_unborn();
 			break;
diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh
index a404be0a10f..d6f19682d65 100755
--- a/t/t1092-sparse-checkout-compatibility.sh
+++ b/t/t1092-sparse-checkout-compatibility.sh
@@ -1411,7 +1411,8 @@ test_expect_success 'sparse index is not expanded: read-tree' '
 	init_repos &&
 
 	ensure_not_expanded checkout -b test-branch update-folder1 &&
-	for MERGE_TREES in "update-folder2"
+	for MERGE_TREES in "update-folder2" \
+			   "base update-folder2"
 	do
 		ensure_not_expanded read-tree -mu $MERGE_TREES &&
 		ensure_not_expanded reset --hard HEAD || return 1
diff --git a/unpack-trees.c b/unpack-trees.c
index dba122a02bb..a4ace53904e 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1360,6 +1360,42 @@ static int is_sparse_directory_entry(struct cache_entry *ce,
 	return sparse_dir_matches_path(ce, info, name);
 }
 
+static int unpack_sparse_callback(int n, unsigned long mask, unsigned long dirmask, struct name_entry *names, struct traverse_info *info)
+{
+	struct cache_entry *src[MAX_UNPACK_TREES + 1] = { NULL, };
+	struct unpack_trees_options *o = info->data;
+	int ret;
+
+	assert(o->merge);
+
+	/*
+	 * Unlike in 'unpack_callback', where src[0] is derived from the index when
+	 * merging, src[0] is a transient cache entry derived from the first tree
+	 * provided. Create the temporary entry as if it came from a non-sparse index.
+	 */
+	if (!is_null_oid(&names[0].oid)) {
+		src[0] = create_ce_entry(info, &names[0], 0,
+					&o->result, 1,
+					dirmask & (1ul << 0));
+		src[0]->ce_flags |= (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);
+	}
+
+	/*
+	 * 'unpack_single_entry' assumes that src[0] is derived directly from
+	 * the index, rather than from an entry in 'names'. This is *not* true when
+	 * merging a sparse directory, in which case names[0] is the "index" source
+	 * entry. To match the expectations of 'unpack_single_entry', shift past the
+	 * "index" tree (i.e., names[0]) and adjust 'names', 'n', 'mask', and
+	 * 'dirmask' accordingly.
+	 */
+	ret = unpack_single_entry(n - 1, mask >> 1, dirmask >> 1, src, names + 1, info);
+
+	if (src[0])
+		discard_cache_entry(src[0]);
+
+	return ret >= 0 ? mask : -1;
+}
+
 /*
  * Note that traverse_by_cache_tree() duplicates some logic in this function
  * without actually calling it. If you change the logic here you may need to
@@ -2464,6 +2500,37 @@ static int merged_entry(const struct cache_entry *ce,
 	return 1;
 }
 
+static int merged_sparse_dir(const struct cache_entry * const *src, int n,
+			     struct unpack_trees_options *o)
+{
+	struct tree_desc t[MAX_UNPACK_TREES + 1];
+	void * tree_bufs[MAX_UNPACK_TREES + 1];
+	struct traverse_info info;
+	int i, ret;
+
+	/*
+	 * Create the tree traversal information for traversing into *only* the
+	 * sparse directory.
+	 */
+	setup_traverse_info(&info, src[0]->name);
+	info.fn = unpack_sparse_callback;
+	info.data = o;
+	info.show_all_errors = o->show_all_errors;
+	info.pathspec = o->pathspec;
+
+	/* Get the tree descriptors of the sparse directory in each of the merging trees */
+	for (i = 0; i < n; i++)
+		tree_bufs[i] = fill_tree_descriptor(o->src_index->repo, &t[i],
+						    src[i] && !is_null_oid(&src[i]->oid) ? &src[i]->oid : NULL);
+
+	ret = traverse_trees(o->src_index, n, t, &info);
+
+	for (i = 0; i < n; i++)
+		free(tree_bufs[i]);
+
+	return ret;
+}
+
 static int deleted_entry(const struct cache_entry *ce,
 			 const struct cache_entry *old,
 			 struct unpack_trees_options *o)
@@ -2734,6 +2801,14 @@ int twoway_merge(const struct cache_entry * const *src,
 			 * reject the merge instead.
 			 */
 			return merged_entry(newtree, current, o);
+		} else if (S_ISSPARSEDIR(current->ce_mode)) {
+			/*
+			 * The sparse directories differ, but we don't know whether that's
+			 * because of two different files in the directory being modified
+			 * (can be trivially merged) or if there is a real file conflict.
+			 * Merge the sparse directory by OID to compare file-by-file.
+			 */
+			return merged_sparse_dir(src, 3, o);
 		} else
 			return reject_merge(current, o);
 	}
-- 
gitgitgadget
Previous: Victoria Dye via GitGitGadgetNext: Elijah Newren
Message 9 of 48 in “Sparse index: integrate with 'read-tree'”
  1. 0/7 Sparse index: integrate with 'read-tree'Victoria Dye via GitGitGadget, Feb 23, 2022
  2. 1/7 sparse-index: prevent repo root from becoming sparseVictoria Dye via GitGitGadget, Feb 23, 2022
  3. Derrick StoleeFeb 24, 2022
  4. Victoria DyeFeb 24, 2022
  5. 2/7 status: fix nested sparse directory diff in sparse indexVictoria Dye via GitGitGadget, Feb 23, 2022
  6. 4/7 read-tree: integrate with sparse indexVictoria Dye via GitGitGadget, Feb 23, 2022
  7. 3/7 read-tree: expand sparse checkout test coverageVictoria Dye via GitGitGadget, Feb 23, 2022
  8. 5/7 read-tree: narrow scope of index expansion for '--prefix'Victoria Dye via GitGitGadget, Feb 23, 2022
  9. 6/7 read-tree: make two-way merge sparse-awareVictoria Dye via GitGitGadget, Feb 23, 2022
  10. Elijah NewrenFeb 26, 2022
  11. Victoria DyeFeb 28, 2022
  12. Elijah NewrenMar 1, 2022
  13. 7/7 read-tree: make three-way merge sparse-awareVictoria Dye via GitGitGadget, Feb 23, 2022
  14. Derrick StoleeFeb 24, 2022
  15. 0/7 Sparse index: integrate with 'read-tree'Victoria Dye via GitGitGadget, Feb 24, 2022
  16. 1/7 sparse-index: prevent repo root from becoming sparseVictoria Dye via GitGitGadget, Feb 24, 2022
  17. 2/7 status: fix nested sparse directory diff in sparse indexVictoria Dye via GitGitGadget, Feb 24, 2022
  18. Elijah NewrenFeb 25, 2022
  19. Victoria DyeFeb 28, 2022
  20. 5/7 read-tree: narrow scope of index expansion for '--prefix'Victoria Dye via GitGitGadget, Feb 24, 2022
  21. Elijah NewrenFeb 25, 2022
  22. Victoria DyeFeb 25, 2022
  23. Elijah NewrenFeb 26, 2022
  24. Victoria DyeFeb 28, 2022
  25. 3/7 read-tree: expand sparse checkout test coverageVictoria Dye via GitGitGadget, Feb 24, 2022
  26. Elijah NewrenFeb 26, 2022
  27. Victoria DyeFeb 28, 2022
  28. Ævar Arnfjörð BjarmasonFeb 28, 2022
  29. Victoria DyeFeb 28, 2022
  30. Ævar Arnfjörð BjarmasonFeb 28, 2022
  31. 7/7 read-tree: make three-way merge sparse-awareVictoria Dye via GitGitGadget, Feb 24, 2022
  32. 4/7 read-tree: integrate with sparse indexVictoria Dye via GitGitGadget, Feb 24, 2022
  33. 6/7 read-tree: make two-way merge sparse-awareVictoria Dye via GitGitGadget, Feb 24, 2022
  34. Elijah NewrenFeb 26, 2022
  35. 0/8 Sparse index: integrate with 'read-tree'Victoria Dye via GitGitGadget, Mar 1, 2022
  36. 1/8 sparse-index: prevent repo root from becoming sparseVictoria Dye via GitGitGadget, Mar 1, 2022
  37. 2/8 status: fix nested sparse directory diff in sparse indexVictoria Dye via GitGitGadget, Mar 1, 2022
  38. 3/8 read-tree: explicitly disallow prefixes with a leading '/'Victoria Dye via GitGitGadget, Mar 1, 2022
  39. 8/8 read-tree: make three-way merge sparse-awareVictoria Dye via GitGitGadget, Mar 1, 2022
  40. 6/8 read-tree: narrow scope of index expansion for '--prefix'Victoria Dye via GitGitGadget, Mar 1, 2022
  41. Glen ChooMar 3, 2022
  42. Victoria DyeMar 3, 2022
  43. Glen ChooMar 4, 2022
  44. 7/8 read-tree: make two-way merge sparse-awareVictoria Dye via GitGitGadget, Mar 1, 2022
  45. 5/8 read-tree: integrate with sparse indexVictoria Dye via GitGitGadget, Mar 1, 2022
  46. 4/8 read-tree: expand sparse checkout test coverageVictoria Dye via GitGitGadget, Mar 1, 2022
  47. Elijah NewrenMar 2, 2022
  48. Derrick StoleeMar 2, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.