threads / patch / 27574

patch, 48 partsHandling more corner cases in merge-recursive.c

Subject: [PATCH 00/48] Handling more corner cases in merge-recursive.c

## tl;dr

97 messages between Jun 8, 2011 and Aug 9, 2011. Diffs are folded; open one to read it.

replies: 96people: 4as markdown or json

Elijah Newren· Jun 8, 2011, 07:30 UTC · lore

Well, this took me a LOT longer than I expected, but I now have a (monster) patch series designed to fix various corner cases in merge-recursive.c. For those waiting for the 1.7.6 freeze to end, reading this patch series makes for some great mental gymnastics. :-)

You might be surprised by just how corner cases we have and the types of bugs that can be triggered (including deleting untracked files). This series includes testcases for the unnecessary file update issues[1] triggered by my series from last fall[2] and also a testcase that had been regressed by my earlier (and now reverted) bandaids designed to fix the unnecessary file update issue[3] (to make sure we fix both those issues). The current series aims to fix these problems using the strategy Junio outlined[4]. However, this series is about a lot more than just those issues.

[1] http://thread.gmane.org/gmane.comp.version-control.git/167800 [2] http://thread.gmane.org/gmane.comp.version-control.git/156559 [3] http://thread.gmane.org/gmane.comp.version-control.git/172617 [4] http://thread.gmane.org/gmane.comp.version-control.git/169209

You can note that I did not fix all the testcases I added. I consider some cases either unfixable or not worth fixing (see [5] for mailing list discussion about the main such case).

[5] http://thread.gmane.org/gmane.comp.version-control.git/157591

However, there is one large class of problems that I think is fixable, I'm just not sure whether it is worth fixing. git's rename detection optimization of only considering files that exist on one side of the diff but not the other causes issues with merges (undetected conflicts, spurious conflicts, or merged cleanly but wrongly due to deleting files that should be present). To fix these cases, we would need some way of including rewritten files as potential rename candidates, meaning we would also need to somehow check for rewritten files. (I refer to these as rename/add-source situations). The potential problem with applying such a fix is that it could make merges significantly more costly in the common case, in order to fix some relatively rare problems that might occur. I don't know whether it is justified, especially since git's merging can't be expected to know how to merge everything; at some point the user is responsible for checking the result. Patch 32 does add some comments to the code if anyone does decide they want to look into this, though.

Please test! I apologize for the earlier regressions I caused in merge-recursive. I've tried to be MUCH more careful this time, but the changes are numerous and much more invasive too. I'm trying to put this series through the testing wringer and will continue to do so, but extra testing from others would be great.

Anyway, if you've read this far...thanks!  
Elijah Newren (48):

In patches 1-12, I add a bunch of corner case tests. Since there are so many I want to add and I think putting them all in t6036 would be a bit unwieldy, I split the tests. Since t6036 currently only contains testcases that involve criss-cross merges, I mark it for such usage, and create t6039 for dealing with corner cases involving renames without criss-cross merges.

  [01] t6039: Add a testcase where git deletes an untracked file
  [02] t6039: Add failing testcase for rename/modify/add-source conflict
  [03] t6039: Add a pair of cases where undetected renames cause issues
  [04] t6039: Add a testcase where undetected rename causes silent file deletion
  [05] t6039: Add tests for content issues with modify/rename/directory conflicts
  [06] t6039: Add failing testcases for rename/rename/add-{source,dest} conflicts
  [07] t6039: Ensure rename/rename conflicts leave index and workdir in sane state
  [08] t6036: Add differently resolved modify/delete conflict in criss-cross test
  [09] t6036: criss-cross with weird content can fool git into clean merge
  [10] t6036: tests for criss-cross merges with various directory/file conflicts
  [11] t6036: criss-cross w/ rename/rename(1to2)/modify+rename/rename(2to1)/modify
  [12] t6036: criss-cross + rename/rename(1to2)/add-source + modify/modify

In patches 13-14, I add two tests that Junio reverted from master (6db41050) which we would still like fixed, and a test that my previous band-aids had broken to make sure we don't break it again.

  [13] t6022: Remove unnecessary untracked files to make test cleaner
  [14] t6022: New tests checking for unnecessary updates of files
  [15] t6022: Add testcase for merging a renamed file with a simple change

In patches 16-19, I'm just doing minor cleanups. None of these are critical; I'll be happy to drop any of them.

  [16] merge-recursive: Make BUG message more legible by adding a newline
  [17] merge-recursive: Correct a comment
  [18] merge-recursive: Mark some diff_filespec struct arguments const
  [19] merge-recursive: Remember to free generated unique path names

Patches 20-22 are small but important bug fixes. Some of these might make sense to be pulled to maint independent of the other changes.

  [20] merge-recursive: Avoid working directory changes during recursive case
  [21] merge-recursive: Fix recursive case with D/F conflict via add/add conflict
  [22] merge-recursive: Fix sorting order and directory change assumptions

Patches 23-29 are where I try to implement what I understood Junio wanted merge-recursive to do to avoid so many lstat() and other unnecessary working directory changes (see link [4] above).

  [23] merge-recursive: Fix code checking for D/F conflicts still being present
  [24] merge-recursive: Save D/F conflict filenames instead of unlinking them
  [25] merge-recursive: Split was_tracked() out of would_lose_untracked()
  [26] merge-recursive: Allow make_room_for_path() to remove D/F entries
  [27] merge-recursive: Consolidate different update_stages functions
  [28] merge-recursive: Split update_stages_and_entry; only update stages at end
  [29] merge-recursive: When we detect we can skip an update, actually skip it

Patch 30 fixes the deletion of an untracked file; I believe it has the highest ratio of commit message length to patch size, since it has to explain why we have modified a certain line of code several times and still didn't have it quite right.

  [30] merge-recursive: Fix deletion of untracked file in rename/delete conflicts

The commit message for patch 31 seems like there's some good zombie joke in there that someone cleverer than myself ought to be able to come up with. Interestingly, our code for rename/rename (2to1) conflicts was never executed, and would trigger segfaults had there been a way to execute it.

  [31] merge-recursive: Make dead code for rename/rename(2to1) conflicts undead

There are a few cases where git's rename detection optimization of only considering files that exist on one side of the diff but not the other causes issues with merges (undetected conflicts, spurious conflicts, or merged cleanly but wrongly due to deleting files that should be present). To fix these cases, we would need some way of including rewritten files as potential rename candidates, meaning we would also need to somehow check for rewritten files. My series does not address these cases (I'm unsure whether the performance hit would be worth it). Instead, I simply have a half-dozen relevant test cases and add some comments to the code about some things that would need to be considered if we ever want to tackle these cases.

  [32] merge-recursive: Add comments about handling rename/add-source cases

Patch 33 was specifically written because of a rename/directory conflict testcase, but tries to worry about three-way content merges in combination with renames and D/F conflicts and possible criss-cross merges and getting the appropriate index and working directory contents.

  [33] merge-recursive: Improve handling of rename target vs. directory addition

Patch 34 is an attempted simplification of merge-recursive.c. We really don't need process_df_entry(), since we can simply iterate through the unmerged entries in the order we need. That can make it easier to keep track of things, which is particularly important for the rename/rename fixes below...

  [34] merge-recursive: Consolidate process_entry() and process_df_entry()

Patches 35-48 try to fix our handling of rename/rename conflicts (both one file being renamed to two or two files being renamed to one). In such cases, we need to consider that the renames may not be the only change; there could also be modifications requiring a three-way content merge. Further, as with all other rename cases, we need to consider the recursive case (o->call_depth > 0) and potential D/F conflicts. There is also the possibility of rename/add-source issues for each side of the rename, though we currently can't detect those anyway. The rename_rename_1to2 case is especially interesting, as we also have to worry about each rename potentially also being involved in rename/delete and rename/add-dest conflicts. This all makes handling rename/rename conflicts a bit hairy.

  [35] merge-recursive: Cleanup and consolidation of rename_conflict_info
  [36] merge-recursive: Provide more info in conflict markers with file renames
  [37] merge-recursive: Fix modify/delete resolution in the recursive case
  [38] merge-recursive: Introduce a merge_file convenience function
  [39] merge-recursive: Fix rename/rename(1to2) resolution for virtual merge base
  [40] merge-recursive: Small cleanups for conflict_rename_rename_1to2
  [41] merge-recursive: Defer rename/rename(2to1) handling until process_entry
  [42] merge-recursive: Record more data needed for merging with dual renames
  [43] merge-recursive: Create function for merging with branchname:file markers
  [44] merge-recursive: Consider modifications in rename/rename(2to1) conflicts
  [45] merge-recursive: Make modify/delete handling code reusable
  [46] merge-recursive: Have conflict_rename_delete reuse modify/delete code
  [47] merge-recursive: add handling for rename/rename/add-dest/add-dest
  [48] merge-recursive: Fix working copy handling for rename/rename/add/add
 merge-recursive.c                    |  985 +++++++++++++++++++++-------------
 merge-recursive.h                    |    1 +
 t/t6020-merge-df.sh                  |   26 +-
 t/t6022-merge-rename.sh              |  102 ++++-
 t/t6036-recursive-corner-cases.sh    |  502 ++++++++++++++++-
 t/t6039-merge-rename-corner-cases.sh |  556 +++++++++++++++++++
 6 files changed, 1752 insertions(+), 420 deletions(-)
 create mode 100755 t/t6039-merge-rename-corner-cases.sh
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 01/48] t6039: Add a testcase where git deletes an untracked file

Current git will nuke an untracked file during a rename/delete conflict if (a) there is an untracked file whose name matches the source of a rename and (b) the merge is done in a certain direction. Add a simple testcase demonstrating this bug.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6036-recursive-corner-cases.sh    |    2 +-
 t/t6039-merge-rename-corner-cases.sh |   36 ++++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+), 1 deletions(-)
 create mode 100755 t/t6039-merge-rename-corner-cases.sh
Show changes to 2 files +37 −1

t/t6036-recursive-corner-cases.sh, t/t6039-merge-rename-corner-cases.sh

diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index 871577d..319b6fa 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -1,6 +1,6 @@
 #!/bin/sh
 
-test_description='recursive merge corner cases'
+test_description='recursive merge corner cases involving criss-cross merges'
 
 . ./test-lib.sh
 
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
new file mode 100755
index 0000000..5054459
--- /dev/null
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -0,0 +1,36 @@
+#!/bin/sh
+
+test_description="recursive merge corner cases w/ renames but not criss-crosses"
+# t6036 has corner cases that involve both criss-cross merges and renames
+
+. ./test-lib.sh
+
+test_expect_success 'setup rename/delete + untracked file' '
+	echo "A pretty inscription" >ring &&
+	git add ring &&
+	test_tick &&
+	git commit -m beginning &&
+
+	git branch people &&
+	git checkout -b rename-the-ring &&
+	git mv ring one-ring-to-rule-them-all &&
+	test_tick &&
+	git commit -m fullname &&
+
+	git checkout people &&
+	git rm ring &&
+	echo gollum >owner &&
+	git add owner &&
+	test_tick &&
+	git commit -m track-people-instead-of-objects &&
+	echo "Myyy PRECIOUSSS" >ring
+'
+
+test_expect_failure "Does git preserve Gollum's precious artifact?" '
+	test_must_fail git merge -s recursive rename-the-ring &&
+
+	# Make sure git did not delete an untracked file
+	test -f ring
+'
+
+test_done
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 02/48] t6039: Add failing testcase for rename/modify/add-source conflict

If there is a cleanly resolvable rename/modify conflict AND there is a new file introduced on the renamed side of the merge whose name happens to match that of the source of the rename (but is otherwise unrelated to the rename), then git fails to cleanly resolve the merge despite the fact that the new file should not cause any problems.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6039-merge-rename-corner-cases.sh |   39 ++++++++++++++++++++++++++++++++++
 1 files changed, 39 insertions(+), 0 deletions(-)
Show changes to t/t6039-merge-rename-corner-cases.sh +39 −0
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index 5054459..276d7dd 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -33,4 +33,43 @@ test_expect_failure "Does git preserve Gollum's precious artifact?" '
 	test -f ring
 '
 
+# Testcase setup for rename/modify/add-source:
+#   Commit A: new file: a
+#   Commit B: modify a slightly
+#   Commit C: rename a->b, add completely different a
+#
+# We should be able to merge B & C cleanly
+
+test_expect_success 'setup rename/modify/add-source conflict' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	printf "1\n2\n3\n4\n5\n6\n7\n" >a &&
+	git add a &&
+	git commit -m A &&
+	git tag A &&
+
+	git checkout -b B A &&
+	echo 8 >>a &&
+	git add a &&
+	git commit -m B &&
+
+	git checkout -b C A &&
+	git mv a b &&
+	echo something completely different >a &&
+	git add a &&
+	git commit -m C
+'
+
+test_expect_failure 'rename/modify/add-source conflict resolvable' '
+	git checkout B^0 &&
+
+	git merge -s recursive C^0 &&
+
+	test $(git rev-parse B:a) = $(git rev-parse b) &&
+	test $(git rev-parse C:a) = $(git rev-parse a)
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 03/48] t6039: Add a pair of cases where undetected renames cause issues

An undetected rename can cause a silent success where a conflict should have been detected, or can cause an erroneous conflict state where the merge should have been resolvable. Add testcases for both.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6039-merge-rename-corner-cases.sh |   61 ++++++++++++++++++++++++++++++++++
 1 files changed, 61 insertions(+), 0 deletions(-)
Show changes to t/t6039-merge-rename-corner-cases.sh +61 −0
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index 276d7dd..f338fb4 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -72,4 +72,65 @@ test_expect_failure 'rename/modify/add-source conflict resolvable' '
 	test $(git rev-parse C:a) = $(git rev-parse a)
 '
 
+test_expect_success 'setup resolvable conflict missed if rename missed' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	printf "1\n2\n3\n4\n5\n" >a &&
+	echo foo >b &&
+	git add a b &&
+	git commit -m A &&
+	git tag A &&
+
+	git checkout -b B A &&
+	git mv a c &&
+	echo "Completely different content" >a &&
+	git add a &&
+	git commit -m B &&
+
+	git checkout -b C A &&
+	echo 6 >>a &&
+	git add a &&
+	git commit -m C
+'
+
+test_expect_failure 'conflict caused if rename not detected' '
+	git checkout -q C^0 &&
+	git merge -s recursive B^0 &&
+
+	test 3 -eq $(git ls-files -s | wc -l) &&
+	test 0 -eq $(git ls-files -u | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l) &&
+
+	test 6 -eq $(wc -l < c) &&
+	test $(git rev-parse HEAD:a) = $(git rev-parse B:a) &&
+	test $(git rev-parse HEAD:b) = $(git rev-parse A:b)
+'
+
+test_expect_success 'setup conflict resolved wrong if rename missed' '
+	git reset --hard &&
+	git clean -f &&
+
+	git checkout -b D A &&
+	echo 7 >>a &&
+	git add a &&
+	git mv a c &&
+	echo "Completely different content" >a &&
+	git add a &&
+	git commit -m D &&
+
+	git checkout -b E A &&
+	git rm a &&
+	echo "Completely different content" >>a &&
+	git add a &&
+	git commit -m E
+'
+
+test_expect_failure 'missed conflict if rename not detected' '
+	git checkout -q E^0 &&
+	test_must_fail git merge -s recursive D^0
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 04/48] t6039: Add a testcase where undetected rename causes silent file deletion

There are cases where history should merge cleanly, and which current git does merge cleanly despite not detecting a rename; however the merge currently nukes files that should not be removed.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6039-merge-rename-corner-cases.sh |   65 ++++++++++++++++++++++++++++++++++
 1 files changed, 65 insertions(+), 0 deletions(-)
Show changes to t/t6039-merge-rename-corner-cases.sh +65 −0
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index f338fb4..db5560c 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -133,4 +133,69 @@ test_expect_failure 'missed conflict if rename not detected' '
 	test_must_fail git merge -s recursive D^0
 '
 
+# Tests for undetected rename/add-source causing a file to erroneously be
+# deleted (and for mishandled rename/rename(1to1) causing the same issue).
+#
+# This test uses a rename/rename(1to1)+add-source conflict (1to1 means the
+# same file is renamed on both sides to the same thing; it should trigger
+# the 1to2 logic, which it would do if the add-source didn't cause issues
+# for git's rename detection):
+#   Commit A: new file: a
+#   Commit B: rename a->b
+#   Commit C: rename a->b, add unrelated a
+
+test_expect_success 'setup undetected rename/add-source causes data loss' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	printf "1\n2\n3\n4\n5\n" >a &&
+	git add a &&
+	git commit -m A &&
+	git tag A &&
+
+	git checkout -b B A &&
+	git mv a b &&
+	git commit -m B &&
+
+	git checkout -b C A &&
+	git mv a b &&
+	echo foobar >a &&
+	git add a &&
+	git commit -m C
+'
+
+test_expect_failure 'detect rename/add-source and preserve all data' '
+	git checkout B^0 &&
+
+	git merge -s recursive C^0 &&
+
+	test 2 -eq $(git ls-files -s | wc -l) &&
+	test 2 -eq $(git ls-files -u | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l) &&
+
+	test -f a &&
+	test -f b &&
+
+	test $(git rev-parse HEAD:b) = $(git rev-parse A:a) &&
+	test $(git rev-parse HEAD:a) = $(git rev-parse C:a)
+'
+
+test_expect_failure 'detect rename/add-source and preserve all data, merge other way' '
+	git checkout C^0 &&
+
+	git merge -s recursive B^0 &&
+
+	test 2 -eq $(git ls-files -s | wc -l) &&
+	test 2 -eq $(git ls-files -u | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l) &&
+
+	test -f a &&
+	test -f b &&
+
+	test $(git rev-parse HEAD:b) = $(git rev-parse A:a) &&
+	test $(git rev-parse HEAD:a) = $(git rev-parse C:a)
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 05/48] t6039: Add tests for content issues with modify/rename/directory conflicts

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6039-merge-rename-corner-cases.sh |  141 ++++++++++++++++++++++++++++++++++
 1 files changed, 141 insertions(+), 0 deletions(-)
Show changes to t/t6039-merge-rename-corner-cases.sh +141 −0
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index db5560c..b465667 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -198,4 +198,145 @@ test_expect_failure 'detect rename/add-source and preserve all data, merge other
 	test $(git rev-parse HEAD:a) = $(git rev-parse C:a)
 '
 
+test_expect_success 'setup content merge + rename/directory conflict' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	printf "1\n2\n3\n4\n5\n6\n" >file &&
+	git add file &&
+	test_tick &&
+	git commit -m base &&
+	git tag base &&
+
+	git checkout -b right &&
+	echo 7 >>file &&
+	mkdir newfile &&
+	echo junk >newfile/realfile &&
+	git add file newfile/realfile &&
+	test_tick &&
+	git commit -m right &&
+
+	git checkout -b left-conflict base &&
+	echo 8 >>file &&
+	git add file &&
+	git mv file newfile &&
+	test_tick &&
+	git commit -m left &&
+
+	git checkout -b left-clean base &&
+	echo 0 >newfile &&
+	cat file >>newfile &&
+	git add newfile &&
+	git rm file &&
+	test_tick &&
+	git commit -m left
+'
+
+test_expect_failure 'rename/directory conflict + clean content merge' '
+	git reset --hard &&
+	git reset --hard &&
+	git clean -fdqx &&
+
+	git checkout left-clean^0 &&
+
+	test_must_fail git merge -s recursive right^0 &&
+
+	test 2 -eq $(git ls-files -s | wc -l) &&
+	test 1 -eq $(git ls-files -u | wc -l) &&
+	test 1 -eq $(git ls-files -o | wc -l) &&
+
+	echo 0 >expect &&
+	git cat-file -p base:file >>expect &&
+	echo 7 >>expect &&
+	test_cmp expect newfile~HEAD &&
+
+	test $(git rev-parse :2:newfile) = $(git hash-object expect) &&
+
+	test -f newfile/realfile &&
+	test -f newfile~HEAD
+'
+
+test_expect_failure 'rename/directory conflict + content merge conflict' '
+	git reset --hard &&
+	git reset --hard &&
+	git clean -fdqx &&
+
+	git checkout left-conflict^0 &&
+
+	test_must_fail git merge -s recursive right^0 &&
+
+	test 4 -eq $(git ls-files -s | wc -l) &&
+	test 3 -eq $(git ls-files -u | wc -l) &&
+	test 1 -eq $(git ls-files -o | wc -l) &&
+
+	git cat-file -p left-conflict:newfile >left &&
+	git cat-file -p base:file    >base &&
+	git cat-file -p right:file   >right &&
+	test_must_fail git merge-file \
+		-L "HEAD:newfile" \
+		-L "" \
+		-L "right^0:file" \
+		left base right &&
+	test_cmp left newfile~HEAD &&
+
+	test $(git rev-parse :1:newfile) = $(git rev-parse base:file) &&
+	test $(git rev-parse :2:newfile) = $(git rev-parse left-conflict:newfile) &&
+	test $(git rev-parse :3:newfile) = $(git rev-parse right:file) &&
+
+	test -f newfile/realfile &&
+	test -f newfile~HEAD
+'
+
+test_expect_success 'setup content merge + rename/directory conflict w/ disappearing dir' '
+	git reset --hard &&
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	mkdir sub &&
+	printf "1\n2\n3\n4\n5\n6\n" >sub/file &&
+	git add sub/file &&
+	test_tick &&
+	git commit -m base &&
+	git tag base &&
+
+	git checkout -b right &&
+	echo 7 >>sub/file &&
+	git add sub/file &&
+	test_tick &&
+	git commit -m right &&
+
+	git checkout -b left base &&
+	echo 0 >newfile &&
+	cat sub/file >>newfile &&
+	git rm sub/file &&
+	mv newfile sub &&
+	git add sub &&
+	test_tick &&
+	git commit -m left
+'
+
+test_expect_success 'disappearing dir in rename/directory conflict handled' '
+	git reset --hard &&
+	git clean -fdqx &&
+
+	git checkout left^0 &&
+
+	git merge -s recursive right^0 &&
+
+	test 1 -eq $(git ls-files -s | wc -l) &&
+	test 0 -eq $(git ls-files -u | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l) &&
+
+	echo 0 >expect &&
+	git cat-file -p base:sub/file >>expect &&
+	echo 7 >>expect &&
+	test_cmp expect sub &&
+
+	test -f sub
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:37 UTC · re: Elijah Newren · lore

Re: [PATCH 05/48] t6039: Add tests for content issues with modify/rename/directory conflicts

Elijah Newren <newren@gmail.com> writes:
> Signed-off-by: Elijah Newren <newren@gmail.com>

Could you add a description for this? Your description on 06/48 is beautifly done. Here is my attempt.

    One branch renames a file and creates a directory where the file used to
    be, and the other branch updates the file in place. Merging them should
    resolve it cleanly as long as the content level change on the branches do
    not overlap and rename is detected, or should leave conflict without
    losing information.
    One branch renames a file and makes a file where the directory the renamed
    file used to be in, and the other branch updates the file in
    place. Merging them should resolve it cleanly as long as the content level
    change on the branches do not overlap and rename is detected, or should
    leave conflict without losing information.

I think I said this in my earlier review, and this is not limited to 05/48 but also applies to 03/48 and 04/48 as well, but there won't be perfect rename detection, and the rename detection logic could change (improve) over time. Ideally, I think the test should declare either case as a success: (1) detection succeeds and avoids unnecessary conflict, or (2) rename is missed and conflict is reported, but otherwise there is no data loss. If it expects only one but not the other, any time the rename logic is improved, the behaviour could change between (1) and (2) and it will cause a false positive "breakage" of these tests.

Thanks.
Elijah Newren· Aug 8, 2011, 15:49 UTC · re: Junio C Hamano · lore

Re: [PATCH 05/48] t6039: Add tests for content issues with modify/rename/directory conflicts

Hi,

Thanks for the many detailed reviews. I've spent much of the weekend (and before) trying to go through and clean things up, and am still working on it. There are some things to comment on outside of the series re-roll I am putting together...

On Mon, Jul 18, 2011 at 5:37 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Could you add a description for this? Your description on 06/48 is
> beautifly done. Here is my attempt.

... Done. :-) It'll be in the reroll I submit.

Show 9 quoted lines
> I think I said this in my earlier review, and this is not limited to 05/48
> but also applies to 03/48 and 04/48 as well, but there won't be perfect
> rename detection, and the rename detection logic could change (improve)
> over time. Ideally, I think the test should declare either case as a
> success: (1) detection succeeds and avoids unnecessary conflict, or (2)
> rename is missed and conflict is reported, but otherwise there is no data
> loss. If it expects only one but not the other, any time the rename logic
> is improved, the behaviour could change between (1) and (2) and it will
> cause a false positive "breakage" of these tests.

The same could be said for all the rename tests, but I'm a bit worried about doing this, particularly for the set of tests I added. I came up with my testcases by carefully reviewing the code looking for ways I could trigger problems (which were almost never covered by an existing regression test), plus trying a few small modifications of any tests I did come up with. The tests I added that involve renames really are specifically designed to test renaming situations. If rename detection logic is modified in a way that affects these tests, and these tests aren't updated, then we are turning some useful unique tests into mere duplication of simpler tests we have elsewhere.

If you are worried that these tests lack robustness in the face of rename-detection changes, perhaps we should modify the tests to use file contents that are more likely to continue being detected as renames (e.g. longer lines or maybe more of them)? Or, maybe add an earlier check in the test to independently verify that the rename is detected?

Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 06/48] t6039: Add failing testcases for rename/rename/add-{source,dest} conflicts

Add testcases that cover three failures with current git merge, all involving renaming one file on both sides of history:

Case 1: If a single file is renamed to two different filenames on different sides of history, there should be a conflict. Adding a new file on one of those sides of history whose name happens to match the rename source should not cause the merge to suddenly succeed.

Case 2: If a single file is renamed on both sides of history but renamed identically, there should not be a conflict. This works fine. However, if one of those sides also added a new file that happened to match the rename source, then that file should be left alone. Currently, the rename/rename conflict handling causes that new file to become untracked.

Case 3: If a single file is renamed to two different filenames on different sides of history, there should be a conflict. This works currently. However, if those renames also involve rename/add conflicts (i.e. there are new files on one side of history that match the destination of the rename of the other side of history), then the resulting conflict should be recorded in the index, showing that there were multiple files with a given filename. Currently, git silently discards one of file versions.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6039-merge-rename-corner-cases.sh |  112 ++++++++++++++++++++++++++++++++++
 1 files changed, 112 insertions(+), 0 deletions(-)
Show changes to t/t6039-merge-rename-corner-cases.sh +112 −0
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index b465667..fd8337f 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -339,4 +339,116 @@ test_expect_success 'disappearing dir in rename/directory conflict handled' '
 	test -f sub
 '
 
+# Testcase setup for rename-rename-add-1:
+#   Commit A: new file: a
+#   Commit B: rename a->b
+#   Commit C: rename a->c, add completely different a
+#
+# Merging of B & C should NOT be clean; there's a rename/rename conflict
+
+test_expect_success 'setup rename/rename(1to2)/add-source conflict' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	printf "1\n2\n3\n4\n5\n6\n7\n" >a &&
+	git add a &&
+	git commit -m A &&
+	git tag A &&
+
+	git checkout -b B A &&
+	git mv a b &&
+	git commit -m B &&
+
+	git checkout -b C A &&
+	git mv a c &&
+	echo something completely different >a &&
+	git add a &&
+	git commit -m C
+'
+
+test_expect_failure 'detect conflict with rename/rename(1to2)/add-source merge' '
+	git checkout B^0 &&
+
+	test_must_fail git merge -s recursive C^0 &&
+
+	test -f a &&
+	test -f b &&
+	test -f c
+'
+
+test_expect_success 'setup rename/rename(1to2)/add-source resolvable conflict' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	>a &&
+	git add a &&
+	test_tick &&
+	git commit -m base &&
+	git tag A &&
+
+	git checkout -b B A &&
+	git mv a b &&
+	test_tick &&
+	git commit -m one &&
+
+	git checkout -b C A &&
+	git mv a b &&
+	echo important-info >a &&
+	test_tick &&
+	git commit -m two
+'
+
+test_expect_failure 'rename/rename/add-source still tracks new a file' '
+	git checkout C^0 &&
+	git merge -s recursive B^0 &&
+
+	test 2 -eq $(git ls-files -s | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l)
+'
+
+test_expect_success 'setup rename/rename(1to2)/add-dest conflict' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	echo stuff >a &&
+	git add a &&
+	test_tick &&
+	git commit -m base &&
+	git tag A &&
+
+	git checkout -b B A &&
+	git mv a b &&
+	echo precious-data >c &&
+	git add c &&
+	test_tick &&
+	git commit -m one &&
+
+	git checkout -b C A &&
+	git mv a c &&
+	echo important-info >b &&
+	git add b &&
+	test_tick &&
+	git commit -m two
+'
+
+test_expect_failure 'rename/rename/add-dest merge still knows about conflicting file versions' '
+	git checkout C^0 &&
+	test_must_fail git merge -s recursive B^0 &&
+
+	test 5 -eq $(git ls-files -s | wc -l) &&
+	test 2 -eq $(git ls-files -u b | wc -l) &&
+	test 2 -eq $(git ls-files -u c | wc -l) &&
+
+	test $(git rev-parse :2:b) = $(git rev-parse C:b) &&
+	test $(git rev-parse :3:b) = $(git rev-parse B:b) &&
+	test $(git rev-parse :2:c) = $(git rev-parse C:c) &&
+	test $(git rev-parse :3:c) = $(git rev-parse B:c)
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:38 UTC · re: Elijah Newren · lore

Re: [PATCH 06/48] t6039: Add failing testcases for rename/rename/add-{source,dest} conflicts

Elijah Newren <newren@gmail.com> writes:
Show 9 quoted lines
> Add testcases that cover three failures with current git merge, all
> involving renaming one file on both sides of history:
>
> Case 1:
> ...
> Case 3:
> ...
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
Very nicely described.
Show 8 quoted lines
> +test_expect_failure 'detect conflict with rename/rename(1to2)/add-source merge' '
> +	git checkout B^0 &&
> +
> +	test_must_fail git merge -s recursive C^0 &&
> +
> +	test -f a &&
> +	test -f b &&
> +	test -f c
Please check the contents of the index here as well.  I think:
 - "a" should be at stage#0, with "something completely different" from C;
 - "b" should have stage #1 and #2, both with the original "a" from A; and
 - "c" should have stage #1 and #3, both with the original "a" from A
right?  The last test in this patch is nicely done in this regard.
Show 29 quoted lines
> +test_expect_success 'setup rename/rename(1to2)/add-source resolvable conflict' '
> +	git rm -rf . &&
> +	git clean -fdqx &&
> +	rm -rf .git &&
> +	git init &&
> +
> +	>a &&
> +	git add a &&
> +	test_tick &&
> +	git commit -m base &&
> +	git tag A &&
> +
> +	git checkout -b B A &&
> +	git mv a b &&
> +	test_tick &&
> +	git commit -m one &&
> +
> +	git checkout -b C A &&
> +	git mv a b &&
> +	echo important-info >a &&
> +	test_tick &&
> +	git commit -m two
> +'
> +
> +test_expect_failure 'rename/rename/add-source still tracks new a file' '
> +	git checkout C^0 &&
> +	git merge -s recursive B^0 &&
> +
> +	test 2 -eq $(git ls-files -s | wc -l) &&

Instead of just checking the numbers, could you make the expected result a bit more explicit? What do we expect here? Perhaps

 - "a" in the index is at stage #0, with "important-info";
 - "a" in the working tree is clean;
 - "b" in the index is at stage #0, with empty content; and
 - "b" in the working tree is clean.
Again, the last test in this patch is nicely done in this regard.
Show 38 quoted lines
> +test_expect_success 'setup rename/rename(1to2)/add-dest conflict' '
> +	git rm -rf . &&
> +	git clean -fdqx &&
> +	rm -rf .git &&
> +	git init &&
> +
> +	echo stuff >a &&
> +	git add a &&
> +	test_tick &&
> +	git commit -m base &&
> +	git tag A &&
> +
> +	git checkout -b B A &&
> +	git mv a b &&
> +	echo precious-data >c &&
> +	git add c &&
> +	test_tick &&
> +	git commit -m one &&
> +
> +	git checkout -b C A &&
> +	git mv a c &&
> +	echo important-info >b &&
> +	git add b &&
> +	test_tick &&
> +	git commit -m two
> +'
> +
> +test_expect_failure 'rename/rename/add-dest merge still knows about conflicting file versions' '
> +	git checkout C^0 &&
> +	test_must_fail git merge -s recursive B^0 &&
> +
> +	test 5 -eq $(git ls-files -s | wc -l) &&
> +	test 2 -eq $(git ls-files -u b | wc -l) &&
> +	test 2 -eq $(git ls-files -u c | wc -l) &&
> +	test $(git rev-parse :2:b) = $(git rev-parse C:b) &&
> +	test $(git rev-parse :3:b) = $(git rev-parse B:b) &&
> +	test $(git rev-parse :2:c) = $(git rev-parse C:c) &&
> +	test $(git rev-parse :3:c) = $(git rev-parse B:c)

5 - 2 - 2 = 1 so we know that it is merged cleanly in the index, but we would want to also test "a" in the working tree is merged and clean.

Thanks.
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 07/48] t6039: Ensure rename/rename conflicts leave index and workdir in sane state

rename/rename conflicts, both with one file being renamed to two different files and with two files being renamed to the same file, should leave the index and the working copy in a sane state with appropriate conflict recording, auxiliary files, etc. Git seems to handle one of the two cases alright, but has some problems with the two files being renamed to one case. Add tests for both cases.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6039-merge-rename-corner-cases.sh |   93 ++++++++++++++++++++++++++++++++++
 1 files changed, 93 insertions(+), 0 deletions(-)
Show changes to t/t6039-merge-rename-corner-cases.sh +93 −0
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index fd8337f..06c7ea5 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -451,4 +451,97 @@ test_expect_failure 'rename/rename/add-dest merge still knows about conflicting
 	test $(git rev-parse :3:c) = $(git rev-parse B:c)
 '
 
+test_expect_success 'setup simple rename/rename (1to2) conflict' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	echo stuff >a &&
+	git add a &&
+	test_tick &&
+	git commit -m A &&
+	git tag A &&
+
+	git checkout -b B A &&
+	git mv a b &&
+	test_tick &&
+	git commit -m B &&
+
+	git checkout -b C A &&
+	git mv a c &&
+	test_tick &&
+	git commit -m C
+'
+
+test_expect_success 'merge has correct working tree contents' '
+	git checkout C^0 &&
+
+	test_must_fail git merge -s recursive B^0 &&
+
+	test 3 -eq $(git ls-files -s | wc -l) &&
+	test 3 -eq $(git ls-files -u | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l) &&
+
+	test -f b &&
+	test -f c
+'
+
+# Test for all kinds of things that can go wrong with rename/rename (2to1):
+#   Commit A: new files: a & b
+#   Commit B: rename a->c, modify b
+#   Commit C: rename b->c, modify a
+#
+# Merging of B & C should NOT be clean.  Questions:
+#   * Both a & b should be removed by the merge; are they?
+#   * The two c's should contain modifications to a & b; do they?
+#   * The index should contain two files, both for c; does it?
+#   * The working copy should have two files, both of form c~<unique>; does it?
+#   * Nothing else should be present.  Is anything?
+
+test_expect_success 'setup rename/rename (2to1) + modify/modify' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	printf "1\n2\n3\n4\n5\n" >a &&
+	printf "5\n4\n3\n2\n1\n" >b &&
+	git add a b &&
+	git commit -m A &&
+	git tag A &&
+
+	git checkout -b B A &&
+	git mv a c &&
+	echo 0 >>b &&
+	git add b &&
+	git commit -m B &&
+
+	git checkout -b C A &&
+	git mv b c &&
+	echo 6 >>a &&
+	git add a &&
+	git commit -m C
+'
+
+test_expect_failure 'handle rename/rename (2to1) conflict correctly' '
+	git checkout B^0 &&
+
+	test_must_fail git merge -s recursive C^0 >out &&
+	grep "CONFLICT (rename/rename)" out &&
+
+	test 2 -eq $(git ls-files -s | wc -l) &&
+	test 2 -eq $(git ls-files -u | wc -l) &&
+	test 2 -eq $(git ls-files -u c | wc -l) &&
+	test 3 -eq $(git ls-files -o | wc -l) &&
+
+	test ! -f a &&
+	test ! -f b &&
+	test -f c~HEAD &&
+	test -f c~C^0 &&
+
+	test $(git hash-object c~HEAD) = $(git rev-parse C:a) &&
+	test $(git hash-object c~C^0) = $(git rev-parse B:b)
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:40 UTC · re: Elijah Newren · lore

Re: [PATCH 07/48] t6039: Ensure rename/rename conflicts leave index and workdir in sane state

Elijah Newren <newren@gmail.com> writes:
Show 22 quoted lines
> +test_expect_success 'setup simple rename/rename (1to2) conflict' '
> +	git rm -rf . &&
> +	git clean -fdqx &&
> +	rm -rf .git &&
> +	git init &&
> +
> +	echo stuff >a &&
> +	git add a &&
> +	test_tick &&
> +	git commit -m A &&
> +	git tag A &&
> +
> +	git checkout -b B A &&
> +	git mv a b &&
> +	test_tick &&
> +	git commit -m B &&
> +
> +	git checkout -b C A &&
> +	git mv a c &&
> +	test_tick &&
> +	git commit -m C
> +'

This looks like a simpler variant of the last test in 06/48; shouldn't it come before the more complex one?

Show 13 quoted lines
> +
> +test_expect_success 'merge has correct working tree contents' '
> +	git checkout C^0 &&
> +
> +	test_must_fail git merge -s recursive B^0 &&
> +
> +	test 3 -eq $(git ls-files -s | wc -l) &&
> +	test 3 -eq $(git ls-files -u | wc -l) &&
> +	test 0 -eq $(git ls-files -o | wc -l) &&
> +
> +	test -f b &&
> +	test -f c
> +'

This test is being a bit too lazy, compared to the better ones in 06/48, no? What do we want to see here? Perhaps

 - "a" at stage #1 with "stuff", not in the working tree;
 - "b" at stage #3 with "stuff", same in the working tree; and
 - "c" at stage #2 with "stuff", same in the working tree
Show 11 quoted lines
> +# Test for all kinds of things that can go wrong with rename/rename (2to1):
> +#   Commit A: new files: a & b
> +#   Commit B: rename a->c, modify b
> +#   Commit C: rename b->c, modify a
> +#
> +# Merging of B & C should NOT be clean.  Questions:
> +#   * Both a & b should be removed by the merge; are they?
> +#   * The two c's should contain modifications to a & b; do they?
> +#   * The index should contain two files, both for c; does it?
> +#   * The working copy should have two files, both of form c~<unique>; does it?
> +#   * Nothing else should be present.  Is anything?

What is the most useful thing to leave in the index and in the working tree for the person who needs to resolve such a merge using the working tree, starting from B and merging C? The above "Questions" lists what the current code might try to do but I am not sure if it is really useful. For example, in the index, you would have to stuff two stage #1 entries ("a" from A and "b" from A) for path "c", with stage #2 ("c" from B) and stage #3 ("c" from C) entries, and represent what B tried to do to "a" (in the above you said "rename a->c" but it does not have to be a rename without content change) and what C tried to do to "b" in the half-conflicted result that is in a single file "c". Because the result are totally unrelated files (one side wants a variant of original "a" there, the other side wants a variant of "b"), such a half-merge result is totally useless to help the person to come up with anything.

Also renaming "c" to "c~<unique>", if they do not have corresponding entries in the index to let you check with "git diff", would make the result _harder_ to use, not easier. So if you are going to rename "c" to "c-B" and "c-C", at least it would make much more sense to have in the index:

 - "c-B", with stage #1 ("a" from A), stage #2 ("c" from B) and stage #3
   ("a" from C);
 - "c-C", with stage #1 ("b" from A), stage #2 ("b" from B) and stage #3
   ("c" from C); and
 - No "a" nor "b" in the index nor in the working tree.
no?

That way, you could run "git diff" to get what happened to the two variants of "a" and "b" at the content level, and decide to clean things up with:

    $ git diff ;# view content level merge
    $ edit c-B c-C; git add c-B c-C
    $ git mv c-B c-some-saner-name
    $ git mv c-C c-another-saner-name
    $ edit other files that refer to c like Makefile
    $ git commit

To take it one step further to the extreme, it might give us a more reasonable and useful conflicted state if we deliberately dropped some information instead in a case like this, e.g.:

 - We may want to have "a" at stage #1 (from A) in the index;
 - No "a" remains in the working tree;
 - "b" at stage #1 (from A), stage #2 (from B) and stage #3 ("c" from C);
 - "b" in the working tree a conflicted-merge of the above three;
 - "c" at stage #1 ("a" from A), stage #2 (from B), and stage #3 ("a" from
   C); and
 - "c" in the working tree a conflicted-merge of the above three.

Note that unlike the current merge-recursive that tries to come up with a temporary pathname to store both versions of C, this would ignore "mv b c" on the A->C branch, and make the conflicted tentative merge asymmetric (merging B into C and merging C into B would give different conflicts), but I suspect that the asymmetry may not hurt us.

Whether the merger wants to keep "c" that was derived from "a" (in line with the HEAD) or "c" that was derived from "b" (in line with MERGE_HEAD), if the result were to keep both files in some shape, the content level edit, renaming of at least one side, and adjusting other files that refer to it, are all required anyway, e.g.

    $ git diff ;# view content level merge
    $ edit b c; git add b c
    $ edit other files that refer to c line Makefile (the content C's
      change wants is now in "b").
    $ git commit

would be a way to pick "c" as "c-some-saner-name" and "b" as "c-another-saner-name" in the previous workflow, but needs much less typing. The complexity of the workflow would be the same if the final resolution is to take what one side did and dropping the other's work, I think.

Elijah Newren· Aug 8, 2011, 17:59 UTC · re: Junio C Hamano · lore

Re: [PATCH 07/48] t6039: Ensure rename/rename conflicts leave index and workdir in sane state

On Mon, Jul 18, 2011 at 5:40 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 50 quoted lines
>> +# Test for all kinds of things that can go wrong with rename/rename (2to1):
>> +#   Commit A: new files: a & b
>> +#   Commit B: rename a->c, modify b
>> +#   Commit C: rename b->c, modify a
>> +#
>> +# Merging of B & C should NOT be clean.  Questions:
>> +#   * Both a & b should be removed by the merge; are they?
>> +#   * The two c's should contain modifications to a & b; do they?
>> +#   * The index should contain two files, both for c; does it?
>> +#   * The working copy should have two files, both of form c~<unique>; does it?
>> +#   * Nothing else should be present.  Is anything?
>
> What is the most useful thing to leave in the index and in the working
> tree for the person who needs to resolve such a merge using the working
> tree, starting from B and merging C? The above "Questions" lists what the
> current code might try to do but I am not sure if it is really useful. For
> example, in the index, you would have to stuff two stage #1 entries ("a"
> from A and "b" from A) for path "c", with stage #2 ("c" from B) and stage
> #3 ("c" from C) entries, and represent what B tried to do to "a" (in the
> above you said "rename a->c" but it does not have to be a rename without
> content change) and what C tried to do to "b" in the half-conflicted
> result that is in a single file "c". Because the result are totally
> unrelated files (one side wants a variant of original "a" there, the other
> side wants a variant of "b"), such a half-merge result is totally useless
> to help the person to come up with anything.
>
> Also renaming "c" to "c~<unique>", if they do not have corresponding
> entries in the index to let you check with "git diff", would make the
> result _harder_ to use, not easier. So if you are going to rename "c" to
> "c-B" and "c-C", at least it would make much more sense to have in the
> index:
>
>  - "c-B", with stage #1 ("a" from A), stage #2 ("c" from B) and stage #3
>   ("a" from C);
>  - "c-C", with stage #1 ("b" from A), stage #2 ("b" from B) and stage #3
>   ("c" from C); and
>  - No "a" nor "b" in the index nor in the working tree.
>
> no?
>
> That way, you could run "git diff" to get what happened to the two
> variants of "a" and "b" at the content level, and decide to clean things
> up with:
>
>    $ git diff ;# view content level merge
>    $ edit c-B c-C; git add c-B c-C
>    $ git mv c-B c-some-saner-name
>    $ git mv c-C c-another-saner-name
>    $ edit other files that refer to c like Makefile
>    $ git commit

That sounds very interesting. My first thought is that you'd have to do the same thing in the case of a D/F conflict, but I notice that later in the patch series you asked for exactly that. The idea certainly has potential, though I might need to think it through a little more.

Show 35 quoted lines
> To take it one step further to the extreme, it might give us a more
> reasonable and useful conflicted state if we deliberately dropped some
> information instead in a case like this, e.g.:
>
>  - We may want to have "a" at stage #1 (from A) in the index;
>  - No "a" remains in the working tree;
>  - "b" at stage #1 (from A), stage #2 (from B) and stage #3 ("c" from C);
>  - "b" in the working tree a conflicted-merge of the above three;
>  - "c" at stage #1 ("a" from A), stage #2 (from B), and stage #3 ("a" from
>   C); and
>  - "c" in the working tree a conflicted-merge of the above three.
>
> Note that unlike the current merge-recursive that tries to come up with a
> temporary pathname to store both versions of C, this would ignore "mv b c"
> on the A->C branch, and make the conflicted tentative merge asymmetric
> (merging B into C and merging C into B would give different conflicts),
> but I suspect that the asymmetry may not hurt us.
>
> Whether the merger wants to keep "c" that was derived from "a" (in line
> with the HEAD) or "c" that was derived from "b" (in line with MERGE_HEAD),
> if the result were to keep both files in some shape, the content level
> edit, renaming of at least one side, and adjusting other files that refer
> to it, are all required anyway, e.g.
>
>    $ git diff ;# view content level merge
>    $ edit b c; git add b c
>    $ edit other files that refer to c line Makefile (the content C's
>      change wants is now in "b").
>    $ git commit
>
> would be a way to pick "c" as "c-some-saner-name" and "b" as
> "c-another-saner-name" in the previous workflow, but needs much less
> typing. The complexity of the workflow would be the same if the final
> resolution is to take what one side did and dropping the other's work,
> I think.

I think the asymmetry is slightly confusing and could become problematic. If we decide to turn on break detection, then we would hit problems in a scenario such as:

Commit A: files a, b are present Commit B: rename a->c, add an unrelated a Commit C: rename b->c, add an unrelated b

In that case, "undoing" the rename as you suggest gives us a conflict with other content that was added at the path.

Also, as mentioned above, D/F conflicts hit similar cases where we need to rename the path in the working copy. If we try to handle them similarly to how you are suggesting for the rename/rename(2to1) case, we can do so in some cases but hit problems in others. For example, take a rename/delete conflict with D/F conflicts:

Commit A: file a is present Commit B: rename a -> df, possibly also modifying it Commit C: delete a, add files a/foo and df/bar

We can't use either the path 'df' or 'a' for recording the content. I think the rules become too confusing for "selectively undoing renames" and it'd be easier to just use <bad-dest-path>~<unique> in all cases. However, I think your suggestion to move index stage information to these uniquely renamed paths could probably work and may be useful.

Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 08/48] t6036: Add differently resolved modify/delete conflict in criss-cross test

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6036-recursive-corner-cases.sh |   83 +++++++++++++++++++++++++++++++++++++
 1 files changed, 83 insertions(+), 0 deletions(-)
Show changes to t/t6036-recursive-corner-cases.sh +83 −0
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index 319b6fa..52d2ecf 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -231,4 +231,87 @@ test_expect_success 'git detects differently handled merges conflict' '
 	test $(git rev-parse :1:new_a) = $(git hash-object merged)
 '
 
+#
+# criss-cross + modify/delete:
+#
+#      B   D
+#      o---o
+#     / \ / \
+#  A o   X   ? F
+#     \ / \ /
+#      o---o
+#      C   E
+#
+#   Commit A: file with contents 'A\n'
+#   Commit B: file with contents 'B\n'
+#   Commit C: file not present
+#   Commit D: file with contents 'B\n'
+#   Commit E: file not present
+#
+# Now, when we merge commits D & E, does git detect the conflict?
+
+test_expect_success 'setup criss-cross + modify/delete resolved differently' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	echo A >file &&
+	git add file &&
+	test_tick &&
+	git commit -m A &&
+
+	git branch B &&
+	git checkout -b C &&
+	git rm file &&
+	test_tick &&
+	git commit -m C &&
+
+	git checkout B &&
+	echo B >file &&
+	git add file &&
+	test_tick &&
+	git commit -m B &&
+
+	git checkout B^0 &&
+	test_must_fail git merge C &&
+	echo B >file &&
+	git add file &&
+	test_tick &&
+	git commit -m D &&
+	git tag D &&
+
+	git checkout C^0 &&
+	test_must_fail git merge B &&
+	git rm file &&
+	test_tick &&
+	git commit -m E &&
+	git tag E
+'
+
+test_expect_failure 'git detects conflict merging criss-cross+modify/delete' '
+	git checkout D^0 &&
+
+	test_must_fail git merge -s recursive E^0 &&
+
+	test 2 -eq $(git ls-files -s | wc -l) &&
+	test 2 -eq $(git ls-files -u | wc -l) &&
+
+	test $(git rev-parse :1:file) = $(git rev-parse master:file) &&
+	test $(git rev-parse :2:file) = $(git rev-parse B:file)
+'
+
+test_expect_failure 'git detects conflict merging criss-cross+modify/delete, reverse direction' '
+	git reset --hard &&
+	git checkout E^0 &&
+
+	test_must_fail git merge -s recursive D^0 &&
+
+	test 2 -eq $(git ls-files -s | wc -l) &&
+	test 2 -eq $(git ls-files -u | wc -l) &&
+
+	test $(git rev-parse :1:file) = $(git rev-parse master:file) &&
+	test $(git rev-parse :3:file) = $(git rev-parse B:file)
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:38 UTC · re: Elijah Newren · lore

Re: [PATCH 08/48] t6036: Add differently resolved modify/delete conflict in criss-cross test

Elijah Newren <newren@gmail.com> writes:
Show 31 quoted lines
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  t/t6036-recursive-corner-cases.sh |   83 +++++++++++++++++++++++++++++++++++++
>  1 files changed, 83 insertions(+), 0 deletions(-)
>
> diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
> index 319b6fa..52d2ecf 100755
> --- a/t/t6036-recursive-corner-cases.sh
> +++ b/t/t6036-recursive-corner-cases.sh
> @@ -231,4 +231,87 @@ test_expect_success 'git detects differently handled merges conflict' '
>  	test $(git rev-parse :1:new_a) = $(git hash-object merged)
>  '
>  
> +#
> +# criss-cross + modify/delete:
> +#
> +#      B   D
> +#      o---o
> +#     / \ / \
> +#  A o   X   ? F
> +#     \ / \ /
> +#      o---o
> +#      C   E
> +#
> +#   Commit A: file with contents 'A\n'
> +#   Commit B: file with contents 'B\n'
> +#   Commit C: file not present
> +#   Commit D: file with contents 'B\n'
> +#   Commit E: file not present
> +#
> +# Now, when we merge commits D & E, does git detect the conflict?

Starting from A, upper branch wants to have B instead of A while the lower branch wants the path gone, so it should result in a modify/delete conflict.

  s/# Now,.*$/Merging commits D & E should result in modify/delete conflict./
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 09/48] t6036: criss-cross with weird content can fool git into clean merge

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6036-recursive-corner-cases.sh |   83 +++++++++++++++++++++++++++++++++++++
 1 files changed, 83 insertions(+), 0 deletions(-)
Show changes to t/t6036-recursive-corner-cases.sh +83 −0
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index 52d2ecf..dab52a4 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -314,4 +314,87 @@ test_expect_failure 'git detects conflict merging criss-cross+modify/delete, rev
 	test $(git rev-parse :3:file) = $(git rev-parse B:file)
 '
 
+#
+# criss-cross + modify/modify with very contrived file contents:
+#
+#      B   D
+#      o---o
+#     / \ / \
+#  A o   X   ? F
+#     \ / \ /
+#      o---o
+#      C   E
+#
+#   Commit A: file with contents 'A\n'
+#   Commit B: file with contents 'B\n'
+#   Commit C: file with contents 'C\n'
+#   Commit D: file with contents 'D\n'
+#   Commit E: file with contents:
+#      <<<<<<< Temporary merge branch 1
+#      C
+#      =======
+#      B
+#      >>>>>>> Temporary merge branch 2
+#
+# Now, when we merge commits D & E, does git detect the conflict?
+
+test_expect_success 'setup differently handled merges of content conflict' '
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	echo A >file &&
+	git add file &&
+	test_tick &&
+	git commit -m A &&
+
+	git branch B &&
+	git checkout -b C &&
+	echo C >file &&
+	git add file &&
+	test_tick &&
+	git commit -m C &&
+
+	git checkout B &&
+	echo B >file &&
+	git add file &&
+	test_tick &&
+	git commit -m B &&
+
+	git checkout B^0 &&
+	test_must_fail git merge C &&
+	echo D >file &&
+	git add file &&
+	test_tick &&
+	git commit -m D &&
+	git tag D &&
+
+	git checkout C^0 &&
+	test_must_fail git merge B &&
+	cat <<EOF >file &&
+<<<<<<< Temporary merge branch 1
+C
+=======
+B
+>>>>>>> Temporary merge branch 2
+EOF
+	git add file &&
+	test_tick &&
+	git commit -m E &&
+	git tag E
+'
+
+test_expect_failure 'git detects conflict w/ criss-cross+contrived resolution' '
+	git checkout D^0 &&
+
+	test_must_fail git merge -s recursive E^0 &&
+
+	test 3 -eq $(git ls-files -s | wc -l) &&
+	test 3 -eq $(git ls-files -u | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l) &&
+
+	test $(git rev-parse :2:file) = $(git rev-parse D:file) &&
+	test $(git rev-parse :3:file) = $(git rev-parse E:file)
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:38 UTC · re: Elijah Newren · lore

Re: [PATCH 09/48] t6036: criss-cross with weird content can fool git into clean merge

Elijah Newren <newren@gmail.com> writes:
Show 8 quoted lines
> +#   Commit E: file with contents:
> +#      <<<<<<< Temporary merge branch 1
> +#      C
> +#      =======
> +#      B
> +#      >>>>>>> Temporary merge branch 2
> +#
> +# Now, when we merge commits D & E, does git detect the conflict?

This is way too artificial that I personally feel not worth worrying about.

Elijah Newren· Aug 8, 2011, 18:02 UTC · re: Junio C Hamano · lore

Re: [PATCH 09/48] t6036: criss-cross with weird content can fool git into clean merge

On Mon, Jul 18, 2011 at 5:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 13 quoted lines
> Elijah Newren <newren@gmail.com> writes:
>
>> +#   Commit E: file with contents:
>> +#      <<<<<<< Temporary merge branch 1
>> +#      C
>> +#      =======
>> +#      B
>> +#      >>>>>>> Temporary merge branch 2
>> +#
>> +# Now, when we merge commits D & E, does git detect the conflict?
>
> This is way too artificial that I personally feel not worth worrying
> about.

I wasn't going to try to do anything to fix it. Should I drop this patch from the series or is it worth keeping around for completeness?

Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 10/48] t6036: tests for criss-cross merges with various directory/file conflicts

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6036-recursive-corner-cases.sh |  149 +++++++++++++++++++++++++++++++++++++
 1 files changed, 149 insertions(+), 0 deletions(-)
Show changes to t/t6036-recursive-corner-cases.sh +149 −0
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index dab52a4..4993f67 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -397,4 +397,153 @@ test_expect_failure 'git detects conflict w/ criss-cross+contrived resolution' '
 	test $(git rev-parse :3:file) = $(git rev-parse E:file)
 '
 
+#
+# criss-cross + d/f conflict via add/add:
+#   Commit A: Neither file 'a' nor directory 'a/' exist.
+#   Commit B: Introduce 'a'
+#   Commit C: Introduce 'a/file'
+# Two different later cases:
+#   Commit D1: Merge B & C, keeping 'a' and deleting 'a/'
+#   Commit E1: Merge B & C, deleting 'a' but keeping 'a/file'
+#
+#   Commit D2: Merge B & C, keeping a modified 'a' and deleting 'a/'
+#   Commit E2: Merge B & C, deleting 'a' but keeping a modified 'a/file'
+#
+#   Note: D == D1.
+# Finally, someone goes to merge D1&E1 or D1&E2 or D2&E1.  What happens?
+#
+#      B   D1 or D2
+#      o---o
+#     / \ / \
+#  A o   X   ? F
+#     \ / \ /
+#      o---o
+#      C   E1 or E2
+#
+
+test_expect_success 'setup differently handled merges of directory/file conflict' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	>irrelevant-file &&
+	git add irrelevant-file &&
+	test_tick &&
+	git commit -m A &&
+
+	git branch B &&
+	git checkout -b C &&
+	mkdir a &&
+	echo 10 >a/file &&
+	git add a/file &&
+	test_tick &&
+	git commit -m C &&
+
+	git checkout B &&
+	echo 5 >a &&
+	git add a &&
+	test_tick &&
+	git commit -m B &&
+
+	git checkout B^0 &&
+	test_must_fail git merge C &&
+	git clean -f &&
+	rm -rf a/ &&
+	echo 5 >a &&
+	git add a &&
+	test_tick &&
+	git commit -m D &&
+	git tag D &&
+
+	git checkout C^0 &&
+	test_must_fail git merge B &&
+	git clean -f &&
+	git rm --cached a &&
+	echo 10 >a/file &&
+	git add a/file &&
+	test_tick &&
+	git commit -m E1 &&
+	git tag E1 &&
+
+	git checkout C^0 &&
+	test_must_fail git merge B &&
+	git clean -f &&
+	git rm --cached a &&
+	printf "10\n11\n" >a/file &&
+	git add a/file &&
+	test_tick &&
+	git commit -m E2 &&
+	git tag E2
+'
+
+test_expect_failure 'git detects conflict and handles merge of D & E1 correctly' '
+	git reset --hard &&
+	git reset --hard &&
+	git clean -fdqx &&
+	git checkout D^0 &&
+
+	# FIXME: If merge-base could keep both a and a/file in its tree, then
+	# we could this merge would actually be able to succeed.
+	test_must_fail git merge -s recursive E1^0 &&
+
+	test 2 -eq $(git ls-files -s | wc -l) &&
+	test 1 -eq $(git ls-files -u | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l) &&
+
+	test $(git rev-parse :2:a) = $(git rev-parse B:a)
+'
+
+test_expect_failure 'git detects conflict and handles merge of E1 & D correctly' '
+	git reset --hard &&
+	git reset --hard &&
+	git clean -fdqx &&
+	git checkout E1^0 &&
+
+	# FIXME: If merge-base could keep both a and a/file in its tree, then
+	# we could this merge would actually be able to succeed.
+	test_must_fail git merge -s recursive D^0 &&
+
+	test 2 -eq $(git ls-files -s | wc -l) &&
+	test 1 -eq $(git ls-files -u | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l) &&
+
+	test $(git rev-parse :3:a) = $(git rev-parse B:a)
+'
+
+test_expect_success 'git detects conflict and handles merge of D & E2 correctly' '
+	git reset --hard &&
+	git reset --hard &&
+	git clean -fdqx &&
+	git checkout D^0 &&
+
+	test_must_fail git merge -s recursive E2^0 &&
+
+	test 3 -eq $(git ls-files -s | wc -l) &&
+	test 2 -eq $(git ls-files -u | wc -l) &&
+	test 1 -eq $(git ls-files -o | wc -l) &&
+
+	test $(git rev-parse :2:a) = $(git rev-parse B:a) &&
+	test $(git rev-parse :3:a/file) = $(git rev-parse E1:a/file)
+	test $(git rev-parse :1:a/file) = $(git rev-parse C:a/file)
+'
+
+test_expect_failure 'git detects conflict and handles merge of E2 & D correctly' '
+	git reset --hard &&
+	git reset --hard &&
+	git clean -fdqx &&
+	git checkout E2^0 &&
+
+	test_must_fail git merge -s recursive D^0 &&
+
+	test 3 -eq $(git ls-files -s | wc -l) &&
+	test 2 -eq $(git ls-files -u | wc -l) &&
+	test 1 -eq $(git ls-files -o | wc -l) &&
+
+	test $(git rev-parse :3:a) = $(git rev-parse B:a) &&
+	test $(git rev-parse :2:a/file) = $(git rev-parse E1:a/file)
+	test $(git rev-parse :1:a/file) = $(git rev-parse C:a/file)
+'
+
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:40 UTC · re: Elijah Newren · lore

Re: [PATCH 10/48] t6036: tests for criss-cross merges with various directory/file conflicts

Elijah Newren <newren@gmail.com> writes:
Show 21 quoted lines
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  t/t6036-recursive-corner-cases.sh |  149 +++++++++++++++++++++++++++++++++++++
>  1 files changed, 149 insertions(+), 0 deletions(-)
>
> diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
> index dab52a4..4993f67 100755
> --- a/t/t6036-recursive-corner-cases.sh
> +++ b/t/t6036-recursive-corner-cases.sh
> @@ -397,4 +397,153 @@ test_expect_failure 'git detects conflict w/ criss-cross+contrived resolution' '
>  	test $(git rev-parse :3:file) = $(git rev-parse E:file)
>  '
>  
> +#
> +# criss-cross + d/f conflict via add/add:
> +#   Commit A: Neither file 'a' nor directory 'a/' exist.
> +#   Commit B: Introduce 'a'
> +#   Commit C: Introduce 'a/file'
> +# Two different later cases:
> +#   Commit D1: Merge B & C, keeping 'a' and deleting 'a/'
> +#   Commit E1: Merge B & C, deleting 'a' but keeping 'a/file'
Where does the content of a/file (or a) go?  Discarded?
> +#   Commit D2: Merge B & C, keeping a modified 'a' and deleting 'a/'
> +#   Commit E2: Merge B & C, deleting 'a' but keeping a modified 'a/file'
What "modification" is there to be made?  Are these evil merges?
> +#   Note: D == D1.

If you are not building nor testing D2, do not bring it up in the above description. Otherwise, use D1 and D2 throughout the test and description.

> +# Finally, someone goes to merge D1&E1 or D1&E2 or D2&E1.  What happens?

State what _should_ happen here. We are not wasting brainwave for mental masturbation but are trying to solve real problems for real projects here.

Also aren't we interested in what should happen when D2 and E2 are merged? If not why not?

Show 8 quoted lines
> +#      B   D1 or D2
> +#      o---o			(file)
> +#     / \ / \
> +#  A o   X   ? F
> +#     \ / \ /
> +#      o---o
> +#      C   E1 or E2		(dir)
> +#
Show 5 quoted lines
> +test_expect_failure 'git detects conflict and handles merge of D & E1 correctly' '
> +	git reset --hard &&
> +	git reset --hard &&
> +	git clean -fdqx &&
> +	git checkout D^0 &&

Why do you need to reset twice? Superstition (you have a commented one introduced later in the series --- perhaps this shows a bug in reset)?

"Start with a clean state at this commit (in this case D^0)" needs a helper shell function to make everything below easier to read.

> +	# FIXME: If merge-base could keep both a and a/file in its tree, then
> +	# we could this merge would actually be able to succeed.
What exactly do you refer to in the above sentence with "merge-base"?
Show 8 quoted lines
> +	test_must_fail git merge -s recursive E1^0 &&
> +
> +	test 2 -eq $(git ls-files -s | wc -l) &&
> +	test 1 -eq $(git ls-files -u | wc -l) &&
> +	test 0 -eq $(git ls-files -o | wc -l) &&
> +
> +	test $(git rev-parse :2:a) = $(git rev-parse B:a)
> +'

What is expected here? "a" at stage #2 ("a" from B) is being tested, and what is the other entry? "a/file" at stage #3 ("a/file" from E1) or something else? What should the working tree have?

> +test_expect_failure 'git detects conflict and handles merge of E1 & D correctly' '

Instead of saying "correctly", state what you think should happen, in other words, what your definition of "correct" is, because in these made-up cases what is expected could be vastly different from people to people.

This is the same merge in the opposite direction, right?
Elijah Newren· Aug 8, 2011, 19:07 UTC · re: Junio C Hamano · lore

Re: [PATCH 10/48] t6036: tests for criss-cross merges with various directory/file conflicts

On Mon, Jul 18, 2011 at 5:40 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 8 quoted lines
>> +test_expect_failure 'git detects conflict and handles merge of D & E1 correctly' '
>> +     git reset --hard &&
>> +     git reset --hard &&
>> +     git clean -fdqx &&
>> +     git checkout D^0 &&
>
> Why do you need to reset twice?  Superstition (you have a commented one
> introduced later in the series --- perhaps this shows a bug in reset)?

I could have sworn that when I first created these tests last September that the double reset was needed to get back to a clean state. I can't seem to duplicate the issue now, though I do see some "error" messages printed on the first reset despite the reset working successfully. I'll drop the extra one. *shrug*

[I'm not ignoring the many other good comments you made on this patch; they just got incorporated into code and comment changes for the next version of the series I send out.]

Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 11/48] t6036: criss-cross w/ rename/rename(1to2)/modify+rename/rename(2to1)/modify

This test is mostly just designed for testing optimality of the virtual merge base in the event of a rename/rename(1to2) conflict. The current choice for resolving this in git seems somewhat confusing and suboptimal.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6036-recursive-corner-cases.sh |   76 +++++++++++++++++++++++++++++++++++++
 1 files changed, 76 insertions(+), 0 deletions(-)
Show changes to t/t6036-recursive-corner-cases.sh +76 −0
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index 4993f67..eee183e 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -545,5 +545,81 @@ test_expect_failure 'git detects conflict and handles merge of E2 & D correctly'
 	test $(git rev-parse :1:a/file) = $(git rev-parse C:a/file)
 '
 
+#
+# criss-cross with rename/rename(1to2)/modify followed by
+# rename/rename(2to1)/modify:
+#
+#      B   D
+#      o---o
+#     / \ / \
+#  A o   X   ? F
+#     \ / \ /
+#      o---o
+#      C   E
+#
+#   Commit A: new file: a
+#   Commit B: rename a->b, modifying by adding a line
+#   Commit C: rename a->c
+#   Commit D: merge B&C, resolving conflict by keeping contents in newname
+#   Commit E: merge B&C, resolving conflict similar to D but adding another line
+#
+# There is a conflict merging B & C, but one of filename not of file
+# content.  Whoever created D and E chose specific resolutions for that
+# conflict resolution.  Now, since: (1) there is no content conflict
+# merging B & C, (2) D does not modify that merged content further, and (3)
+# both D & E resolve the name conflict in the same way, the modification to
+# newname in E should not cause any conflicts when it is merged with D.
+# (Note that this can be accomplished by having the virtual merge base have
+# the merged contents of b and c stored in a file named a, which seems like
+# the most logical choice anyway.)
+
+test_expect_success 'setup rename/rename(1to2)/modify followed by what looks like rename/rename(2to1)/modify' '
+	git reset --hard &&
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	printf "1\n2\n3\n4\n5\n6\n" >a &&
+	git add a &&
+	git commit -m A &&
+	git tag A &&
+
+	git checkout -b B A &&
+	git mv a b &&
+	echo 7 >>b &&
+	git add -u &&
+	git commit -m B &&
+
+	git checkout -b C A &&
+	git mv a c &&
+	git commit -m C &&
+
+	git checkout -q B^0 &&
+	git merge --no-commit -s ours C^0 &&
+	git mv b newname &&
+	git commit -m "Merge commit C^0 into HEAD" &&
+	git tag D &&
+
+	git checkout -q C^0 &&
+	git merge --no-commit -s ours B^0 &&
+	git mv c newname &&
+	printf "7\n8\n" >>newname &&
+	git add -u &&
+	git commit -m "Merge commit B^0 into HEAD" &&
+	git tag E
+'
+
+test_expect_failure 'handle rename/rename(1to2)/modify followed by what looks like rename/rename(2to1)/modify' '
+	git checkout D^0 &&
+
+	git merge -s recursive E^0 &&
+
+	test 1 -eq $(git ls-files -s | wc -l) &&
+	test 0 -eq $(git ls-files -u | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l) &&
+
+	test 8 -eq $(wc -l < newname)
+'
 
 test_done
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:38 UTC · re: Elijah Newren · lore

Re: [PATCH 11/48] t6036: criss-cross w/ rename/rename(1to2)/modify+rename/rename(2to1)/modify

Elijah Newren <newren@gmail.com> writes:
Show 26 quoted lines
> +# criss-cross with rename/rename(1to2)/modify followed by
> +# rename/rename(2to1)/modify:
> +#
> +#      B   D
> +#      o---o
> +#     / \ / \
> +#  A o   X   ? F
> +#     \ / \ /
> +#      o---o
> +#      C   E
> +#
> +#   Commit A: new file: a
> +#   Commit B: rename a->b, modifying by adding a line
> +#   Commit C: rename a->c
> +#   Commit D: merge B&C, resolving conflict by keeping contents in newname
> +#   Commit E: merge B&C, resolving conflict similar to D but adding another line
> +#
> +# There is a conflict merging B & C, but one of filename not of file
> +# content.  Whoever created D and E chose specific resolutions for that
> +# conflict resolution.  Now, since: (1) there is no content conflict
> +# merging B & C, (2) D does not modify that merged content further, and (3)
> +# both D & E resolve the name conflict in the same way, the modification to
> +# newname in E should not cause any conflicts when it is merged with D.
> +# (Note that this can be accomplished by having the virtual merge base have
> +# the merged contents of b and c stored in a file named a, which seems like
> +# the most logical choice anyway.)

Interesting. I do not necessarily agree with the choice "a", but it feels sound to say "B and C do not agree what the final pathname should be, but we know this content was derived from the common A:a so we use one path whose name is arbitrary in the virtual merge base X between D and E" and then further let the rename detection to notice that that arbitrary path gets renamed between X-D to "newname" and X-E also to "newname" to resolve it as both sides renaming it to the same new name. It is akin to what we do at the content level, i.e. "B and C do not agree what the final contents should be, so we leave the conflict marker but that may cancel out at the final merge stage".

Show 10 quoted lines
> +test_expect_failure 'handle rename/rename(1to2)/modify followed by what looks like rename/rename(2to1)/modify' '
> +	git checkout D^0 &&
> +
> +	git merge -s recursive E^0 &&
> +
> +	test 1 -eq $(git ls-files -s | wc -l) &&
> +	test 0 -eq $(git ls-files -u | wc -l) &&
> +	test 0 -eq $(git ls-files -o | wc -l) &&
> +
> +	test 8 -eq $(wc -l < newname)

Be a bit more explicit with what you are actually testing. Are you expecting that the index has a single stage #0 entry for "newname" whose contents are the same as E?

> +'
>  
>  test_done
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 12/48] t6036: criss-cross + rename/rename(1to2)/add-source + modify/modify

Now THAT's a corner case.
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6036-recursive-corner-cases.sh |   72 +++++++++++++++++++++++++++++++++++++
 1 files changed, 72 insertions(+), 0 deletions(-)
Show changes to t/t6036-recursive-corner-cases.sh +72 −0
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index eee183e..2fa7c3e 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -622,4 +622,76 @@ test_expect_failure 'handle rename/rename(1to2)/modify followed by what looks li
 	test 8 -eq $(wc -l < newname)
 '
 
+#
+# criss-cross with rename/rename(1to2)/add-source + resolvable modify/modify:
+#
+#      B   D
+#      o---o
+#     / \ / \
+#  A o   X   ? F
+#     \ / \ /
+#      o---o
+#      C   E
+#
+#   Commit A: new file: a
+#   Commit B: rename a->b
+#   Commit C: rename a->c, add different a
+#   Commit D: merge B&C, keeping b&c and (new) a modified at beginning
+#   Commit E: merge B&C, keeping b&c and (new) a modified at end
+#
+# Now, when we merge commits D & E, there should be no conflict...
+
+test_expect_success 'setup criss-cross + rename/rename/add + modify/modify' '
+	git rm -rf . &&
+	git clean -fdqx &&
+	rm -rf .git &&
+	git init &&
+
+	printf "lots\nof\nwords\nand\ncontent\n" >a &&
+	git add a &&
+	git commit -m A &&
+	git tag A &&
+
+	git checkout -b B A &&
+	git mv a b &&
+	git commit -m B &&
+
+	git checkout -b C A &&
+	git mv a c &&
+	printf "2\n3\n4\n5\n6\n7\n" >a &&
+	git add a &&
+	git commit -m C &&
+
+	git checkout B^0 &&
+	git merge --no-commit -s ours C^0 &&
+	git checkout C -- a c &&
+	mv a old_a &&
+	echo 1 >a &&
+	cat old_a >>a &&
+	rm old_a &&
+	git add -u &&
+	git commit -m "Merge commit C^0 into HEAD" &&
+	git tag D &&
+
+	git checkout C^0 &&
+	git merge --no-commit -s ours B^0 &&
+	git checkout B -- b &&
+	echo 8 >> a &&
+	git add -u &&
+	git commit -m "Merge commit B^0 into HEAD" &&
+	git tag E
+'
+
+test_expect_failure 'correctly resolves criss-cross with rename/rename/add and modify/modify conflict' '
+	git checkout D^0 &&
+
+	git merge -s recursive E^0 &&
+
+	test 3 -eq $(git ls-files -s | wc -l) &&
+	test 0 -eq $(git ls-files -u | wc -l) &&
+	test 0 -eq $(git ls-files -o | wc -l) &&
+
+	test 6 -eq $(wc -l < a)
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:38 UTC · re: Elijah Newren · lore

Re: [PATCH 12/48] t6036: criss-cross + rename/rename(1to2)/add-source + modify/modify

Elijah Newren <newren@gmail.com> writes:
Show 17 quoted lines
> Now THAT's a corner case.
> ...
> +# criss-cross with rename/rename(1to2)/add-source + resolvable modify/modify:
> +#
> +#      B   D
> +#      o---o
> +#     / \ / \
> +#  A o   X   ? F
> +#     \ / \ /
> +#      o---o
> +#      C   E
> +#
> +#   Commit A: new file: a
> +#   Commit B: rename a->b
> +#   Commit C: rename a->c, add different a
> +#   Commit D: merge B&C, keeping b&c and (new) a modified at beginning
> +#   Commit E: merge B&C, keeping b&c and (new) a modified at end

THAT may be a corner case, but is it a useful corner case? What on earth the person who did D (or E) was thinking to keep both b and c that are derived from A:a to begin with?

> +test_expect_failure 'correctly resolves criss-cross with rename/rename/add and modify/modify conflict' '

I won't repeat the same two comments here from reviews for the previous patches.

Phil Hord· Jul 20, 2011, 23:15 UTC · re: Junio C Hamano · lore

Re: [PATCH 12/48] t6036: criss-cross + rename/rename(1to2)/add-source + modify/modify

On 07/18/2011 07:38 PM, Junio C Hamano wrote:
Show 22 quoted lines
> Elijah Newren<newren@gmail.com>  writes:
>
>> Now THAT's a corner case.
>> ...
>> +# criss-cross with rename/rename(1to2)/add-source + resolvable modify/modify:
>> +#
>> +#      B   D
>> +#      o---o
>> +#     / \ / \
>> +#  A o   X   ? F
>> +#     \ / \ /
>> +#      o---o
>> +#      C   E
>> +#
>> +#   Commit A: new file: a
>> +#   Commit B: rename a->b
>> +#   Commit C: rename a->c, add different a
>> +#   Commit D: merge B&C, keeping b&c and (new) a modified at beginning
>> +#   Commit E: merge B&C, keeping b&c and (new) a modified at end
> THAT may be a corner case, but is it a useful corner case?  What on earth
> the person who did D (or E) was thinking to keep both b and c that are
> derived from A:a to begin with?

I think this may happen rather commonly in one of our repos. I must say I was quite surprised when I recognized it in this discussion.

The details are boring, so feel free to ignore the rest of this message unless you just want to understand or contradict me. There is one difference in my real workflow, but for now I'll imagine this difference is not there.

Ciao.
Begin boring details:

We have an expensive, enterprise, config control system that tracks "low-volume" file revisions by unique filenames in a specific format. Here's the life story of a file named foo:

    foo_00    # Revision 00 (initial rev)
    foo_01    # Revision 01
    foo_02    # Revision 02
    foo_A     # Revision A (first 'published' version)
    foo_B     # Revision B (all subsequent versions get published)

When we transition to an externally published version (a one-time event), the revisions change from numbers to letters.

We often need to refer to a specific file version by 'versioned name', so we keep all the revisions around in a pool of files.

If I have pending changes not yet in the "config control" system, I keep a source file in git to hold them until I'm ready to label them with the next revision number.

I have some tools that promote my current edition into the next appropriately named-revision.

   'next' is used to promote the current source to the next appropriate 
revision-named file.
   'publish' does the same thing, but it explicitly promotes the file to 
revision 'A' from the current numeric revision.
A few weeks ago, this is how foo showed up in my repo:
     pending/foo
     deploy/foo_00
     deploy/foo_01
Here's a greatly simplified overview of the tools with this state:
     next foo        # "mv -n pending/foo deploy/foo_02"
     publish foo     # "mv -n pending/foo deploy/foo_A"
[Elijah's "Commit A: new file: a": a=pending/foo]
The story begins.
[Elijah's "Commit B: rename a->b": b=deploy/foo_01]
Changes in pending/foo need to be released, so I prep with 'next foo'.
     $ next foo        # "mv pending/foo deploy/foo_02"
     deploy/foo_00
     deploy/foo_01
     deploy/foo_02
Then I commit and push.
     $ git add -u . && git add . && git commit -m "next foo" && git push
[Elijah's "Commit C: rename a->c, add different a": c=deploy/foo_A]
My boss has been told (but he forgot to tell me) that we need to go to rev A in this release so the package can be published externally.  So, in his repo he preps with 'publish foo'.
     $publish foo     #"mv -n pending/foo deploy/foo_A"
     deploy/foo_00
     deploy/foo_01
     deploy/foo_A
My boss also has some changes for a future version (probably rev B) so 
he adds those, too.
     $ echo Frabnotz > pending/foo
Then he commits and pushes.
     $ git add -u . && git add . && git commit -m "publish foo" && git push
It's easy to embellish the rest.  It's so easy, I won't.  But you can imagine it.
The only deviation in this story from my real workflow is that I never delete pending/foo.  But if it changed radically each time, I probably would.
Phil
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 13/48] t6022: Remove unnecessary untracked files to make test cleaner

Since this test later does a git add -A, we should clean out unnecessary untracked files as part of our cleanup.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6022-merge-rename.sh |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
Show changes to t/t6022-merge-rename.sh +1 −0
diff --git a/t/t6022-merge-rename.sh b/t/t6022-merge-rename.sh
index 1ed259d..1d1b32e 100755
--- a/t/t6022-merge-rename.sh
+++ b/t/t6022-merge-rename.sh
@@ -252,6 +252,7 @@ test_expect_success 'setup for rename + d/f conflicts' '
 	git reset --hard &&
 	git checkout --orphan dir-in-way &&
 	git rm -rf . &&
+	git clean -fdqx &&
 
 	mkdir sub &&
 	mkdir dir &&
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 14/48] t6022: New tests checking for unnecessary updates of files

This testcase was part of en/merge-recursive that was reverted in 6db4105 (Revert "Merge branch 'en/merge-recursive'" 2011-05-19). While the other changes in that series caused unfortunate breakage, this testcase is still useful; reinstate it.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6022-merge-rename.sh |   63 +++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 63 insertions(+), 0 deletions(-)
Show changes to t/t6022-merge-rename.sh +63 −0
diff --git a/t/t6022-merge-rename.sh b/t/t6022-merge-rename.sh
index 1d1b32e..11c5c60 100755
--- a/t/t6022-merge-rename.sh
+++ b/t/t6022-merge-rename.sh
@@ -610,4 +610,67 @@ test_expect_success 'check handling of differently renamed file with D/F conflic
 	! test -f original
 '
 
+test_expect_success 'setup avoid unnecessary update, normal rename' '
+	git reset --hard &&
+	git checkout --orphan avoid-unnecessary-update-1 &&
+	git rm -rf . &&
+	git clean -fdqx &&
+
+	printf "1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n" >original &&
+	git add -A &&
+	git commit -m "Common commmit" &&
+
+	git mv original rename &&
+	echo 11 >>rename &&
+	git add -u &&
+	git commit -m "Renamed and modified" &&
+
+	git checkout -b merge-branch-1 HEAD~1 &&
+	echo "random content" >random-file &&
+	git add -A &&
+	git commit -m "Random, unrelated changes"
+'
+
+test_expect_failure 'avoid unnecessary update, normal rename' '
+	git checkout -q avoid-unnecessary-update-1^0 &&
+	test-chmtime =1000000000 rename &&
+	test-chmtime -v +0 rename >expect &&
+	git merge merge-branch-1 &&
+	test-chmtime -v +0 rename >actual &&
+	test_cmp expect actual # "rename" should have stayed intact
+'
+
+test_expect_success 'setup to test avoiding unnecessary update, with D/F conflict' '
+	git reset --hard &&
+	git checkout --orphan avoid-unnecessary-update-2 &&
+	git rm -rf . &&
+	git clean -fdqx &&
+
+	mkdir df &&
+	printf "1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n" >df/file &&
+	git add -A &&
+	git commit -m "Common commmit" &&
+
+	git mv df/file temp &&
+	rm -rf df &&
+	git mv temp df &&
+	echo 11 >>df &&
+	git add -u &&
+	git commit -m "Renamed and modified" &&
+
+	git checkout -b merge-branch-2 HEAD~1 &&
+	>unrelated-change &&
+	git add unrelated-change &&
+	git commit -m "Only unrelated changes"
+'
+
+test_expect_failure 'avoid unnecessary update, with D/F conflict' '
+	git checkout -q avoid-unnecessary-update-2^0 &&
+	test-chmtime =1000000000 df &&
+	test-chmtime -v +0 df >expect &&
+	git merge merge-branch-2 &&
+	test-chmtime -v +0 df >actual &&
+	test_cmp expect actual # "df" should have stayed intact
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 15/48] t6022: Add testcase for merging a renamed file with a simple change

This is a testcase that was broken by b2c8c0a (merge-recursive: When we detect we can skip an update, actually skip it 2011-02-28) and fixed by 6db4105 (Revert "Merge branch 'en/merge-recursive'" 2011-05-19). Include this testcase to ensure we don't regress it again.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t6022-merge-rename.sh |   22 ++++++++++++++++++++++
 1 files changed, 22 insertions(+), 0 deletions(-)
Show changes to t/t6022-merge-rename.sh +22 −0
diff --git a/t/t6022-merge-rename.sh b/t/t6022-merge-rename.sh
index 11c5c60..52a8f0b 100755
--- a/t/t6022-merge-rename.sh
+++ b/t/t6022-merge-rename.sh
@@ -673,4 +673,26 @@ test_expect_failure 'avoid unnecessary update, with D/F conflict' '
 	test_cmp expect actual # "df" should have stayed intact
 '
 
+test_expect_success 'setup merge of rename + small change' '
+	git reset --hard &&
+	git checkout --orphan rename-plus-small-change &&
+	git rm -rf . &&
+
+	echo ORIGINAL >file &&
+	git add file &&
+
+	test_tick &&
+	git commit -m Initial &&
+	git checkout -b rename_branch &&
+	git mv file renamed_file &&
+	git commit -m Rename &&
+	git checkout rename-plus-small-change &&
+	echo NEW-VERSION >file &&
+	git commit -a -m Reformat
+'
+
+test_expect_success 'merge rename + small change' '
+	git merge rename_branch
+'
+
 test_done
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 16/48] merge-recursive: Make BUG message more legible by adding a newline

Hopefully no one ever hits this error except when making large changes to merge-recursive.c and debugging...

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
Show changes to merge-recursive.c +1 −1
diff --git a/merge-recursive.c b/merge-recursive.c
index db9ba19..3fcd0a5 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -230,7 +230,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 		for (i = 0; i < active_nr; i++) {
 			struct cache_entry *ce = active_cache[i];
 			if (ce_stage(ce))
-				fprintf(stderr, "BUG: %d %.*s", ce_stage(ce),
+				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
 					(int)ce_namelen(ce), ce->name);
 		}
 		die("Bug in merge-recursive.c");
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 17/48] merge-recursive: Correct a comment

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
Show changes to merge-recursive.c +1 −1
diff --git a/merge-recursive.c b/merge-recursive.c
index 3fcd0a5..3d464d9 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1439,7 +1439,7 @@ static int process_df_entry(struct merge_options *o,
 		handle_delete_modify(o, path, new_path,
 				     a_sha, a_mode, b_sha, b_mode);
 	} else if (!o_sha && !!a_sha != !!b_sha) {
-		/* directory -> (directory, file) */
+		/* directory -> (directory, file) or <nothing> -> (directory, file) */
 		const char *add_branch;
 		const char *other_branch;
 		unsigned mode;
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 18/48] merge-recursive: Mark some diff_filespec struct arguments const

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   19 ++++++++++---------
 1 files changed, 10 insertions(+), 9 deletions(-)
Show changes to merge-recursive.c +10 −9
diff --git a/merge-recursive.c b/merge-recursive.c
index 3d464d9..317bf23 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -461,9 +461,10 @@ static struct string_list *get_renames(struct merge_options *o,
 	return renames;
 }
 
-static int update_stages_options(const char *path, struct diff_filespec *o,
-			 struct diff_filespec *a, struct diff_filespec *b,
-			 int clear, int options)
+static int update_stages_options(const char *path, const struct diff_filespec *o,
+				 const struct diff_filespec *a,
+				 const struct diff_filespec *b,
+				 int clear, int options)
 {
 	if (clear)
 		if (remove_file_from_cache(path))
@@ -712,9 +713,9 @@ struct merge_file_info {
 
 static int merge_3way(struct merge_options *o,
 		      mmbuffer_t *result_buf,
-		      struct diff_filespec *one,
-		      struct diff_filespec *a,
-		      struct diff_filespec *b,
+		      const struct diff_filespec *one,
+		      const struct diff_filespec *a,
+		      const struct diff_filespec *b,
 		      const char *branch1,
 		      const char *branch2)
 {
@@ -772,9 +773,9 @@ static int merge_3way(struct merge_options *o,
 }
 
 static struct merge_file_info merge_file(struct merge_options *o,
-				         struct diff_filespec *one,
-					 struct diff_filespec *a,
-					 struct diff_filespec *b,
+					 const struct diff_filespec *one,
+					 const struct diff_filespec *a,
+					 const struct diff_filespec *b,
 					 const char *branch1,
 					 const char *branch2)
 {
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 19/48] merge-recursive: Remember to free generated unique path names

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   21 ++++++++++++++-------
 1 files changed, 14 insertions(+), 7 deletions(-)
Show changes to merge-recursive.c +14 −7
diff --git a/merge-recursive.c b/merge-recursive.c
index 317bf23..45675b6 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1072,7 +1072,6 @@ static int process_renames(struct merge_options *o,
 				   renamed: clean merge */
 				update_file(o, 1, ren1->pair->two->sha1, ren1->pair->two->mode, ren1_dst);
 			} else if (!sha_eq(dst_other.sha1, null_sha1)) {
-				const char *new_path;
 				clean_merge = 0;
 				try_merge = 1;
 				output(o, 1, "CONFLICT (rename/add): Rename %s->%s in %s. "
@@ -1101,9 +1100,10 @@ static int process_renames(struct merge_options *o,
 						    ren1_dst);
 					try_merge = 0;
 				} else {
-					new_path = unique_path(o, ren1_dst, branch2);
+					char *new_path = unique_path(o, ren1_dst, branch2);
 					output(o, 1, "Adding as %s instead", new_path);
 					update_file(o, 0, dst_other.sha1, dst_other.mode, new_path);
+					free(new_path);
 				}
 			} else if ((item = string_list_lookup(renames2Dst, ren1_dst))) {
 				ren2 = item->util;
@@ -1271,13 +1271,14 @@ static int merge_content(struct merge_options *o,
 	}
 
 	if (df_conflict_remains) {
-		const char *new_path;
+		char *new_path;
 		update_file_flags(o, mfi.sha, mfi.mode, path,
 				  o->call_depth || mfi.clean, 0);
 		new_path = unique_path(o, path, df_rename_conflict_branch);
 		mfi.clean = 0;
 		output(o, 1, "Adding as %s instead", new_path);
 		update_file_flags(o, mfi.sha, mfi.mode, new_path, 0, 1);
+		free(new_path);
 	} else {
 		update_file(o, mfi.clean, mfi.sha, mfi.mode, path);
 	}
@@ -1433,12 +1434,17 @@ static int process_df_entry(struct merge_options *o,
 		}
 	} else if (o_sha && (!a_sha || !b_sha)) {
 		/* Modify/delete; deleted side may have put a directory in the way */
-		const char *new_path = path;
-		if (lstat(path, &st) == 0 && S_ISDIR(st.st_mode))
+		char *new_path;
+		int free_me = 0;
+		if (lstat(path, &st) == 0 && S_ISDIR(st.st_mode)) {
 			new_path = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
+			free_me = 1;
+		}
 		clean_merge = 0;
-		handle_delete_modify(o, path, new_path,
+		handle_delete_modify(o, path, free_me ? new_path : path,
 				     a_sha, a_mode, b_sha, b_mode);
+		if (free_me)
+			free(new_path);
 	} else if (!o_sha && !!a_sha != !!b_sha) {
 		/* directory -> (directory, file) or <nothing> -> (directory, file) */
 		const char *add_branch;
@@ -1461,12 +1467,13 @@ static int process_df_entry(struct merge_options *o,
 			conf = "directory/file";
 		}
 		if (lstat(path, &st) == 0 && S_ISDIR(st.st_mode)) {
-			const char *new_path = unique_path(o, path, add_branch);
+			char *new_path = unique_path(o, path, add_branch);
 			clean_merge = 0;
 			output(o, 1, "CONFLICT (%s): There is a directory with name %s in %s. "
 			       "Adding %s as %s",
 			       conf, path, other_branch, path, new_path);
 			update_file(o, 0, sha, mode, new_path);
+			free(new_path);
 		} else {
 			output(o, 2, "Adding %s", path);
 			update_file(o, 1, sha, mode, path);
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:39 UTC · re: Elijah Newren · lore

Re: [PATCH 19/48] merge-recursive: Remember to free generated unique path names

Elijah Newren <newren@gmail.com> writes:
Show 16 quoted lines
> @@ -1433,12 +1434,17 @@ static int process_df_entry(struct merge_options *o,
>  		}
>  	} else if (o_sha && (!a_sha || !b_sha)) {
>  		/* Modify/delete; deleted side may have put a directory in the way */
> -		const char *new_path = path;
> -		if (lstat(path, &st) == 0 && S_ISDIR(st.st_mode))
> +		char *new_path;
> +		int free_me = 0;
> +		if (lstat(path, &st) == 0 && S_ISDIR(st.st_mode)) {
>  			new_path = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
> +			free_me = 1;
> +		}
>  		clean_merge = 0;
> -		handle_delete_modify(o, path, new_path,
> +		handle_delete_modify(o, path, free_me ? new_path : path,
>  				     a_sha, a_mode, b_sha, b_mode);

Here free_me is not about free_me but is used as a substitute for "did I stuff the path to be used in new_path, or should I use the original path?", and I find the variable misnamed. How about doing it this way instead?

	char *renamed = NULL;
        if (lstat(path, &st) ...)
        	renamed = unique_path(...);
	clean_merge = 0;
	handle_delete_modify(o, path, renamed ? renamed : path, ...);
> +		if (free_me)
> +			free(new_path);
and then you can free renamed unconditoinally here.
Other than that looks good to me.
Thanks.
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 20/48] merge-recursive: Avoid working directory changes during recursive case

make_room_for_directories_of_df_conflicts() is about making sure necessary working directory changes can succeed. When o->call_depth > 0 (i.e. the recursive case), we do not want to make any working directory changes so this function should be skipped.

Note that make_room_for_directories_of_df_conflicts() is broken as has been pointed out by Junio; it should NOT be unlinking files. What it should do is keep track of files that could be unlinked if a directory later needs to be written in their place. However, that work also is only relevant in the non-recursive case, so this change is helpful either way.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)
Show changes to merge-recursive.c +7 −0
diff --git a/merge-recursive.c b/merge-recursive.c
index 45675b6..7da6aa0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -355,6 +355,13 @@ static void make_room_for_directories_of_df_conflicts(struct merge_options *o,
 	int last_len = 0;
 	int i;
 
+	/*
+	 * If we're merging merge-bases, we don't want to bother with
+	 * any working directory changes.
+	 */
+	if (o->call_depth)
+		return;
+
 	for (i = 0; i < entries->nr; i++) {
 		const char *path = entries->items[i].string;
 		int len = strlen(path);
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 21/48] merge-recursive: Fix recursive case with D/F conflict via add/add conflict

When a D/F conflict is introduced via an add/add conflict, when o->call_depth > 0 we need to ensure that the higher stage entry from the base stage is removed.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                 |    2 ++
 t/t6036-recursive-corner-cases.sh |    4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)
Show changes to 2 files +4 −2

merge-recursive.c, t/t6036-recursive-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 7da6aa0..4771fb4 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1480,6 +1480,8 @@ static int process_df_entry(struct merge_options *o,
 			       "Adding %s as %s",
 			       conf, path, other_branch, path, new_path);
 			update_file(o, 0, sha, mode, new_path);
+			if (o->call_depth)
+				remove_file_from_cache(path);
 			free(new_path);
 		} else {
 			output(o, 2, "Adding %s", path);
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index 2fa7c3e..423fb62 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -494,7 +494,7 @@ test_expect_failure 'git detects conflict and handles merge of D & E1 correctly'
 	test $(git rev-parse :2:a) = $(git rev-parse B:a)
 '
 
-test_expect_failure 'git detects conflict and handles merge of E1 & D correctly' '
+test_expect_success 'git detects conflict and handles merge of E1 & D correctly' '
 	git reset --hard &&
 	git reset --hard &&
 	git clean -fdqx &&
@@ -528,7 +528,7 @@ test_expect_success 'git detects conflict and handles merge of D & E2 correctly'
 	test $(git rev-parse :1:a/file) = $(git rev-parse C:a/file)
 '
 
-test_expect_failure 'git detects conflict and handles merge of E2 & D correctly' '
+test_expect_success 'git detects conflict and handles merge of E2 & D correctly' '
 	git reset --hard &&
 	git reset --hard &&
 	git clean -fdqx &&
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:40 UTC · re: Elijah Newren · lore

Re: [PATCH 21/48] merge-recursive: Fix recursive case with D/F conflict via add/add conflict

Elijah Newren <newren@gmail.com> writes:
> When a D/F conflict is introduced via an add/add conflict, when
> o->call_depth > 0 we need to ensure that the higher stage entry from the
> base stage is removed.
Show 17 quoted lines
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c                 |    2 ++
>  t/t6036-recursive-corner-cases.sh |    4 ++--
>  2 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 7da6aa0..4771fb4 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -1480,6 +1480,8 @@ static int process_df_entry(struct merge_options *o,
>  			       "Adding %s as %s",
>  			       conf, path, other_branch, path, new_path);
>  			update_file(o, 0, sha, mode, new_path);
> +			if (o->call_depth)
> +				remove_file_from_cache(path);
Puzzling... This codepath is inside
	if (lstat(path, &st) == 0 && S_ISDIR(st.st_mode)) {
        	...

which _is_ all about the working tree. Why should the code even looking at the working tree if it can be called for recursive virtual ancestor merge? I'll keep reading the series, hoping that this is something that will be fixed in a later patch (ah, 23/48 does that).

At the outermost merge, you would keep the made-up name "new_path" as well as the original name "path"?

When you added "d/f" (file in a directory) and I added "d" (file), merging your work to my tree with this code would result in

	d at stage #2 (no 0/1/3);
	d~madeupsuffix in the working tree;
        d/f at stage #3 (no 0/1/2); and
        d/f in the working tree.
right? I am not sure if you also have d-madeupsuffix in the index.

As I said in my review of an earlier patch for test suite, having "d" in the index would be helpful to the user ("git diff" cannot be used to see what's different) so we _might_ be better off leaving this instead:

	d-madeupsuffix at stage #2 (no 0/1/3);
	d~madeupsuffix in the working tree;
        d/f at stage #3 (no 0/1/2); and
        d/f in the working tree.
and removing "d" from the index even for the outermost merge.
Thanks.
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 22/48] merge-recursive: Fix sorting order and directory change assumptions

We cannot assume that directory/file conflicts will appear in sorted order; for example, 'letters.txt' comes between 'letters' and 'letters/file'.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c   |   31 ++++++++++++++++++++++++++-----
 t/t6020-merge-df.sh |   26 ++++++++++++++++++--------
 2 files changed, 44 insertions(+), 13 deletions(-)
Show changes to 2 files +44 −13

merge-recursive.c, t/t6020-merge-df.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 4771fb4..ed1fdb2 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -333,6 +333,28 @@ static struct string_list *get_unmerged(void)
 	return unmerged;
 }
 
+static int string_list_df_name_compare(const void *a, const void *b)
+{
+	const struct string_list_item *one = a;
+	const struct string_list_item *two = b;
+	/*
+	 * Here we only care that entries for D/F conflicts are
+	 * adjacent, in particular with the file of the D/F conflict
+	 * appearing before files below the corresponding directory.
+	 * The order of the rest of the list is irrelevant for us.
+	 *
+	 * To achieve this, we sort with df_name_compare and provide
+	 * the mode S_IFDIR so that D/F conflicts will sort correctly.
+	 * We use the mode S_IFDIR for everything else for simplicity,
+	 * since in other cases any changes in their order due to
+	 * sorting cause no problems for us.
+	 */
+	return df_name_compare(one->string, strlen(one->string), S_IFDIR,
+			       two->string, strlen(two->string), S_IFDIR);
+}
+
+
+
 static void make_room_for_directories_of_df_conflicts(struct merge_options *o,
 						      struct string_list *entries)
 {
@@ -345,11 +367,6 @@ static void make_room_for_directories_of_df_conflicts(struct merge_options *o,
 	 * otherwise, if the file is not supposed to be removed by the
 	 * merge, the contents of the file will be placed in another
 	 * unique filename.
-	 *
-	 * NOTE: This function relies on the fact that entries for a
-	 * D/F conflict will appear adjacent in the index, with the
-	 * entries for the file appearing before entries for paths
-	 * below the corresponding directory.
 	 */
 	const char *last_file = NULL;
 	int last_len = 0;
@@ -362,6 +379,10 @@ static void make_room_for_directories_of_df_conflicts(struct merge_options *o,
 	if (o->call_depth)
 		return;
 
+	/* Ensure D/F conflicts are adjacent in the entries list. */
+	qsort(entries->items, entries->nr, sizeof(*entries->items),
+	      string_list_df_name_compare);
+
 	for (i = 0; i < entries->nr; i++) {
 		const char *path = entries->items[i].string;
 		int len = strlen(path);
diff --git a/t/t6020-merge-df.sh b/t/t6020-merge-df.sh
index eec8f4e..27c3d73 100755
--- a/t/t6020-merge-df.sh
+++ b/t/t6020-merge-df.sh
@@ -59,15 +59,19 @@ test_expect_success 'setup modify/delete + directory/file conflict' '
 	git add letters &&
 	git commit -m initial &&
 
+	# Throw in letters.txt for sorting order fun
+	# ("letters.txt" sorts between "letters" and "letters/file")
 	echo i >>letters &&
-	git add letters &&
+	echo "version 2" >letters.txt &&
+	git add letters letters.txt &&
 	git commit -m modified &&
 
 	git checkout -b delete HEAD^ &&
 	git rm letters &&
 	mkdir letters &&
 	>letters/file &&
-	git add letters &&
+	echo "version 1" >letters.txt &&
+	git add letters letters.txt &&
 	git commit -m deleted
 '
 
@@ -75,25 +79,31 @@ test_expect_success 'modify/delete + directory/file conflict' '
 	git checkout delete^0 &&
 	test_must_fail git merge modify &&
 
-	test 3 = $(git ls-files -s | wc -l) &&
-	test 2 = $(git ls-files -u | wc -l) &&
-	test 1 = $(git ls-files -o | wc -l) &&
+	test 5 -eq $(git ls-files -s | wc -l) &&
+	test 4 -eq $(git ls-files -u | wc -l) &&
+	test 1 -eq $(git ls-files -o | wc -l) &&
 
 	test -f letters/file &&
+	test -f letters.txt &&
 	test -f letters~modify
 '
 
 test_expect_success 'modify/delete + directory/file conflict; other way' '
+	# Yes, we really need the double reset since "letters" appears as
+	# both a file and a directory.
+	git reset --hard &&
 	git reset --hard &&
 	git clean -f &&
 	git checkout modify^0 &&
+
 	test_must_fail git merge delete &&
 
-	test 3 = $(git ls-files -s | wc -l) &&
-	test 2 = $(git ls-files -u | wc -l) &&
-	test 1 = $(git ls-files -o | wc -l) &&
+	test 5 -eq $(git ls-files -s | wc -l) &&
+	test 4 -eq $(git ls-files -u | wc -l) &&
+	test 1 -eq $(git ls-files -o | wc -l) &&
 
 	test -f letters/file &&
+	test -f letters.txt &&
 	test -f letters~HEAD
 '
 
-- 
1.7.6.rc0.62.g2d69f
Johannes Sixt· Jul 11, 2011, 07:04 UTC · re: Elijah Newren · lore

Re: [PATCH 22/48] merge-recursive: Fix sorting order and directory change assumptions

Am 6/8/2011 9:30, schrieb Elijah Newren:
Show 21 quoted lines
>  test_expect_success 'modify/delete + directory/file conflict; other way' '
> +	# Yes, we really need the double reset since "letters" appears as
> +	# both a file and a directory.
> +	git reset --hard &&
>  	git reset --hard &&
>  	git clean -f &&
>  	git checkout modify^0 &&
> +
>  	test_must_fail git merge delete &&
>  
> -	test 3 = $(git ls-files -s | wc -l) &&
> -	test 2 = $(git ls-files -u | wc -l) &&
> -	test 1 = $(git ls-files -o | wc -l) &&
> +	test 5 -eq $(git ls-files -s | wc -l) &&
> +	test 4 -eq $(git ls-files -u | wc -l) &&
> +	test 1 -eq $(git ls-files -o | wc -l) &&
>  
>  	test -f letters/file &&
> +	test -f letters.txt &&
>  	test -f letters~HEAD
>  '
A heads-up: This test case fails here on Windows. The messages produced are:

Merging: 63e2d2b modified virtual delete found 1 common ancestor(s): e6b31cd initial CONFLICT (modify/delete): letters deleted in delete and modified in HEAD. Version HEAD of letters left in tree at letters~HEAD. Adding letters/file error: failed to create path 'letters/file': perhaps a D/F conflict? Auto-merging letters.txt CONFLICT (add/add): Merge conflict in letters.txt Automatic merge failed; fix conflicts and then commit the result.

whereas they are on Linux:

Merging: 274ce87 modified virtual delete found 1 common ancestor(s): 343abf7 initial Removing letters to make room for subdirectory; may re-add later. Adding letters/file CONFLICT (modify/delete): letters deleted in delete and modified in HEAD. Version HEAD of letters left in tree at letters~HEAD. Auto-merging letters.txt CONFLICT (add/add): Merge conflict in letters.txt Automatic merge failed; fix conflicts and then commit the result.

As you can see, "Removing letters..." is missing on Windows, and the file 'letters' is indeed left in the working tree. Any quick ideas where to begin debugging this?

-- Hannes
Johannes Sixt· Jul 12, 2011, 07:27 UTC · re: Johannes Sixt · lore

Re: [PATCH 22/48] merge-recursive: Fix sorting order and directory change assumptions

Am 7/11/2011 9:04, schrieb Johannes Sixt:
> A heads-up: This test case fails here on Windows. The messages produced are:
> As you can see, "Removing letters..." is missing on Windows, and the file
> 'letters' is indeed left in the working tree. Any quick ideas where to
> begin debugging this?

And the reason for this is that the qsort call in record_df_conflict_files assumes that qsort is a stable sort; but this is not guaranteed. In particular, the entry "letters" can be moved after "letters/file", which is not expected by the loop that follows the qsort.

-- Hannes
Johannes Sixt· Jul 13, 2011, 07:24 UTC · re: Johannes Sixt · lore

Re: [PATCH 22/48] merge-recursive: Fix sorting order and directory change assumptions

Am 7/12/2011 9:27, schrieb Johannes Sixt:
Show 11 quoted lines
> Am 7/11/2011 9:04, schrieb Johannes Sixt:
>> A heads-up: This test case fails here on Windows. The messages produced are:
> 
>> As you can see, "Removing letters..." is missing on Windows, and the file
>> 'letters' is indeed left in the working tree. Any quick ideas where to
>> begin debugging this?
> 
> And the reason for this is that the qsort call in record_df_conflict_files
> assumes that qsort is a stable sort; but this is not guaranteed. In
> particular, the entry "letters" can be moved after "letters/file", which
> is not expected by the loop that follows the qsort.
Perhaps something like below. It passes the tests that previously failed.
--- >8 ---
From: Johannes Sixt <j6t@kdbg.org>
Subject: [PATCH] fixup! Do not assume that qsort is stable
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 merge-recursive.c |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +11 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index 88167c3..81fac5f 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -361,6 +361,8 @@ static int string_list_df_name_compare(const void *a, const void *b)
 {
 	const struct string_list_item *one = a;
 	const struct string_list_item *two = b;
+	int onelen = strlen(one->string);
+	int twolen = strlen(two->string);
 	/*
 	 * Here we only care that entries for D/F conflicts are
 	 * adjacent, in particular with the file of the D/F conflict
@@ -373,8 +375,15 @@ static int string_list_df_name_compare(const void *a, const void *b)
 	 * since in other cases any changes in their order due to
 	 * sorting cause no problems for us.
 	 */
-	return df_name_compare(one->string, strlen(one->string), S_IFDIR,
-			       two->string, strlen(two->string), S_IFDIR);
+	int cmp = df_name_compare(one->string, onelen, S_IFDIR,
+				  two->string, twolen, S_IFDIR);
+	/*
+	 * Now that 'foo' and 'foo/bar' compare equal, we have to make sure
+	 * that 'foo' comes before 'foo/bar'.
+	 */
+	if (cmp)
+		return cmp;
+	return onelen - twolen;
 }
 
 static void record_df_conflict_files(struct merge_options *o,
-- 
1.7.6.1317.gd8c5a.dirty
Junio C Hamano· Jul 18, 2011, 23:39 UTC · re: Elijah Newren · lore

Re: [PATCH 22/48] merge-recursive: Fix sorting order and directory change assumptions

Elijah Newren <newren@gmail.com> writes:
Show 34 quoted lines
> We cannot assume that directory/file conflicts will appear in sorted
> order; for example, 'letters.txt' comes between 'letters' and
> 'letters/file'.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c   |   31 ++++++++++++++++++++++++++-----
>  t/t6020-merge-df.sh |   26 ++++++++++++++++++--------
>  2 files changed, 44 insertions(+), 13 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 4771fb4..ed1fdb2 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -333,6 +333,28 @@ static struct string_list *get_unmerged(void)
>  	return unmerged;
>  }
>  
> +static int string_list_df_name_compare(const void *a, const void *b)
> +{
> +	const struct string_list_item *one = a;
> +	const struct string_list_item *two = b;
> +	/*
> +	 * Here we only care that entries for D/F conflicts are
> +	 * adjacent, in particular with the file of the D/F conflict
> +	 * appearing before files below the corresponding directory.
> +	 * The order of the rest of the list is irrelevant for us.
> +	 *
> +	 * To achieve this, we sort with df_name_compare and provide
> +	 * the mode S_IFDIR so that D/F conflicts will sort correctly.
> +	 * We use the mode S_IFDIR for everything else for simplicity,
> +	 * since in other cases any changes in their order due to
> +	 * sorting cause no problems for us.
> +	 */
I recall there was an issue of this sorting reported earlier...
Show 14 quoted lines
> diff --git a/t/t6020-merge-df.sh b/t/t6020-merge-df.sh
> index eec8f4e..27c3d73 100755
> --- a/t/t6020-merge-df.sh
> +++ b/t/t6020-merge-df.sh
> @@ -59,15 +59,19 @@ test_expect_success 'setup modify/delete + directory/file conflict' '
>  	git add letters &&
>  	git commit -m initial &&
>  
> +	# Throw in letters.txt for sorting order fun
> +	# ("letters.txt" sorts between "letters" and "letters/file")
>  	echo i >>letters &&
> -	git add letters &&
> +	echo "version 2" >letters.txt &&
> +	git add letters letters.txt &&
Nice.
Elijah Newren· Aug 8, 2011, 19:21 UTC · re: Junio C Hamano · lore

Re: [PATCH 22/48] merge-recursive: Fix sorting order and directory change assumptions

On Mon, Jul 18, 2011 at 5:39 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 18 quoted lines
>> +static int string_list_df_name_compare(const void *a, const void *b)
>> +{
>> +     const struct string_list_item *one = a;
>> +     const struct string_list_item *two = b;
>> +     /*
>> +      * Here we only care that entries for D/F conflicts are
>> +      * adjacent, in particular with the file of the D/F conflict
>> +      * appearing before files below the corresponding directory.
>> +      * The order of the rest of the list is irrelevant for us.
>> +      *
>> +      * To achieve this, we sort with df_name_compare and provide
>> +      * the mode S_IFDIR so that D/F conflicts will sort correctly.
>> +      * We use the mode S_IFDIR for everything else for simplicity,
>> +      * since in other cases any changes in their order due to
>> +      * sorting cause no problems for us.
>> +      */
>
> I recall there was an issue of this sorting reported earlier...

Yes, in git-fast-export, though. It was 060df62 (fast-export: Fix output order of D/F changes 2010-07-09), if you're curious.

Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 23/48] merge-recursive: Fix code checking for D/F conflicts still being present

Previously, we were using lstat() to determine if a directory was still present after a merge (and thus in the way of adding a file). We should have been using lstat() only to determine if untracked directories were in the way (and then only when necessary to check for untracked directories); we should instead using the index to determine if there is a tracked directory in the way. Create a new function to do this and use it to replace the existing checks for directories being in the way.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   49 ++++++++++++++++++++++++++++++++++---------------
 1 files changed, 34 insertions(+), 15 deletions(-)
Show changes to merge-recursive.c +34 −15
diff --git a/merge-recursive.c b/merge-recursive.c
index ed1fdb2..3a8e64e 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -592,6 +592,30 @@ static void flush_buffer(int fd, const char *buf, unsigned long size)
 	}
 }
 
+static int dir_in_way(const char *path, int check_working_copy)
+{
+	int pathlen = strlen(path);
+	char *dirpath = xmalloc(pathlen + 2);
+	struct stat st;
+
+	strcpy(dirpath, path);
+	dirpath[pathlen] = '/';
+	dirpath[pathlen+1] = '\0';
+
+	int pos = cache_name_pos(dirpath, pathlen+1);
+
+	if (pos < 0)
+		pos = -1 - pos;
+	if (pos < active_nr &&
+	    !strncmp(dirpath, active_cache[pos]->name, pathlen+1)) {
+		free(dirpath);
+		return 1;
+	}
+
+	free(dirpath);
+	return check_working_copy && !lstat(path, &st) && S_ISDIR(st.st_mode);
+}
+
 static int would_lose_untracked(const char *path)
 {
 	int pos = cache_name_pos(path, strlen(path));
@@ -881,7 +905,6 @@ static void conflict_rename_delete(struct merge_options *o,
 {
 	char *dest_name = pair->two->path;
 	int df_conflict = 0;
-	struct stat st;
 
 	output(o, 1, "CONFLICT (rename/delete): Rename %s->%s in %s "
 	       "and deleted in %s",
@@ -892,7 +915,7 @@ static void conflict_rename_delete(struct merge_options *o,
 			      rename_branch == o->branch1 ? pair->two : NULL,
 			      rename_branch == o->branch1 ? NULL : pair->two,
 			      1);
-	if (lstat(dest_name, &st) == 0 && S_ISDIR(st.st_mode)) {
+	if (dir_in_way(dest_name, !o->call_depth)) {
 		dest_name = unique_path(o, dest_name, rename_branch);
 		df_conflict = 1;
 	}
@@ -914,13 +937,12 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 	const char *ren2_dst = pair2->two->path;
 	const char *dst_name1 = ren1_dst;
 	const char *dst_name2 = ren2_dst;
-	struct stat st;
-	if (lstat(ren1_dst, &st) == 0 && S_ISDIR(st.st_mode)) {
+	if (dir_in_way(ren1_dst, !o->call_depth)) {
 		dst_name1 = del[delp++] = unique_path(o, ren1_dst, branch1);
 		output(o, 1, "%s is a directory in %s adding as %s instead",
 		       ren1_dst, branch2, dst_name1);
 	}
-	if (lstat(ren2_dst, &st) == 0 && S_ISDIR(st.st_mode)) {
+	if (dir_in_way(ren2_dst, !o->call_depth)) {
 		dst_name2 = del[delp++] = unique_path(o, ren2_dst, branch2);
 		output(o, 1, "%s is a directory in %s adding as %s instead",
 		       ren2_dst, branch1, dst_name2);
@@ -1080,7 +1102,7 @@ static int process_renames(struct merge_options *o,
 			try_merge = 0;
 
 			if (sha_eq(src_other.sha1, null_sha1)) {
-				if (string_list_has_string(&o->current_directory_set, ren1_dst)) {
+				if (dir_in_way(ren1_dst, 0 /*check_wc*/)) {
 					ren1->dst_entry->processed = 0;
 					setup_rename_df_conflict_info(RENAME_DELETE,
 								      ren1->pair,
@@ -1159,7 +1181,7 @@ static int process_renames(struct merge_options *o,
 					a = &src_other;
 				}
 				update_stages_and_entry(ren1_dst, ren1->dst_entry, one, a, b, 1);
-				if (string_list_has_string(&o->current_directory_set, ren1_dst)) {
+				if (dir_in_way(ren1_dst, 0 /*check_wc*/)) {
 					setup_rename_df_conflict_info(RENAME_NORMAL,
 								      ren1->pair,
 								      NULL,
@@ -1264,7 +1286,6 @@ static int merge_content(struct merge_options *o,
 	const char *reason = "content";
 	struct merge_file_info mfi;
 	struct diff_filespec one, a, b;
-	struct stat st;
 	unsigned df_conflict_remains = 0;
 
 	if (!o_sha) {
@@ -1281,7 +1302,7 @@ static int merge_content(struct merge_options *o,
 
 	mfi = merge_file(o, &one, &a, &b, o->branch1, o->branch2);
 	if (df_rename_conflict_branch &&
-	    lstat(path, &st) == 0 && S_ISDIR(st.st_mode)) {
+	    dir_in_way(path, !o->call_depth)) {
 		df_conflict_remains = 1;
 	}
 
@@ -1346,8 +1367,7 @@ static int process_entry(struct merge_options *o,
 				output(o, 2, "Removing %s", path);
 			/* do not touch working file if it did not exist */
 			remove_file(o, 1, path, !a_sha);
-		} else if (string_list_has_string(&o->current_directory_set,
-						  path)) {
+		} else if (dir_in_way(path, 0 /*check_wc*/)) {
 			entry->processed = 0;
 			return 1; /* Assume clean until processed */
 		} else {
@@ -1370,7 +1390,7 @@ static int process_entry(struct merge_options *o,
 			mode = b_mode;
 			sha = b_sha;
 		}
-		if (string_list_has_string(&o->current_directory_set, path)) {
+		if (dir_in_way(path, 0 /*check_wc*/)) {
 			/* Handle D->F conflicts after all subfiles */
 			entry->processed = 0;
 			return 1; /* Assume clean until processed */
@@ -1418,7 +1438,6 @@ static int process_df_entry(struct merge_options *o,
 	unsigned char *o_sha = stage_sha(entry->stages[1].sha, o_mode);
 	unsigned char *a_sha = stage_sha(entry->stages[2].sha, a_mode);
 	unsigned char *b_sha = stage_sha(entry->stages[3].sha, b_mode);
-	struct stat st;
 
 	entry->processed = 1;
 	if (entry->rename_df_conflict_info) {
@@ -1464,7 +1483,7 @@ static int process_df_entry(struct merge_options *o,
 		/* Modify/delete; deleted side may have put a directory in the way */
 		char *new_path;
 		int free_me = 0;
-		if (lstat(path, &st) == 0 && S_ISDIR(st.st_mode)) {
+		if (dir_in_way(path, !o->call_depth)) {
 			new_path = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
 			free_me = 1;
 		}
@@ -1494,7 +1513,7 @@ static int process_df_entry(struct merge_options *o,
 			sha = b_sha;
 			conf = "directory/file";
 		}
-		if (lstat(path, &st) == 0 && S_ISDIR(st.st_mode)) {
+		if (dir_in_way(path, !o->call_depth)) {
 			char *new_path = unique_path(o, path, add_branch);
 			clean_merge = 0;
 			output(o, 1, "CONFLICT (%s): There is a directory with name %s in %s. "
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 24/48] merge-recursive: Save D/F conflict filenames instead of unlinking them

Rename make_room_for_directories_of_df_conflicts() to record_df_conflict_files() to reflect the change in functionality.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   40 ++++++++++++++++++++++++----------------
 merge-recursive.h |    1 +
 2 files changed, 25 insertions(+), 16 deletions(-)
Show changes to 2 files +25 −16

merge-recursive.c, merge-recursive.h

diff --git a/merge-recursive.c b/merge-recursive.c
index 3a8e64e..e59eec0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -353,20 +353,24 @@ static int string_list_df_name_compare(const void *a, const void *b)
 			       two->string, strlen(two->string), S_IFDIR);
 }
 
-
-
-static void make_room_for_directories_of_df_conflicts(struct merge_options *o,
-						      struct string_list *entries)
+static void record_df_conflict_files(struct merge_options *o,
+				     struct string_list *entries)
 {
-	/* If there are D/F conflicts, and the paths currently exist
-	 * in the working copy as a file, we want to remove them to
-	 * make room for the corresponding directory.  Such paths will
-	 * later be processed in process_df_entry() at the end.  If
-	 * the corresponding directory ends up being removed by the
-	 * merge, then the file will be reinstated at that time;
-	 * otherwise, if the file is not supposed to be removed by the
-	 * merge, the contents of the file will be placed in another
-	 * unique filename.
+	/* If there is a D/F conflict and the file for such a conflict
+	 * currently exist in the working copy, we want to allow it to
+	 * be removed to make room for the corresponding directory if
+	 * needed.  The files underneath the directories of such D/F
+	 * conflicts will be handled in process_entry(), while the
+	 * files of such D/F conflicts will be processed later in
+	 * process_df_entry().  If the corresponding directory ends up
+	 * being removed by the merge, then no additional work needs
+	 * to be done by process_df_entry() for the conflicting file.
+	 * If the directory needs to be written to the working copy,
+	 * then the conflicting file will simply be removed (e.g. in
+	 * make_room_for_path).  If the directory is written to the
+	 * working copy but the file also has a conflict that needs to
+	 * be resolved, then process_df_entry() will reinstate the
+	 * file with a new unique name.
 	 */
 	const char *last_file = NULL;
 	int last_len = 0;
@@ -383,6 +387,7 @@ static void make_room_for_directories_of_df_conflicts(struct merge_options *o,
 	qsort(entries->items, entries->nr, sizeof(*entries->items),
 	      string_list_df_name_compare);
 
+	string_list_clear(&o->df_conflict_file_set, 1);
 	for (i = 0; i < entries->nr; i++) {
 		const char *path = entries->items[i].string;
 		int len = strlen(path);
@@ -391,14 +396,15 @@ static void make_room_for_directories_of_df_conflicts(struct merge_options *o,
 		/*
 		 * Check if last_file & path correspond to a D/F conflict;
 		 * i.e. whether path is last_file+'/'+<something>.
-		 * If so, remove last_file to make room for path and friends.
+		 * If so, record that it's okay to remove last_file to make
+		 * room for path and friends if needed.
 		 */
 		if (last_file &&
 		    len > last_len &&
 		    memcmp(path, last_file, last_len) == 0 &&
 		    path[last_len] == '/') {
 			output(o, 3, "Removing %s to make room for subdirectory; may re-add later.", last_file);
-			unlink(last_file);
+			string_list_insert(&o->df_conflict_file_set, last_file);
 		}
 
 		/*
@@ -1574,7 +1580,7 @@ int merge_trees(struct merge_options *o,
 		get_files_dirs(o, merge);
 
 		entries = get_unmerged();
-		make_room_for_directories_of_df_conflicts(o, entries);
+		record_df_conflict_files(o, entries);
 		re_head  = get_renames(o, head, common, head, merge, entries);
 		re_merge = get_renames(o, merge, common, head, merge, entries);
 		clean = process_renames(o, re_head, re_merge);
@@ -1800,6 +1806,8 @@ void init_merge_options(struct merge_options *o)
 	o->current_file_set.strdup_strings = 1;
 	memset(&o->current_directory_set, 0, sizeof(struct string_list));
 	o->current_directory_set.strdup_strings = 1;
+	memset(&o->df_conflict_file_set, 0, sizeof(struct string_list));
+	o->df_conflict_file_set.strdup_strings = 1;
 }
 
 int parse_merge_opt(struct merge_options *o, const char *s)
diff --git a/merge-recursive.h b/merge-recursive.h
index 7e1e972..58f3435 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -26,6 +26,7 @@ struct merge_options {
 	struct strbuf obuf;
 	struct string_list current_file_set;
 	struct string_list current_directory_set;
+	struct string_list df_conflict_file_set;
 };
 
 /* merge_trees() but with recursive ancestor consolidation */
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 25/48] merge-recursive: Split was_tracked() out of would_lose_untracked()

Checking whether a filename was part of stage 0 or stage 2 is code that we would like to be able to call from a few other places without also lstat()-ing the file to see if it exists in the working copy.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   11 ++++++++---
 1 files changed, 8 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +8 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index e59eec0..d19c519 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -622,7 +622,7 @@ static int dir_in_way(const char *path, int check_working_copy)
 	return check_working_copy && !lstat(path, &st) && S_ISDIR(st.st_mode);
 }
 
-static int would_lose_untracked(const char *path)
+static int was_tracked(const char *path)
 {
 	int pos = cache_name_pos(path, strlen(path));
 
@@ -639,11 +639,16 @@ static int would_lose_untracked(const char *path)
 		switch (ce_stage(active_cache[pos])) {
 		case 0:
 		case 2:
-			return 0;
+			return 1;
 		}
 		pos++;
 	}
-	return file_exists(path);
+	return 0;
+}
+
+static int would_lose_untracked(const char *path)
+{
+	return !was_tracked(path) && file_exists(path);
 }
 
 static int make_room_for_path(const char *path)
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 26/48] merge-recursive: Allow make_room_for_path() to remove D/F entries

If there were several files conflicting below a directory corresponding to a D/F conflict, and the file of that D/F conflict is in the way, we want it to be removed. Since files of D/F conflicts are handled last, they can be reinstated later and possibly with a new unique name.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                 |   18 +++++++++++++++---
 t/t6036-recursive-corner-cases.sh |    2 +-
 2 files changed, 16 insertions(+), 4 deletions(-)
Show changes to 2 files +16 −4

merge-recursive.c, t/t6036-recursive-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index d19c519..e3033f2 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -651,11 +651,23 @@ static int would_lose_untracked(const char *path)
 	return !was_tracked(path) && file_exists(path);
 }
 
-static int make_room_for_path(const char *path)
+static int make_room_for_path(const struct merge_options *o, const char *path)
 {
-	int status;
+	int status, i;
 	const char *msg = "failed to create path '%s'%s";
 
+	/* Unlink any D/F conflict files that are in the way */
+	for (i = 0; i < o->df_conflict_file_set.nr; i++) {
+		const char *df_path = o->df_conflict_file_set.items[i].string;
+		size_t pathlen = strlen(path);
+		size_t df_pathlen = strlen(df_path);
+		if (df_pathlen < pathlen && strncmp(path, df_path, df_pathlen) == 0) {
+			unlink(df_path);
+			break;
+		}
+	}
+
+	/* Make sure leading directories are created */
 	status = safe_create_leading_directories_const(path);
 	if (status) {
 		if (status == -3) {
@@ -723,7 +735,7 @@ static void update_file_flags(struct merge_options *o,
 			}
 		}
 
-		if (make_room_for_path(path) < 0) {
+		if (make_room_for_path(o, path) < 0) {
 			update_wd = 0;
 			free(buf);
 			goto update_index;
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index 423fb62..dea2a65 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -477,7 +477,7 @@ test_expect_success 'setup differently handled merges of directory/file conflict
 	git tag E2
 '
 
-test_expect_failure 'git detects conflict and handles merge of D & E1 correctly' '
+test_expect_success 'git detects conflict and handles merge of D & E1 correctly' '
 	git reset --hard &&
 	git reset --hard &&
 	git clean -fdqx &&
-- 
1.7.6.rc0.62.g2d69f
Johannes Sixt· Jul 11, 2011, 07:14 UTC · re: Elijah Newren · lore

Re: [PATCH 26/48] merge-recursive: Allow make_room_for_path() to remove D/F entries

Am 6/8/2011 9:30, schrieb Elijah Newren:
Show 13 quoted lines
> diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
> index 423fb62..dea2a65 100755
> --- a/t/t6036-recursive-corner-cases.sh
> +++ b/t/t6036-recursive-corner-cases.sh
> @@ -477,7 +477,7 @@ test_expect_success 'setup differently handled merges of directory/file conflict
>  	git tag E2
>  '
>  
> -test_expect_failure 'git detects conflict and handles merge of D & E1 correctly' '
> +test_expect_success 'git detects conflict and handles merge of D & E1 correctly' '
>  	git reset --hard &&
>  	git reset --hard &&
>  	git clean -fdqx &&

This fails on Windows, very likely for the same reason as t6020 that I posted earlier: the message "Removing a... to make room..." is missing as well.

-- Hannes
Johannes Sixt· Jul 13, 2011, 07:17 UTC · re: Elijah Newren · lore

Re: [PATCH 26/48] merge-recursive: Allow make_room_for_path() to remove D/F entries

Am 6/8/2011 9:30, schrieb Elijah Newren:
Show 16 quoted lines
> +static int make_room_for_path(const struct merge_options *o, const char *path)
>  {
> -	int status;
> +	int status, i;
>  	const char *msg = "failed to create path '%s'%s";
>  
> +	/* Unlink any D/F conflict files that are in the way */
> +	for (i = 0; i < o->df_conflict_file_set.nr; i++) {
> +		const char *df_path = o->df_conflict_file_set.items[i].string;
> +		size_t pathlen = strlen(path);
> +		size_t df_pathlen = strlen(df_path);
> +		if (df_pathlen < pathlen && strncmp(path, df_path, df_pathlen) == 0) {
> +			unlink(df_path);
> +			break;
> +		}
> +	}

Each time this loop is entered it tries to remove the same path again, even if it does not exist anymore or was morphed into a directory in the meantime. I suggest to remove a path from o->df_conflict_file_set after it was unlinked. Or even better: have a separate "make room" phase somewhere in the merge process.

-- Hannes
Elijah Newren· Aug 8, 2011, 20:56 UTC · re: Johannes Sixt · lore

Re: [PATCH 26/48] merge-recursive: Allow make_room_for_path() to remove D/F entries

Hi,
On Wed, Jul 13, 2011 at 1:17 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
Show 23 quoted lines
> Am 6/8/2011 9:30, schrieb Elijah Newren:
>> +static int make_room_for_path(const struct merge_options *o, const char *path)
>>  {
>> -     int status;
>> +     int status, i;
>>       const char *msg = "failed to create path '%s'%s";
>>
>> +     /* Unlink any D/F conflict files that are in the way */
>> +     for (i = 0; i < o->df_conflict_file_set.nr; i++) {
>> +             const char *df_path = o->df_conflict_file_set.items[i].string;
>> +             size_t pathlen = strlen(path);
>> +             size_t df_pathlen = strlen(df_path);
>> +             if (df_pathlen < pathlen && strncmp(path, df_path, df_pathlen) == 0) {
>> +                     unlink(df_path);
>> +                     break;
>> +             }
>> +     }
>
> Each time this loop is entered it tries to remove the same path again,
> even if it does not exist anymore or was morphed into a directory in the
> meantime. I suggest to remove a path from o->df_conflict_file_set after it
> was unlinked. Or even better: have a separate "make room" phase somewhere
> in the merge process.

Removing it from o->df_conflict_file_set makes sense. However, there appears to be no API in string_list.h for deleting entries. Are such operations discouraged? I'm not sure whether to add such API, just hack it directly, or wait for someone else to come along and change this to a better data structure (such as a hash)...

I don't think it's possible to move this "make room" phase anywhere earlier in the merge process. When we have D/F conflicts, the files of those D/F conflicts should only be removed if at least one of the paths under the corresponding directory are not removed by the merge process. We don't know whether those paths will need to be removed until we call process_entry() on each of them, and from there we go right to this function when we find one that needs to stick around. So I simply don't see how to move it any earlier.

However, there is clearly a bug in this code. If there is a D/F conflict at 'd' (e.g. paths 'd' and 'd/foo' are present), and there is a file named 'd_bla', then the need to merge 'd_bla' can cause 'd' to be deleted. Oops. (Granted, it's a bug that would be masked by the later call to process_entry() on 'd', which would reinstate that file if necessary, but that doesn't mean this code is right.) I'll fix that up.

However, I don't see how any of this would address any failure you're seeing on windows. Maybe one of my other changes, including one or two other bugfixes I've found will help? I'll have to ping you when I submit the re-roll.

Johannes Sixt· Aug 9, 2011, 07:01 UTC · re: Elijah Newren · lore

Re: [PATCH 26/48] merge-recursive: Allow make_room_for_path() to remove D/F entries

Am 8/8/2011 22:56, schrieb Elijah Newren:
Show 41 quoted lines
> Hi,
> 
> On Wed, Jul 13, 2011 at 1:17 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
>> Am 6/8/2011 9:30, schrieb Elijah Newren:
>>> +static int make_room_for_path(const struct merge_options *o, const char *path)
>>>  {
>>> -     int status;
>>> +     int status, i;
>>>       const char *msg = "failed to create path '%s'%s";
>>>
>>> +     /* Unlink any D/F conflict files that are in the way */
>>> +     for (i = 0; i < o->df_conflict_file_set.nr; i++) {
>>> +             const char *df_path = o->df_conflict_file_set.items[i].string;
>>> +             size_t pathlen = strlen(path);
>>> +             size_t df_pathlen = strlen(df_path);
>>> +             if (df_pathlen < pathlen && strncmp(path, df_path, df_pathlen) == 0) {
>>> +                     unlink(df_path);
>>> +                     break;
>>> +             }
>>> +     }
>>
>> Each time this loop is entered it tries to remove the same path again,
>> even if it does not exist anymore or was morphed into a directory in the
>> meantime. I suggest to remove a path from o->df_conflict_file_set after it
>> was unlinked. Or even better: have a separate "make room" phase somewhere
>> in the merge process.
> 
> Removing it from o->df_conflict_file_set makes sense.  However, there
> appears to be no API in string_list.h for deleting entries.  Are such
> operations discouraged?  I'm not sure whether to add such API, just
> hack it directly, or wait for someone else to come along and change
> this to a better data structure (such as a hash)...
> 
> I don't think it's possible to move this "make room" phase anywhere
> earlier in the merge process.  When we have D/F conflicts, the files
> of those D/F conflicts should only be removed if at least one of the
> paths under the corresponding directory are not removed by the merge
> process.  We don't know whether those paths will need to be removed
> until we call process_entry() on each of them, and from there we go
> right to this function when we find one that needs to stick around.
> So I simply don't see how to move it any earlier.

I missed that the loop is selective: It removes only a subset of the paths. Without a deeper understanding of the merge machinery I'll just accept what you explain. Removing the paths from the list that were unlinked seems to be the right approach.

Here's an attempt for a delete_item API (note: only compile-tested). Bike-shed painters welcome: delete_item, remove_item, free_item?

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Show changes to 3 files +20 −1

Documentation/technical/api-string-list.txt, string-list.c, string-list.h

diff --git a/Documentation/technical/api-string-list.txt b/Documentation/technical/api-string-list.txt
index 3f575bd..ce24eb9 100644
--- a/Documentation/technical/api-string-list.txt
+++ b/Documentation/technical/api-string-list.txt
@@ -29,6 +29,9 @@ member (you need this if you add things later) and you should set the
 
 . Can sort an unsorted list using `sort_string_list`.
 
+. Can remove individual items of an unsorted list using
+  `unsorted_string_list_delete_item`.
+
 . Finally it should free the list using `string_list_clear`.
 
 Example:
@@ -112,6 +115,13 @@ write `string_list_insert(...)->util = ...;`.
 The above two functions need to look through all items, as opposed to their
 counterpart for sorted lists, which performs a binary search.
 
+`unsorted_string_list_delete_item`::
+
+	Remove an item from a string_list. The `string` pointer of the items
+	will be freed in case the `strdup_strings` member of the string_list
+	is set. The third parameter controls if the `util` pointer of the
+	items should be freed or not.
+
 Data structures
 ---------------
 
diff --git a/string-list.c b/string-list.c
index 5168118..d9810ab 100644
--- a/string-list.c
+++ b/string-list.c
@@ -185,3 +185,12 @@ int unsorted_string_list_has_string(struct string_list *list,
 	return unsorted_string_list_lookup(list, string) != NULL;
 }
 
+void unsorted_string_list_delete_item(struct string_list *list, int i, int free_util)
+{
+	if (list->strdup_strings)
+		free(list->items[i].string);
+	if (free_util)
+		free(list->items[i].util);
+	list->items[i] = list->items[list->nr-1];
+	list->nr--;
+}
diff --git a/string-list.h b/string-list.h
index bda6983..0684cb7 100644
--- a/string-list.h
+++ b/string-list.h
@@ -44,4 +44,5 @@ void sort_string_list(struct string_list *list);
 int unsorted_string_list_has_string(struct string_list *list, const char *string);
 struct string_list_item *unsorted_string_list_lookup(struct string_list *list,
 						     const char *string);
+void unsorted_string_list_delete_item(struct string_list *list, int i, int free_util);
 #endif /* STRING_LIST_H */


> [...]
> 
> However, I don't see how any of this would address any failure you're
> seeing on windows.  Maybe one of my other changes, including one or
> two other bugfixes I've found will help?  I'll have to ping you when I
> submit the re-roll.

I stumbled over this only because on Windows we have an implementation of
unlink() that asks whether to retry a failed unlink attempt. I was using
an old version that asks when a directory was tried to unlink where a
retry is pointless.

-- Hannes
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 27/48] merge-recursive: Consolidate different update_stages functions

We are only calling update_stages_options() one way really, so we can consolidate the slightly different variants into one and remove some parameters whose values are always the same.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   27 +++++++++------------------
 1 files changed, 9 insertions(+), 18 deletions(-)
Show changes to merge-recursive.c +9 −18
diff --git a/merge-recursive.c b/merge-recursive.c
index e3033f2..b4baa35 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -495,11 +495,12 @@ static struct string_list *get_renames(struct merge_options *o,
 	return renames;
 }
 
-static int update_stages_options(const char *path, const struct diff_filespec *o,
-				 const struct diff_filespec *a,
-				 const struct diff_filespec *b,
-				 int clear, int options)
+static int update_stages(const char *path, const struct diff_filespec *o,
+			 const struct diff_filespec *a,
+			 const struct diff_filespec *b)
 {
+	int clear = 1;
+	int options = ADD_CACHE_OK_TO_ADD | ADD_CACHE_SKIP_DFCHECK;
 	if (clear)
 		if (remove_file_from_cache(path))
 			return -1;
@@ -515,14 +516,6 @@ static int update_stages_options(const char *path, const struct diff_filespec *o
 	return 0;
 }
 
-static int update_stages(const char *path, struct diff_filespec *o,
-			 struct diff_filespec *a, struct diff_filespec *b,
-			 int clear)
-{
-	int options = ADD_CACHE_OK_TO_ADD | ADD_CACHE_OK_TO_REPLACE;
-	return update_stages_options(path, o, a, b, clear, options);
-}
-
 static int update_stages_and_entry(const char *path,
 				   struct stage_data *entry,
 				   struct diff_filespec *o,
@@ -539,8 +532,7 @@ static int update_stages_and_entry(const char *path,
 	hashcpy(entry->stages[1].sha, o->sha1);
 	hashcpy(entry->stages[2].sha, a->sha1);
 	hashcpy(entry->stages[3].sha, b->sha1);
-	options = ADD_CACHE_OK_TO_ADD | ADD_CACHE_SKIP_DFCHECK;
-	return update_stages_options(path, o, a, b, clear, options);
+	return update_stages(path, o, a, b);
 }
 
 static int remove_file(struct merge_options *o, int clean,
@@ -936,8 +928,7 @@ static void conflict_rename_delete(struct merge_options *o,
 	if (!o->call_depth)
 		update_stages(dest_name, NULL,
 			      rename_branch == o->branch1 ? pair->two : NULL,
-			      rename_branch == o->branch1 ? NULL : pair->two,
-			      1);
+			      rename_branch == o->branch1 ? NULL : pair->two);
 	if (dir_in_way(dest_name, !o->call_depth)) {
 		dest_name = unique_path(o, dest_name, rename_branch);
 		df_conflict = 1;
@@ -980,8 +971,8 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		 * update_file(o, 0, pair2->two->sha1, pair2->two->mode, dst_name2);
 		 */
 	} else {
-		update_stages(ren1_dst, NULL, pair1->two, NULL, 1);
-		update_stages(ren2_dst, NULL, NULL, pair2->two, 1);
+		update_stages(ren1_dst, NULL, pair1->two, NULL);
+		update_stages(ren2_dst, NULL, NULL, pair2->two);
 
 		update_file(o, 0, pair1->two->sha1, pair1->two->mode, dst_name1);
 		update_file(o, 0, pair2->two->sha1, pair2->two->mode, dst_name2);
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:39 UTC · re: Elijah Newren · lore

Re: [PATCH 27/48] merge-recursive: Consolidate different update_stages functions

Elijah Newren <newren@gmail.com> writes:
> We are only calling update_stages_options() one way really, so we can
> consolidate the slightly different variants into one and remove some
> parameters whose values are always the same.
Nice; I wonder if this can come much earlier in the series?
Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 28/48] merge-recursive: Split update_stages_and_entry; only update stages at end

Instead of having the process_renames logic update the stages in the index for the rename destination, have the index updated after process_entry or process_df_entry. This will also allow us to have process_entry determine whether a file was tracked and existed in the working copy before the merge started.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   35 +++++++++++++++++------------------
 1 files changed, 17 insertions(+), 18 deletions(-)
Show changes to merge-recursive.c +17 −18
diff --git a/merge-recursive.c b/merge-recursive.c
index b4baa35..7878b30 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -90,6 +90,7 @@ struct stage_data {
 	} stages[4];
 	struct rename_df_conflict_info *rename_df_conflict_info;
 	unsigned processed:1;
+	unsigned involved_in_rename:1;
 };
 
 static inline void setup_rename_df_conflict_info(enum rename_type rename_type,
@@ -516,15 +517,11 @@ static int update_stages(const char *path, const struct diff_filespec *o,
 	return 0;
 }
 
-static int update_stages_and_entry(const char *path,
-				   struct stage_data *entry,
-				   struct diff_filespec *o,
-				   struct diff_filespec *a,
-				   struct diff_filespec *b,
-				   int clear)
+static void update_entry(struct stage_data *entry,
+			 struct diff_filespec *o,
+			 struct diff_filespec *a,
+			 struct diff_filespec *b)
 {
-	int options;
-
 	entry->processed = 0;
 	entry->stages[1].mode = o->mode;
 	entry->stages[2].mode = a->mode;
@@ -532,7 +529,6 @@ static int update_stages_and_entry(const char *path,
 	hashcpy(entry->stages[1].sha, o->sha1);
 	hashcpy(entry->stages[2].sha, a->sha1);
 	hashcpy(entry->stages[3].sha, b->sha1);
-	return update_stages(path, o, a, b);
 }
 
 static int remove_file(struct merge_options *o, int clean,
@@ -1084,12 +1080,11 @@ static int process_renames(struct merge_options *o,
 							      ren2->dst_entry);
 			} else {
 				remove_file(o, 1, ren1_src, 1);
-				update_stages_and_entry(ren1_dst,
-							ren1->dst_entry,
-							ren1->pair->one,
-							ren1->pair->two,
-							ren2->pair->two,
-							1 /* clear */);
+				update_entry(ren1->dst_entry,
+					     ren1->pair->one,
+					     ren1->pair->two,
+					     ren2->pair->two);
+				ren1->dst_entry->involved_in_rename = 1;
 			}
 		} else {
 			/* Renamed in 1, maybe changed in 2 */
@@ -1194,7 +1189,8 @@ static int process_renames(struct merge_options *o,
 					b = ren1->pair->two;
 					a = &src_other;
 				}
-				update_stages_and_entry(ren1_dst, ren1->dst_entry, one, a, b, 1);
+				update_entry(ren1->dst_entry, one, a, b);
+				ren1->dst_entry->involved_in_rename = 1;
 				if (dir_in_way(ren1_dst, 0 /*check_wc*/)) {
 					setup_rename_df_conflict_info(RENAME_NORMAL,
 								      ren1->pair,
@@ -1291,6 +1287,7 @@ static void handle_delete_modify(struct merge_options *o,
 }
 
 static int merge_content(struct merge_options *o,
+			 unsigned involved_in_rename,
 			 const char *path,
 			 unsigned char *o_sha, int o_mode,
 			 unsigned char *a_sha, int a_mode,
@@ -1331,6 +1328,8 @@ static int merge_content(struct merge_options *o,
 			reason = "submodule";
 		output(o, 1, "CONFLICT (%s): Merge conflict in %s",
 				reason, path);
+		if (involved_in_rename)
+			update_stages(path, &one, &a, &b);
 	}
 
 	if (df_conflict_remains) {
@@ -1415,7 +1414,7 @@ static int process_entry(struct merge_options *o,
 	} else if (a_sha && b_sha) {
 		/* Case C: Added in both (check for same permissions) and */
 		/* case D: Modified in both, but differently. */
-		clean_merge = merge_content(o, path,
+		clean_merge = merge_content(o, entry->involved_in_rename, path,
 					    o_sha, o_mode, a_sha, a_mode, b_sha, b_mode,
 					    NULL);
 	} else if (!o_sha && !a_sha && !b_sha) {
@@ -1459,7 +1458,7 @@ static int process_df_entry(struct merge_options *o,
 		char *src;
 		switch (conflict_info->rename_type) {
 		case RENAME_NORMAL:
-			clean_merge = merge_content(o, path,
+			clean_merge = merge_content(o, entry->involved_in_rename, path,
 						    o_sha, o_mode, a_sha, a_mode, b_sha, b_mode,
 						    conflict_info->branch1);
 			break;
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:39 UTC · re: Elijah Newren · lore

Re: [PATCH 28/48] merge-recursive: Split update_stages_and_entry; only update stages at end

Elijah Newren <newren@gmail.com> writes:
Show 55 quoted lines
> Instead of having the process_renames logic update the stages in the index
> for the rename destination, have the index updated after process_entry or
> process_df_entry.  This will also allow us to have process_entry determine
> whether a file was tracked and existed in the working copy before the
> merge started.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c |   35 +++++++++++++++++------------------
>  1 files changed, 17 insertions(+), 18 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index b4baa35..7878b30 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -90,6 +90,7 @@ struct stage_data {
>  	} stages[4];
>  	struct rename_df_conflict_info *rename_df_conflict_info;
>  	unsigned processed:1;
> +	unsigned involved_in_rename:1;
>  };
>  
>  static inline void setup_rename_df_conflict_info(enum rename_type rename_type,
> @@ -516,15 +517,11 @@ static int update_stages(const char *path, const struct diff_filespec *o,
>  	return 0;
>  }
>  
> -static int update_stages_and_entry(const char *path,
> -				   struct stage_data *entry,
> -				   struct diff_filespec *o,
> -				   struct diff_filespec *a,
> -				   struct diff_filespec *b,
> -				   int clear)
> +static void update_entry(struct stage_data *entry,
> +			 struct diff_filespec *o,
> +			 struct diff_filespec *a,
> +			 struct diff_filespec *b)
>  {
> -	int options;
> -
>  	entry->processed = 0;
>  	entry->stages[1].mode = o->mode;
>  	entry->stages[2].mode = a->mode;
> @@ -532,7 +529,6 @@ static int update_stages_and_entry(const char *path,
>  	hashcpy(entry->stages[1].sha, o->sha1);
>  	hashcpy(entry->stages[2].sha, a->sha1);
>  	hashcpy(entry->stages[3].sha, b->sha1);
> -	return update_stages(path, o, a, b);
>  }
>  
>  static int remove_file(struct merge_options *o, int clean,
> @@ -1084,12 +1080,11 @@ static int process_renames(struct merge_options *o,
>  							      ren2->dst_entry);
>  			} else {
>  				remove_file(o, 1, ren1_src, 1);

Hopefully this unconditional removal from the working tree will be fixed by the end of the series, at least for the virtual ancestor merges.

Show 11 quoted lines
> -				update_stages_and_entry(ren1_dst,
> -							ren1->dst_entry,
> -							ren1->pair->one,
> -							ren1->pair->two,
> -							ren2->pair->two,
> -							1 /* clear */);
> +				update_entry(ren1->dst_entry,
> +					     ren1->pair->one,
> +					     ren1->pair->two,
> +					     ren2->pair->two);
> +				ren1->dst_entry->involved_in_rename = 1;

So the idea is to just record what the stage data should be, and delay calling update_stages() until we run the content level merge?

Show 35 quoted lines
> @@ -1291,6 +1287,7 @@ static void handle_delete_modify(struct merge_options *o,
>  }
>  
>  static int merge_content(struct merge_options *o,
> +			 unsigned involved_in_rename,
>  			 const char *path,
>  			 unsigned char *o_sha, int o_mode,
>  			 unsigned char *a_sha, int a_mode,
> @@ -1331,6 +1328,8 @@ static int merge_content(struct merge_options *o,
>  			reason = "submodule";
>  		output(o, 1, "CONFLICT (%s): Merge conflict in %s",
>  				reason, path);
> +		if (involved_in_rename)
> +			update_stages(path, &one, &a, &b);
>  	}
>  
>  	if (df_conflict_remains) {
> @@ -1415,7 +1414,7 @@ static int process_entry(struct merge_options *o,
>  	} else if (a_sha && b_sha) {
>  		/* Case C: Added in both (check for same permissions) and */
>  		/* case D: Modified in both, but differently. */
> -		clean_merge = merge_content(o, path,
> +		clean_merge = merge_content(o, entry->involved_in_rename, path,
>  					    o_sha, o_mode, a_sha, a_mode, b_sha, b_mode,
>  					    NULL);
>  	} else if (!o_sha && !a_sha && !b_sha) {
> @@ -1459,7 +1458,7 @@ static int process_df_entry(struct merge_options *o,
>  		char *src;
>  		switch (conflict_info->rename_type) {
>  		case RENAME_NORMAL:
> -			clean_merge = merge_content(o, path,
> +			clean_merge = merge_content(o, entry->involved_in_rename, path,
>  						    o_sha, o_mode, a_sha, a_mode, b_sha, b_mode,
>  						    conflict_info->branch1);
>  			break;

Feels sane from a cursory look, but are there cases where we do update_entry() above and end up not calling merge_content() at all?

Elijah Newren· Jun 8, 2011, 07:30 UTC · re: Elijah Newren · lore

[PATCH 29/48] merge-recursive: When we detect we can skip an update, actually skip it

In 882fd11 (merge-recursive: Delay content merging for renames 2010-09-20), there was code that checked for whether we could skip updating a file in the working directory, based on whether the merged version matched the current working copy. Due to the desire to handle directory/file conflicts that were resolvable, that commit deferred content merging by first updating the index with the unmerged entries and then moving the actual merging (along with the skip-the-content-update check) to another function that ran later in the merge process. As part moving the content merging code, a bug was introduced such that although the message about skipping the update would be printed (whenever GIT_MERGE_VERBOSITY was sufficiently high), the file would be unconditionally updated in the working copy anyway.

When we detect that the file does not need to be updated in the working copy, update the index appropriately and then return early before updating the working copy.

Note that there was a similar change in b2c8c0a (merge-recursive: When we detect we can skip an update, actually skip it 2011-02-28), but it was reverted by 6db4105 (Revert "Merge branch 'en/merge-recursive'" 2011-05-19) since it did not fix both of the relevant types of unnecessary update breakages and, worse, it made use of some band-aids that caused other problems. The reason this change works is due to the changes of the last few patches to (a) record_df_conflict_files instead of just unlinking them early, (b) allowing make_room_for_path() to remove D/F entries, and (c) the splitting of update_stages_and_entry() to have its functionality called at different points.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c       |    8 ++++++--
 t/t6022-merge-rename.sh |    4 ++--
 2 files changed, 8 insertions(+), 4 deletions(-)
Show changes to 2 files +8 −4

merge-recursive.c, t/t6022-merge-rename.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 7878b30..987a985 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1318,9 +1318,13 @@ static int merge_content(struct merge_options *o,
 	}
 
 	if (mfi.clean && !df_conflict_remains &&
-	    sha_eq(mfi.sha, a_sha) && mfi.mode == a.mode)
+	    sha_eq(mfi.sha, a_sha) && mfi.mode == a.mode &&
+	    was_tracked(path)) {
 		output(o, 3, "Skipped %s (merged same as existing)", path);
-	else
+		add_cacheinfo(mfi.mode, mfi.sha, path,
+			      0 /*stage*/, 1 /*refresh*/, 0 /*options*/);
+		return mfi.clean;
+	} else
 		output(o, 2, "Auto-merging %s", path);
 
 	if (!mfi.clean) {
diff --git a/t/t6022-merge-rename.sh b/t/t6022-merge-rename.sh
index 52a8f0b..5e2a686 100755
--- a/t/t6022-merge-rename.sh
+++ b/t/t6022-merge-rename.sh
@@ -631,7 +631,7 @@ test_expect_success 'setup avoid unnecessary update, normal rename' '
 	git commit -m "Random, unrelated changes"
 '
 
-test_expect_failure 'avoid unnecessary update, normal rename' '
+test_expect_success 'avoid unnecessary update, normal rename' '
 	git checkout -q avoid-unnecessary-update-1^0 &&
 	test-chmtime =1000000000 rename &&
 	test-chmtime -v +0 rename >expect &&
@@ -664,7 +664,7 @@ test_expect_success 'setup to test avoiding unnecessary update, with D/F conflic
 	git commit -m "Only unrelated changes"
 '
 
-test_expect_failure 'avoid unnecessary update, with D/F conflict' '
+test_expect_success 'avoid unnecessary update, with D/F conflict' '
 	git checkout -q avoid-unnecessary-update-2^0 &&
 	test-chmtime =1000000000 df &&
 	test-chmtime -v +0 df >expect &&
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 18, 2011, 23:39 UTC · re: Elijah Newren · lore

Re: [PATCH 29/48] merge-recursive: When we detect we can skip an update, actually skip it

Elijah Newren <newren@gmail.com> writes:
Show 5 quoted lines
> ...  The reason this change works is due to the changes of the
> last few patches to (a) record_df_conflict_files instead of just unlinking
> them early, (b) allowing make_room_for_path() to remove D/F entries, and
> (c) the splitting of update_stages_and_entry() to have its functionality
> called at different points.
Nice.
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 30/48] merge-recursive: Fix deletion of untracked file in rename/delete conflicts

In the recursive case (o->call_depth > 0), we do not modify the working directory. However, when o->call_depth==0, file renames can mean we need to delete the old filename from the working copy. Since there have been lots of changes and mistakes here, let's go through the details. Let's start with a simple explanation of what we are trying to achieve:

  Original goal: If a file is renamed on the side of history being merged
  into head, the filename serving as the source of that rename needs to be
  removed from the working directory.
The path to getting the above statement implemented in merge-recursive took
several steps.  The relevant bits of code may be instructive to keep in
mind for the explanation, especially since an English-only description
involves double negatives that are hard to follow.  These bits of code are:
  int remove_file(..., const char *path, int no_wd)
  {
    ...
    int update_working_directory = !o->call_depth && !no_wd;
and
  remove_file(o, 1, ren1_src, <expression>);
Where the choice for <expression> has morphed over time:

65ac6e9 (merge-recursive: adjust to loosened "working file clobbered" check 2006-10-27), introduced the "no_wd" parameter to remove_file() and used "1" for <expression>. This meant ren1_src was never deleted, leaving it around in the working copy.

In 8371234 (Remove uncontested renamed files during merge. 2006-12-13), <expression> was changed to "index_only" (where index_only == !!o->call_depth; see b7fa51da). This was equivalent to using "0" for <expression> (due to the early logic in remove_file), and is orthogonal to the condition we actually want to check at this point; it resulted in the source file being removed except when index_only was false. This was problematic because the file could have been renamed on the side of history including head, in which case ren1_src could correspond to an untracked file that should not be deleted.

In 183d797 (Keep untracked files not involved in a merge. 2007-02-04), <expression> was changed to "index_only || stage == 3". While this gives correct behavior, the "index_only ||" portion of <expression> is unnecessary and makes the code slightly harder to follow.

There were also two further changes to this expression, though without
any change in behavior.  First in b7fa51d (merge-recursive: get rid of the
index_only global variable 2008-09-02), it was changed to "o->call_depth
|| stage == 3".  (index_only == !!o->call_depth).  Later, in 41d70bd6
(merge-recursive: Small code clarification -- variable name and comments),
this was changed to "o->call_depth || renamed_stage == 2" (where stage was
renamed to other_stage and renamed_stage == other_stage ^ 1).

So we ended with <expression> being "o->call_depth || renamed_stage == 2". But the "o->call_depth ||" piece was unnecessary. We can remove it, leaving us with <expression> being "renamed_stage == 2". This doesn't change behavior at all, but it makes the code clearer. Which is good, because it's about to get uglier.

  Corrected goal: If a file is renamed on the side of history being merged
  into head, the filename serving as the source of that rename needs to be
  removed from the working directory *IF* that file is tracked in head AND
  the file tracked in head is related to the original file.

Note that the only difference between the original goal and the corrected goal is the two extra conditions added at the end. The first condition is relevant in a rename/delete conflict. If the file was deleted on the HEAD side of the merge and an untracked file of the same name was added to the working copy, then without that extra condition the untracked file will be erroneously deleted. This changes <expression> to "renamed_stage == 2 || !was_tracked(ren1_src)".

The second additional condition is relevant in two cases.

The first case the second condition can occur is when a file is deleted and a completely different file is added with the same name. To my knowledge, merge-recursive has no mechanism for detecting deleted-and- replaced-by-different-file cases, so I am simply punting on this possibility.

The second case for the second condition to occur is when there is a rename/rename/add-source conflict. That is, when the original file was renamed on both sides of history AND the original filename is being re-used by some unrelated (but tracked) content. This case also presents some additional difficulties for us since we cannot currently detect these rename/rename/add-source conflicts; as long as the rename detection logic "optimizes" by ignoring filenames that are present at both ends of the diff, these conflicts will go unnoticed. However, rename/rename conflicts are handled by an entirely separate codepath not being discussed here, so this case is not relevant for the line of code under consideration.

In summary:
  Change <expression> from "o->call_depth || renamed_stage == 2" to
  "renamed_stage == 2 || !was_tracked(ren1_src)", in order to
  remove unnecessary code and avoid deleting untracked files.
96 lines of explanation in the changelog to describe a one-line fix...
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                    |    3 ++-
 t/t6039-merge-rename-corner-cases.sh |    2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)
Show changes to 2 files +3 −2

merge-recursive.c, t/t6039-merge-rename-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 987a985..749e501 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1102,7 +1102,8 @@ static int process_renames(struct merge_options *o,
 			int renamed_stage = a_renames == renames1 ? 2 : 3;
 			int other_stage =   a_renames == renames1 ? 3 : 2;
 
-			remove_file(o, 1, ren1_src, o->call_depth || renamed_stage == 2);
+			remove_file(o, 1, ren1_src,
+				    renamed_stage == 2 || !was_tracked(ren1_src));
 
 			hashcpy(src_other.sha1, ren1->src_entry->stages[other_stage].sha);
 			src_other.mode = ren1->src_entry->stages[other_stage].mode;
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index 06c7ea5..4f94528 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -26,7 +26,7 @@ test_expect_success 'setup rename/delete + untracked file' '
 	echo "Myyy PRECIOUSSS" >ring
 '
 
-test_expect_failure "Does git preserve Gollum's precious artifact?" '
+test_expect_success "Does git preserve Gollum's precious artifact?" '
 	test_must_fail git merge -s recursive rename-the-ring &&
 
 	# Make sure git did not delete an untracked file
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 21, 2011, 18:43 UTC · re: Elijah Newren · lore

Re: [PATCH 30/48] merge-recursive: Fix deletion of untracked file in rename/delete conflicts

Elijah Newren <newren@gmail.com> writes:
Show 6 quoted lines
> In summary:
>   Change <expression> from "o->call_depth || renamed_stage == 2" to
>   "renamed_stage == 2 || !was_tracked(ren1_src)", in order to
>   remove unnecessary code and avoid deleting untracked files.
>
> 96 lines of explanation in the changelog to describe a one-line fix...
Beautifully written analysis, though ;-)
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 31/48] merge-recursive: Make dead code for rename/rename(2to1) conflicts undead

The code for rename_rename_2to1 conflicts (two files both being renamed to the same filename) was dead since the rename/add path was always being independently triggered for each of the renames instead. Further, reviving the dead code showed that it was inherently buggy and would always segfault -- among a few other bugs.

Move the else-if branch for the rename/rename block before the rename/add block to make sure it is checked first, and fix up the rename/rename(2to1) code segments to make it handle most cases. Work is still needed to handle higher dimensional corner cases such as rename/rename/modify/modify issues.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                 |   70 +++++++++++++++++++++++++-----------
 t/t6036-recursive-corner-cases.sh |   17 +++++----
 2 files changed, 57 insertions(+), 30 deletions(-)
Show changes to 2 files +57 −30

merge-recursive.c, t/t6036-recursive-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 749e501..4c42838 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -983,17 +983,36 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 					struct rename *ren2,
 					const char *branch2)
 {
+	char *path = ren1->pair->two->path; /* same as ren2->pair->two->path */
 	/* Two files were renamed to the same thing. */
-	char *new_path1 = unique_path(o, ren1->pair->two->path, branch1);
-	char *new_path2 = unique_path(o, ren2->pair->two->path, branch2);
-	output(o, 1, "Renaming %s to %s and %s to %s instead",
-	       ren1->pair->one->path, new_path1,
-	       ren2->pair->one->path, new_path2);
-	remove_file(o, 0, ren1->pair->two->path, 0);
-	update_file(o, 0, ren1->pair->two->sha1, ren1->pair->two->mode, new_path1);
-	update_file(o, 0, ren2->pair->two->sha1, ren2->pair->two->mode, new_path2);
-	free(new_path2);
-	free(new_path1);
+	if (o->call_depth) {
+		struct merge_file_info mfi;
+		struct diff_filespec one, a, b;
+
+		one.path = a.path = b.path = path;
+		hashcpy(one.sha1, null_sha1);
+		one.mode = 0;
+		hashcpy(a.sha1, ren1->pair->two->sha1);
+		a.mode = ren1->pair->two->mode;
+		hashcpy(b.sha1, ren2->pair->two->sha1);
+		b.mode = ren2->pair->two->mode;
+		mfi = merge_file(o, &one, &a, &b, branch1, branch2);
+		output(o, 1, "Adding merged %s", path);
+		update_file(o, 0, mfi.sha, mfi.mode, path);
+	} else {
+		char *new_path1 = unique_path(o, path, branch1);
+		char *new_path2 = unique_path(o, path, branch2);
+		output(o, 1, "Renaming %s to %s and %s to %s instead",
+		       ren1->pair->one->path, new_path1,
+		       ren2->pair->one->path, new_path2);
+		remove_file(o, 0, path, 0);
+		update_file(o, 0, ren1->pair->two->sha1, ren1->pair->two->mode,
+			    new_path1);
+		update_file(o, 0, ren2->pair->two->sha1, ren2->pair->two->mode,
+			    new_path2);
+		free(new_path2);
+		free(new_path1);
+	}
 }
 
 static int process_renames(struct merge_options *o,
@@ -1008,12 +1027,12 @@ static int process_renames(struct merge_options *o,
 	for (i = 0; i < a_renames->nr; i++) {
 		sre = a_renames->items[i].util;
 		string_list_insert(&a_by_dst, sre->pair->two->path)->util
-			= sre->dst_entry;
+			= (void*)sre;
 	}
 	for (i = 0; i < b_renames->nr; i++) {
 		sre = b_renames->items[i].util;
 		string_list_insert(&b_by_dst, sre->pair->two->path)->util
-			= sre->dst_entry;
+			= (void*)sre;
 	}
 
 	for (i = 0, j = 0; i < a_renames->nr || j < b_renames->nr;) {
@@ -1125,6 +1144,23 @@ static int process_renames(struct merge_options *o,
 					clean_merge = 0;
 					conflict_rename_delete(o, ren1->pair, branch1, branch2);
 				}
+			} else if ((item = string_list_lookup(renames2Dst, ren1_dst))) {
+				char *ren2_src, *ren2_dst;
+				ren2 = item->util;
+				ren2_src = ren2->pair->one->path;
+				ren2_dst = ren2->pair->two->path;
+
+				clean_merge = 0;
+				ren2->processed = 1;
+				remove_file(o, 1, ren2_src,
+					    renamed_stage == 3 || would_lose_untracked(ren1_src));
+
+				output(o, 1, "CONFLICT (rename/rename): "
+				       "Rename %s->%s in %s. "
+				       "Rename %s->%s in %s",
+				       ren1_src, ren1_dst, branch1,
+				       ren2_src, ren2_dst, branch2);
+				conflict_rename_rename_2to1(o, ren1, branch1, ren2, branch2);
 			} else if ((dst_other.mode == ren1->pair->two->mode) &&
 				   sha_eq(dst_other.sha1, ren1->pair->two->sha1)) {
 				/* Added file on the other side
@@ -1165,16 +1201,6 @@ static int process_renames(struct merge_options *o,
 					update_file(o, 0, dst_other.sha1, dst_other.mode, new_path);
 					free(new_path);
 				}
-			} else if ((item = string_list_lookup(renames2Dst, ren1_dst))) {
-				ren2 = item->util;
-				clean_merge = 0;
-				ren2->processed = 1;
-				output(o, 1, "CONFLICT (rename/rename): "
-				       "Rename %s->%s in %s. "
-				       "Rename %s->%s in %s",
-				       ren1_src, ren1_dst, branch1,
-				       ren2->pair->one->path, ren2->pair->two->path, branch2);
-				conflict_rename_rename_2to1(o, ren1, branch1, ren2, branch2);
 			} else
 				try_merge = 1;
 
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index dea2a65..8d1d303 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -60,13 +60,13 @@ test_expect_success 'merge simple rename+criss-cross with no modifications' '
 	test $(git rev-parse :2:three) = $(git rev-parse L2:three) &&
 	test $(git rev-parse :3:three) = $(git rev-parse R2:three) &&
 
-	cp two merged &&
+	cp one merged &&
 	>empty &&
 	test_must_fail git merge-file \
-		-L "Temporary merge branch 2" \
-		-L "" \
 		-L "Temporary merge branch 1" \
-		merged empty one &&
+		-L "" \
+		-L "Temporary merge branch 2" \
+		merged empty two &&
 	test $(git rev-parse :1:three) = $(git hash-object merged)
 '
 
@@ -139,11 +139,12 @@ test_expect_success 'merge criss-cross + rename merges with basic modification'
 	cp one merge-me &&
 	>empty &&
 	test_must_fail git merge-file \
-		-L "Temporary merge branch 2" \
-		-L "" \
 		-L "Temporary merge branch 1" \
-		merged empty merge-me &&
-	test $(git rev-parse :1:three) = $(git hash-object merged)
+		-L "" \
+		-L "Temporary merge branch 2" \
+		merge-me empty merged &&
+
+	test $(git rev-parse :1:three) = $(git hash-object merge-me)
 '
 
 #
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 32/48] merge-recursive: Add comments about handling rename/add-source cases

There are a couple of places where changes are needed to for situations involving rename/add-source issues. Add comments about the needed changes (and existing bugs) until git has been enabled to detect such cases.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)
Show changes to merge-recursive.c +11 −0
diff --git a/merge-recursive.c b/merge-recursive.c
index 4c42838..5ccc59c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1072,6 +1072,9 @@ static int process_renames(struct merge_options *o,
 		}
 
 		ren1->dst_entry->processed = 1;
+		/* BUG: We should only mark src_entry as processed if we
+		 * are not dealing with a rename + add-source case.
+		 */
 		ren1->src_entry->processed = 1;
 
 		if (ren1->processed)
@@ -1098,6 +1101,10 @@ static int process_renames(struct merge_options *o,
 							      ren1->dst_entry,
 							      ren2->dst_entry);
 			} else {
+				/* BUG: We should only remove ren1_src in
+				 * the base stage (think of rename +
+				 * add-source cases).
+				 */
 				remove_file(o, 1, ren1_src, 1);
 				update_entry(ren1->dst_entry,
 					     ren1->pair->one,
@@ -1121,6 +1128,10 @@ static int process_renames(struct merge_options *o,
 			int renamed_stage = a_renames == renames1 ? 2 : 3;
 			int other_stage =   a_renames == renames1 ? 3 : 2;
 
+			/* BUG: We should only remove ren1_src in the base
+			 * stage and in other_stage (think of rename +
+			 * add-source case).
+			 */
 			remove_file(o, 1, ren1_src,
 				    renamed_stage == 2 || !was_tracked(ren1_src));
 
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 33/48] merge-recursive: Improve handling of rename target vs. directory addition

When dealing with file merging and renames and D/F conflicts and possible criss-cross merges (how's that for a corner case?), we did not do a thorough job ensuring the index and working directory had the correct contents. Fix the logic in merge_content() to handle this. Also, correct some erroneous tests in t6022 that were expecting the wrong number of unmerged index entries. These changes fix one of the tests in t6039 (and almost fix another one from t6039 as well).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                    |   27 ++++++++++++++++++++++-----
 t/t6022-merge-rename.sh              |    4 ++--
 t/t6039-merge-rename-corner-cases.sh |    2 +-
 3 files changed, 25 insertions(+), 8 deletions(-)
Show changes to 3 files +25 −8

merge-recursive.c, t/t6022-merge-rename.sh, t/t6039-merge-rename-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 5ccc59c..e6a97b2 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1370,19 +1370,34 @@ static int merge_content(struct merge_options *o,
 			reason = "submodule";
 		output(o, 1, "CONFLICT (%s): Merge conflict in %s",
 				reason, path);
-		if (involved_in_rename)
+		if (involved_in_rename && !df_conflict_remains)
 			update_stages(path, &one, &a, &b);
 	}
 
 	if (df_conflict_remains) {
 		char *new_path;
-		update_file_flags(o, mfi.sha, mfi.mode, path,
-				  o->call_depth || mfi.clean, 0);
+		if (o->call_depth) {
+			remove_file_from_cache(path);
+		} else {
+			if (!mfi.clean)
+				update_stages(path, &one, &a, &b);
+			else {
+				int file_from_stage2 = was_tracked(path);
+				struct diff_filespec merged;
+				hashcpy(merged.sha1, mfi.sha);
+				merged.mode = mfi.mode;
+
+				update_stages(path, NULL,
+					      file_from_stage2 ? &merged : NULL,
+					      file_from_stage2 ? NULL : &merged);
+			}
+
+		}
 		new_path = unique_path(o, path, df_rename_conflict_branch);
-		mfi.clean = 0;
 		output(o, 1, "Adding as %s instead", new_path);
-		update_file_flags(o, mfi.sha, mfi.mode, new_path, 0, 1);
+		update_file(o, 0, mfi.sha, mfi.mode, new_path);
 		free(new_path);
+		mfi.clean = 0;
 	} else {
 		update_file(o, mfi.clean, mfi.sha, mfi.mode, path);
 	}
@@ -1574,6 +1589,8 @@ static int process_df_entry(struct merge_options *o,
 			output(o, 1, "CONFLICT (%s): There is a directory with name %s in %s. "
 			       "Adding %s as %s",
 			       conf, path, other_branch, path, new_path);
+			if (o->call_depth)
+				remove_file_from_cache(path);
 			update_file(o, 0, sha, mode, new_path);
 			if (o->call_depth)
 				remove_file_from_cache(path);
diff --git a/t/t6022-merge-rename.sh b/t/t6022-merge-rename.sh
index 5e2a686..cd1e8fb 100755
--- a/t/t6022-merge-rename.sh
+++ b/t/t6022-merge-rename.sh
@@ -307,7 +307,7 @@ test_expect_success 'Rename+D/F conflict; renamed file merges but dir in way' '
 	grep "Auto-merging dir" output &&
 	grep "Adding as dir~HEAD instead" output &&
 
-	test 2 -eq "$(git ls-files -u | wc -l)" &&
+	test 3 -eq "$(git ls-files -u | wc -l)" &&
 	test 2 -eq "$(git ls-files -u dir/file-in-the-way | wc -l)" &&
 
 	test_must_fail git diff --quiet &&
@@ -329,7 +329,7 @@ test_expect_success 'Same as previous, but merged other way' '
 	grep "Auto-merging dir" output &&
 	grep "Adding as dir~renamed-file-has-no-conflicts instead" output &&
 
-	test 2 -eq "$(git ls-files -u | wc -l)" &&
+	test 3 -eq "$(git ls-files -u | wc -l)" &&
 	test 2 -eq "$(git ls-files -u dir/file-in-the-way | wc -l)" &&
 
 	test_must_fail git diff --quiet &&
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index 4f94528..18e7821 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -234,7 +234,7 @@ test_expect_success 'setup content merge + rename/directory conflict' '
 	git commit -m left
 '
 
-test_expect_failure 'rename/directory conflict + clean content merge' '
+test_expect_success 'rename/directory conflict + clean content merge' '
 	git reset --hard &&
 	git reset --hard &&
 	git clean -fdqx &&
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 34/48] merge-recursive: Consolidate process_entry() and process_df_entry()

The whole point of adding process_df_entry() was to ensure that files of D/F conflicts were processed after paths under the corresponding directory. However, given that the entries are in sorted order, all we need to do is iterate through them in reverse order to achieve the same effect. That lets us remove some duplicated code, and lets us keep track of one less thing as we read the code ("do we need to make sure this is processed before process_df_entry() or do we need to defer it until then?").

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |  194 ++++++++++++++++------------------------------------
 1 files changed, 60 insertions(+), 134 deletions(-)
Show changes to merge-recursive.c +60 −134
diff --git a/merge-recursive.c b/merge-recursive.c
index e6a97b2..5673ccb 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -116,7 +116,6 @@ static inline void setup_rename_df_conflict_info(enum rename_type rename_type,
 		ci->dst_entry2 = dst_entry2;
 		ci->pair2 = pair2;
 		dst_entry2->rename_df_conflict_info = ci;
-		dst_entry2->processed = 0;
 	}
 }
 
@@ -358,20 +357,17 @@ static void record_df_conflict_files(struct merge_options *o,
 				     struct string_list *entries)
 {
 	/* If there is a D/F conflict and the file for such a conflict
-	 * currently exist in the working copy, we want to allow it to
-	 * be removed to make room for the corresponding directory if
-	 * needed.  The files underneath the directories of such D/F
-	 * conflicts will be handled in process_entry(), while the
-	 * files of such D/F conflicts will be processed later in
-	 * process_df_entry().  If the corresponding directory ends up
-	 * being removed by the merge, then no additional work needs
-	 * to be done by process_df_entry() for the conflicting file.
-	 * If the directory needs to be written to the working copy,
-	 * then the conflicting file will simply be removed (e.g. in
-	 * make_room_for_path).  If the directory is written to the
-	 * working copy but the file also has a conflict that needs to
-	 * be resolved, then process_df_entry() will reinstate the
-	 * file with a new unique name.
+	 * currently exist in the working copy, we want to allow it to be
+	 * removed to make room for the corresponding directory if needed.
+	 * The files underneath the directories of such D/F conflicts will
+	 * be processed before the corresponding file involved in the D/F
+	 * conflict.  If the D/F directory ends up being removed by the
+	 * merge, then we won't have to touch the D/F file.  If the D/F
+	 * directory needs to be written to the working copy, then the D/F
+	 * file will simply be removed (in make_room_for_path()) to make
+	 * room for the necessary paths.  Note that if both the directory
+	 * and the file need to be present, then the D/F file will be
+	 * reinstated with a new unique name at the time it is processed.
 	 */
 	const char *last_file = NULL;
 	int last_len = 0;
@@ -1310,17 +1306,17 @@ static void handle_delete_modify(struct merge_options *o,
 		       "and modified in %s. Version %s of %s left in tree%s%s.",
 		       path, o->branch1,
 		       o->branch2, o->branch2, path,
-		       path == new_path ? "" : " at ",
-		       path == new_path ? "" : new_path);
-		update_file(o, 0, b_sha, b_mode, new_path);
+		       NULL == new_path ? "" : " at ",
+		       NULL == new_path ? "" : new_path);
+		update_file(o, 0, b_sha, b_mode, new_path ? new_path : path);
 	} else {
 		output(o, 1, "CONFLICT (delete/modify): %s deleted in %s "
 		       "and modified in %s. Version %s of %s left in tree%s%s.",
 		       path, o->branch2,
 		       o->branch1, o->branch1, path,
-		       path == new_path ? "" : " at ",
-		       path == new_path ? "" : new_path);
-		update_file(o, 0, a_sha, a_mode, new_path);
+		       NULL == new_path ? "" : " at ",
+		       NULL == new_path ? "" : new_path);
+		update_file(o, 0, a_sha, a_mode, new_path ? new_path : path);
 	}
 }
 
@@ -1422,93 +1418,6 @@ static int process_entry(struct merge_options *o,
 	unsigned char *a_sha = stage_sha(entry->stages[2].sha, a_mode);
 	unsigned char *b_sha = stage_sha(entry->stages[3].sha, b_mode);
 
-	if (entry->rename_df_conflict_info)
-		return 1; /* Such cases are handled elsewhere. */
-
-	entry->processed = 1;
-	if (o_sha && (!a_sha || !b_sha)) {
-		/* Case A: Deleted in one */
-		if ((!a_sha && !b_sha) ||
-		    (!b_sha && blob_unchanged(o_sha, a_sha, normalize, path)) ||
-		    (!a_sha && blob_unchanged(o_sha, b_sha, normalize, path))) {
-			/* Deleted in both or deleted in one and
-			 * unchanged in the other */
-			if (a_sha)
-				output(o, 2, "Removing %s", path);
-			/* do not touch working file if it did not exist */
-			remove_file(o, 1, path, !a_sha);
-		} else if (dir_in_way(path, 0 /*check_wc*/)) {
-			entry->processed = 0;
-			return 1; /* Assume clean until processed */
-		} else {
-			/* Deleted in one and changed in the other */
-			clean_merge = 0;
-			handle_delete_modify(o, path, path,
-					     a_sha, a_mode, b_sha, b_mode);
-		}
-
-	} else if ((!o_sha && a_sha && !b_sha) ||
-		   (!o_sha && !a_sha && b_sha)) {
-		/* Case B: Added in one. */
-		unsigned mode;
-		const unsigned char *sha;
-
-		if (a_sha) {
-			mode = a_mode;
-			sha = a_sha;
-		} else {
-			mode = b_mode;
-			sha = b_sha;
-		}
-		if (dir_in_way(path, 0 /*check_wc*/)) {
-			/* Handle D->F conflicts after all subfiles */
-			entry->processed = 0;
-			return 1; /* Assume clean until processed */
-		} else {
-			output(o, 2, "Adding %s", path);
-			update_file(o, 1, sha, mode, path);
-		}
-	} else if (a_sha && b_sha) {
-		/* Case C: Added in both (check for same permissions) and */
-		/* case D: Modified in both, but differently. */
-		clean_merge = merge_content(o, entry->involved_in_rename, path,
-					    o_sha, o_mode, a_sha, a_mode, b_sha, b_mode,
-					    NULL);
-	} else if (!o_sha && !a_sha && !b_sha) {
-		/*
-		 * this entry was deleted altogether. a_mode == 0 means
-		 * we had that path and want to actively remove it.
-		 */
-		remove_file(o, 1, path, !a_mode);
-	} else
-		die("Fatal merge failure, shouldn't happen.");
-
-	return clean_merge;
-}
-
-/*
- * Per entry merge function for D/F (and/or rename) conflicts.  In the
- * cases we can cleanly resolve D/F conflicts, process_entry() can
- * clean out all the files below the directory for us.  All D/F
- * conflict cases must be handled here at the end to make sure any
- * directories that can be cleaned out, are.
- *
- * Some rename conflicts may also be handled here that don't necessarily
- * involve D/F conflicts, since the code to handle them is generic enough
- * to handle those rename conflicts with or without D/F conflicts also
- * being involved.
- */
-static int process_df_entry(struct merge_options *o,
-			    const char *path, struct stage_data *entry)
-{
-	int clean_merge = 1;
-	unsigned o_mode = entry->stages[1].mode;
-	unsigned a_mode = entry->stages[2].mode;
-	unsigned b_mode = entry->stages[3].mode;
-	unsigned char *o_sha = stage_sha(entry->stages[1].sha, o_mode);
-	unsigned char *a_sha = stage_sha(entry->stages[2].sha, a_mode);
-	unsigned char *b_sha = stage_sha(entry->stages[3].sha, b_mode);
-
 	entry->processed = 1;
 	if (entry->rename_df_conflict_info) {
 		struct rename_df_conflict_info *conflict_info = entry->rename_df_conflict_info;
@@ -1543,27 +1452,41 @@ static int process_df_entry(struct merge_options *o,
 						    conflict_info->branch1,
 						    conflict_info->pair2,
 						    conflict_info->branch2);
-			conflict_info->dst_entry2->processed = 1;
 			break;
 		default:
 			entry->processed = 0;
 			break;
 		}
 	} else if (o_sha && (!a_sha || !b_sha)) {
-		/* Modify/delete; deleted side may have put a directory in the way */
-		char *new_path;
-		int free_me = 0;
-		if (dir_in_way(path, !o->call_depth)) {
-			new_path = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
-			free_me = 1;
+		/* Case A: Deleted in one */
+		if ((!a_sha && !b_sha) ||
+		    (!b_sha && blob_unchanged(o_sha, a_sha, normalize, path)) ||
+		    (!a_sha && blob_unchanged(o_sha, b_sha, normalize, path))) {
+			/* Deleted in both or deleted in one and
+			 * unchanged in the other */
+			if (a_sha)
+				output(o, 2, "Removing %s", path);
+			/* do not touch working file if it did not exist */
+			remove_file(o, 1, path, !a_sha);
+		} else {
+			/* Modify/delete; deleted side may have put a directory in the way */
+			char *new_path = NULL;
+			int free_me = 0;
+			clean_merge = 0;
+			if (dir_in_way(path, !o->call_depth)) {
+				new_path = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
+				free_me = 1;
+			}
+			handle_delete_modify(o, path, new_path,
+					     a_sha, a_mode, b_sha, b_mode);
+			if (free_me)
+				free(new_path);
 		}
-		clean_merge = 0;
-		handle_delete_modify(o, path, free_me ? new_path : path,
-				     a_sha, a_mode, b_sha, b_mode);
-		if (free_me)
-			free(new_path);
-	} else if (!o_sha && !!a_sha != !!b_sha) {
-		/* directory -> (directory, file) or <nothing> -> (directory, file) */
+	} else if ((!o_sha && a_sha && !b_sha) ||
+		   (!o_sha && !a_sha && b_sha)) {
+		/* Case B: Added in one. */
+		/* [nothing|directory] -> ([nothing|directory], file) */
+
 		const char *add_branch;
 		const char *other_branch;
 		unsigned mode;
@@ -1599,10 +1522,20 @@ static int process_df_entry(struct merge_options *o,
 			output(o, 2, "Adding %s", path);
 			update_file(o, 1, sha, mode, path);
 		}
-	} else {
-		entry->processed = 0;
-		return 1; /* not handled; assume clean until processed */
-	}
+	} else if (a_sha && b_sha) {
+		/* Case C: Added in both (check for same permissions) and */
+		/* case D: Modified in both, but differently. */
+		clean_merge = merge_content(o, entry->involved_in_rename, path,
+					    o_sha, o_mode, a_sha, a_mode, b_sha, b_mode,
+					    NULL);
+	} else if (!o_sha && !a_sha && !b_sha) {
+		/*
+		 * this entry was deleted altogether. a_mode == 0 means
+		 * we had that path and want to actively remove it.
+		 */
+		remove_file(o, 1, path, !a_mode);
+	} else
+		die("Fatal merge failure, shouldn't happen.");
 
 	return clean_merge;
 }
@@ -1650,7 +1583,7 @@ int merge_trees(struct merge_options *o,
 		re_head  = get_renames(o, head, common, head, merge, entries);
 		re_merge = get_renames(o, merge, common, head, merge, entries);
 		clean = process_renames(o, re_head, re_merge);
-		for (i = 0; i < entries->nr; i++) {
+		for (i = entries->nr-1; i >=0; i--) {
 			const char *path = entries->items[i].string;
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed
@@ -1658,13 +1591,6 @@ int merge_trees(struct merge_options *o,
 				clean = 0;
 		}
 		for (i = 0; i < entries->nr; i++) {
-			const char *path = entries->items[i].string;
-			struct stage_data *e = entries->items[i].util;
-			if (!e->processed
-				&& !process_df_entry(o, path, e))
-				clean = 0;
-		}
-		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
 				die("Unprocessed path??? %s",
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 21, 2011, 18:43 UTC · re: Elijah Newren · lore

Re: [PATCH 34/48] merge-recursive: Consolidate process_entry() and process_df_entry()

Elijah Newren <newren@gmail.com> writes:
Show 13 quoted lines
> The whole point of adding process_df_entry() was to ensure that files of
> D/F conflicts were processed after paths under the corresponding
> directory.  However, given that the entries are in sorted order, all we
> need to do is iterate through them in reverse order to achieve the same
> effect.  That lets us remove some duplicated code, and lets us keep
> track of one less thing as we read the code ("do we need to make sure
> this is processed before process_df_entry() or do we need to defer it
> until then?").
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c |  194 ++++++++++++++++------------------------------------
>  1 files changed, 60 insertions(+), 134 deletions(-)
;-).
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 35/48] merge-recursive: Cleanup and consolidation of rename_conflict_info

The consolidation of process_entry() and process_df_entry() allows us to consolidate more code paths concerning rename conflicts, and to do a few additional related cleanups. It also means we are using rename_df_conflict_info in some cases where there is no D/F conflict; rename it to rename_conflict_info.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |  134 ++++++++++++++++++++++++++---------------------------
 1 files changed, 66 insertions(+), 68 deletions(-)
Show changes to merge-recursive.c +66 −68
diff --git a/merge-recursive.c b/merge-recursive.c
index 5673ccb..5d0a62c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -66,10 +66,11 @@ static int sha_eq(const unsigned char *a, const unsigned char *b)
 enum rename_type {
 	RENAME_NORMAL = 0,
 	RENAME_DELETE,
+	RENAME_ONE_FILE_TO_ONE,
 	RENAME_ONE_FILE_TO_TWO
 };
 
-struct rename_df_conflict_info {
+struct rename_conflict_info {
 	enum rename_type rename_type;
 	struct diff_filepair *pair1;
 	struct diff_filepair *pair2;
@@ -88,34 +89,33 @@ struct stage_data {
 		unsigned mode;
 		unsigned char sha[20];
 	} stages[4];
-	struct rename_df_conflict_info *rename_df_conflict_info;
+	struct rename_conflict_info *rename_conflict_info;
 	unsigned processed:1;
-	unsigned involved_in_rename:1;
 };
 
-static inline void setup_rename_df_conflict_info(enum rename_type rename_type,
-						 struct diff_filepair *pair1,
-						 struct diff_filepair *pair2,
-						 const char *branch1,
-						 const char *branch2,
-						 struct stage_data *dst_entry1,
-						 struct stage_data *dst_entry2)
+static inline void setup_rename_conflict_info(enum rename_type rename_type,
+					      struct diff_filepair *pair1,
+					      struct diff_filepair *pair2,
+					      const char *branch1,
+					      const char *branch2,
+					      struct stage_data *dst_entry1,
+					      struct stage_data *dst_entry2)
 {
-	struct rename_df_conflict_info *ci = xcalloc(1, sizeof(struct rename_df_conflict_info));
+	struct rename_conflict_info *ci = xcalloc(1, sizeof(struct rename_conflict_info));
 	ci->rename_type = rename_type;
 	ci->pair1 = pair1;
 	ci->branch1 = branch1;
 	ci->branch2 = branch2;
 
 	ci->dst_entry1 = dst_entry1;
-	dst_entry1->rename_df_conflict_info = ci;
+	dst_entry1->rename_conflict_info = ci;
 	dst_entry1->processed = 0;
 
 	assert(!pair2 == !dst_entry2);
 	if (dst_entry2) {
 		ci->dst_entry2 = dst_entry2;
 		ci->pair2 = pair2;
-		dst_entry2->rename_df_conflict_info = ci;
+		dst_entry2->rename_conflict_info = ci;
 	}
 }
 
@@ -939,10 +939,29 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 	/* One file was renamed in both branches, but to different names. */
 	char *del[2];
 	int delp = 0;
+	const char *src      = pair1->one->path;
 	const char *ren1_dst = pair1->two->path;
 	const char *ren2_dst = pair2->two->path;
 	const char *dst_name1 = ren1_dst;
 	const char *dst_name2 = ren2_dst;
+
+	output(o, 1, "CONFLICT (rename/rename): "
+	       "Rename \"%s\"->\"%s\" in branch \"%s\" "
+	       "rename \"%s\"->\"%s\" in \"%s\"%s",
+	       src, pair1->two->path, branch1,
+	       src, pair2->two->path, branch2,
+	       o->call_depth ? " (left unresolved)" : "");
+	if (o->call_depth) {
+		/*
+		 * FIXME: Why remove file from cache, and then
+		 * immediately readd it?  Why not just overwrite using
+		 * update_file only?  Also...this is buggy for
+		 * rename/add-source situations...
+		 */
+		remove_file_from_cache(src);
+		update_file(o, 0, pair1->one->sha1, pair1->one->mode, src);
+	}
+
 	if (dir_in_way(ren1_dst, !o->call_depth)) {
 		dst_name1 = del[delp++] = unique_path(o, ren1_dst, branch1);
 		output(o, 1, "%s is a directory in %s adding as %s instead",
@@ -1083,20 +1102,16 @@ static int process_renames(struct merge_options *o,
 		if (ren2) {
 			const char *ren2_src = ren2->pair->one->path;
 			const char *ren2_dst = ren2->pair->two->path;
+			enum rename_type rename_type;
 			/* Renamed in 1 and renamed in 2 */
 			if (strcmp(ren1_src, ren2_src) != 0)
 				die("ren1.src != ren2.src");
 			ren2->dst_entry->processed = 1;
 			ren2->processed = 1;
 			if (strcmp(ren1_dst, ren2_dst) != 0) {
-				setup_rename_df_conflict_info(RENAME_ONE_FILE_TO_TWO,
-							      ren1->pair,
-							      ren2->pair,
-							      branch1,
-							      branch2,
-							      ren1->dst_entry,
-							      ren2->dst_entry);
+				rename_type = RENAME_ONE_FILE_TO_TWO;
 			} else {
+				rename_type = RENAME_ONE_FILE_TO_ONE;
 				/* BUG: We should only remove ren1_src in
 				 * the base stage (think of rename +
 				 * add-source cases).
@@ -1106,8 +1121,14 @@ static int process_renames(struct merge_options *o,
 					     ren1->pair->one,
 					     ren1->pair->two,
 					     ren2->pair->two);
-				ren1->dst_entry->involved_in_rename = 1;
 			}
+			setup_rename_conflict_info(rename_type,
+						   ren1->pair,
+						   ren2->pair,
+						   branch1,
+						   branch2,
+						   ren1->dst_entry,
+						   ren2->dst_entry);
 		} else {
 			/* Renamed in 1, maybe changed in 2 */
 			struct string_list_item *item;
@@ -1138,19 +1159,13 @@ static int process_renames(struct merge_options *o,
 			try_merge = 0;
 
 			if (sha_eq(src_other.sha1, null_sha1)) {
-				if (dir_in_way(ren1_dst, 0 /*check_wc*/)) {
-					ren1->dst_entry->processed = 0;
-					setup_rename_df_conflict_info(RENAME_DELETE,
-								      ren1->pair,
-								      NULL,
-								      branch1,
-								      branch2,
-								      ren1->dst_entry,
-								      NULL);
-				} else {
-					clean_merge = 0;
-					conflict_rename_delete(o, ren1->pair, branch1, branch2);
-				}
+				setup_rename_conflict_info(RENAME_DELETE,
+							   ren1->pair,
+							   NULL,
+							   branch1,
+							   branch2,
+							   ren1->dst_entry,
+							   NULL);
 			} else if ((item = string_list_lookup(renames2Dst, ren1_dst))) {
 				char *ren2_src, *ren2_dst;
 				ren2 = item->util;
@@ -1224,16 +1239,13 @@ static int process_renames(struct merge_options *o,
 					a = &src_other;
 				}
 				update_entry(ren1->dst_entry, one, a, b);
-				ren1->dst_entry->involved_in_rename = 1;
-				if (dir_in_way(ren1_dst, 0 /*check_wc*/)) {
-					setup_rename_df_conflict_info(RENAME_NORMAL,
-								      ren1->pair,
-								      NULL,
-								      branch1,
-								      NULL,
-								      ren1->dst_entry,
-								      NULL);
-				}
+				setup_rename_conflict_info(RENAME_NORMAL,
+							   ren1->pair,
+							   NULL,
+							   branch1,
+							   NULL,
+							   ren1->dst_entry,
+							   NULL);
 			}
 		}
 	}
@@ -1321,12 +1333,11 @@ static void handle_delete_modify(struct merge_options *o,
 }
 
 static int merge_content(struct merge_options *o,
-			 unsigned involved_in_rename,
 			 const char *path,
 			 unsigned char *o_sha, int o_mode,
 			 unsigned char *a_sha, int a_mode,
 			 unsigned char *b_sha, int b_mode,
-			 const char *df_rename_conflict_branch)
+			 struct rename_conflict_info *rename_conflict_info)
 {
 	const char *reason = "content";
 	struct merge_file_info mfi;
@@ -1346,8 +1357,7 @@ static int merge_content(struct merge_options *o,
 	b.mode = b_mode;
 
 	mfi = merge_file(o, &one, &a, &b, o->branch1, o->branch2);
-	if (df_rename_conflict_branch &&
-	    dir_in_way(path, !o->call_depth)) {
+	if (rename_conflict_info && dir_in_way(path, !o->call_depth)) {
 		df_conflict_remains = 1;
 	}
 
@@ -1366,7 +1376,7 @@ static int merge_content(struct merge_options *o,
 			reason = "submodule";
 		output(o, 1, "CONFLICT (%s): Merge conflict in %s",
 				reason, path);
-		if (involved_in_rename && !df_conflict_remains)
+		if (rename_conflict_info && !df_conflict_remains)
 			update_stages(path, &one, &a, &b);
 	}
 
@@ -1389,7 +1399,7 @@ static int merge_content(struct merge_options *o,
 			}
 
 		}
-		new_path = unique_path(o, path, df_rename_conflict_branch);
+		new_path = unique_path(o, path, rename_conflict_info->branch1);
 		output(o, 1, "Adding as %s instead", new_path);
 		update_file(o, 0, mfi.sha, mfi.mode, new_path);
 		free(new_path);
@@ -1419,14 +1429,14 @@ static int process_entry(struct merge_options *o,
 	unsigned char *b_sha = stage_sha(entry->stages[3].sha, b_mode);
 
 	entry->processed = 1;
-	if (entry->rename_df_conflict_info) {
-		struct rename_df_conflict_info *conflict_info = entry->rename_df_conflict_info;
-		char *src;
+	if (entry->rename_conflict_info) {
+		struct rename_conflict_info *conflict_info = entry->rename_conflict_info;
 		switch (conflict_info->rename_type) {
 		case RENAME_NORMAL:
-			clean_merge = merge_content(o, entry->involved_in_rename, path,
+		case RENAME_ONE_FILE_TO_ONE:
+			clean_merge = merge_content(o, path,
 						    o_sha, o_mode, a_sha, a_mode, b_sha, b_mode,
-						    conflict_info->branch1);
+						    conflict_info);
 			break;
 		case RENAME_DELETE:
 			clean_merge = 0;
@@ -1435,19 +1445,7 @@ static int process_entry(struct merge_options *o,
 					       conflict_info->branch2);
 			break;
 		case RENAME_ONE_FILE_TO_TWO:
-			src = conflict_info->pair1->one->path;
 			clean_merge = 0;
-			output(o, 1, "CONFLICT (rename/rename): "
-			       "Rename \"%s\"->\"%s\" in branch \"%s\" "
-			       "rename \"%s\"->\"%s\" in \"%s\"%s",
-			       src, conflict_info->pair1->two->path, conflict_info->branch1,
-			       src, conflict_info->pair2->two->path, conflict_info->branch2,
-			       o->call_depth ? " (left unresolved)" : "");
-			if (o->call_depth) {
-				remove_file_from_cache(src);
-				update_file(o, 0, conflict_info->pair1->one->sha1,
-					    conflict_info->pair1->one->mode, src);
-			}
 			conflict_rename_rename_1to2(o, conflict_info->pair1,
 						    conflict_info->branch1,
 						    conflict_info->pair2,
@@ -1525,7 +1523,7 @@ static int process_entry(struct merge_options *o,
 	} else if (a_sha && b_sha) {
 		/* Case C: Added in both (check for same permissions) and */
 		/* case D: Modified in both, but differently. */
-		clean_merge = merge_content(o, entry->involved_in_rename, path,
+		clean_merge = merge_content(o, path,
 					    o_sha, o_mode, a_sha, a_mode, b_sha, b_mode,
 					    NULL);
 	} else if (!o_sha && !a_sha && !b_sha) {
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 36/48] merge-recursive: Provide more info in conflict markers with file renames

Whenever there are merge conflicts in file contents, we would mark the different sides of the conflict with the two branches being merged. However, when there is a rename involved as well, the branchname is not sufficient to specify where the conflicting content came from. In such cases, mark the two sides of the conflict with branchname:filename rather than just branchname.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                    |   20 +++++++++++++++++---
 t/t6022-merge-rename.sh              |    8 ++++----
 t/t6039-merge-rename-corner-cases.sh |    2 +-
 3 files changed, 22 insertions(+), 8 deletions(-)
Show changes to 3 files +22 −8

merge-recursive.c, t/t6022-merge-rename.sh, t/t6039-merge-rename-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 5d0a62c..da507a3 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1340,6 +1340,7 @@ static int merge_content(struct merge_options *o,
 			 struct rename_conflict_info *rename_conflict_info)
 {
 	const char *reason = "content";
+	char *side1 = NULL, *side2 = NULL;
 	struct merge_file_info mfi;
 	struct diff_filespec one, a, b;
 	unsigned df_conflict_remains = 0;
@@ -1356,10 +1357,23 @@ static int merge_content(struct merge_options *o,
 	hashcpy(b.sha1, b_sha);
 	b.mode = b_mode;
 
-	mfi = merge_file(o, &one, &a, &b, o->branch1, o->branch2);
-	if (rename_conflict_info && dir_in_way(path, !o->call_depth)) {
-		df_conflict_remains = 1;
+	if (rename_conflict_info) {
+		const char *path1 = rename_conflict_info->pair2 ?
+				    rename_conflict_info->pair2->one->path : path;
+		const char *path2 = rename_conflict_info->pair1 ?
+				    rename_conflict_info->pair1->one->path : path;
+		side1 = xmalloc(strlen(o->branch1) + strlen(path1) + 2);
+		side2 = xmalloc(strlen(o->branch2) + strlen(path2) + 2);
+		sprintf(side1, "%s:%s", o->branch1, path1);
+		sprintf(side2, "%s:%s", o->branch2, path2);
+
+		if (dir_in_way(path, !o->call_depth))
+			df_conflict_remains = 1;
 	}
+	mfi = merge_file(o, &one, &a, &b,
+			 side1 ? side1:o->branch1, side2 ? side2:o->branch2);
+	free(side1);
+	free(side2);
 
 	if (mfi.clean && !df_conflict_remains &&
 	    sha_eq(mfi.sha, a_sha) && mfi.mode == a.mode &&
diff --git a/t/t6022-merge-rename.sh b/t/t6022-merge-rename.sh
index cd1e8fb..cfce3d3 100755
--- a/t/t6022-merge-rename.sh
+++ b/t/t6022-merge-rename.sh
@@ -351,11 +351,11 @@ cat >expected <<\EOF &&
 8
 9
 10
-<<<<<<< HEAD
+<<<<<<< HEAD:dir
 12
 =======
 11
->>>>>>> dir-not-in-way
+>>>>>>> dir-not-in-way:sub/file
 EOF
 
 test_expect_success 'Rename+D/F conflict; renamed file cannot merge, dir not in way' '
@@ -405,11 +405,11 @@ cat >expected <<\EOF &&
 8
 9
 10
-<<<<<<< HEAD
+<<<<<<< HEAD:dir
 11
 =======
 12
->>>>>>> renamed-file-has-conflicts
+>>>>>>> renamed-file-has-conflicts:sub/file
 EOF
 
 test_expect_success 'Same as previous, but merged other way' '
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index 18e7821..4d2fd10 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -258,7 +258,7 @@ test_expect_success 'rename/directory conflict + clean content merge' '
 	test -f newfile~HEAD
 '
 
-test_expect_failure 'rename/directory conflict + content merge conflict' '
+test_expect_success 'rename/directory conflict + content merge conflict' '
 	git reset --hard &&
 	git reset --hard &&
 	git clean -fdqx &&
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 21, 2011, 18:43 UTC · re: Elijah Newren · lore

Re: [PATCH 36/48] merge-recursive: Provide more info in conflict markers with file renames

Elijah Newren <newren@gmail.com> writes:
Show 8 quoted lines
> Whenever there are merge conflicts in file contents, we would mark the
> different sides of the conflict with the two branches being merged.
> However, when there is a rename involved as well, the branchname is not
> sufficient to specify where the conflicting content came from.  In such
> cases, mark the two sides of the conflict with branchname:filename rather
> than just branchname.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
Nice.
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 37/48] merge-recursive: Fix modify/delete resolution in the recursive case

When o->call_depth>0 and we have conflicts, we try to find "middle ground" when creating the virtual merge base. In the case of content conflicts, this can be done by doing a three-way content merge and using the result. In all parts where the three-way content merge is clean, it is the correct middle ground, and in parts where it conflicts there is no middle ground but the conflict markers provide a good compromise since they are unlikely to accidentally match any further changes.

In the case of a modify/delete conflict, we cannot do the same thing. Accepting either endpoint as the resolution for the virtual merge base runs the risk that when handling the non-recursive case we will silently accept one person's resolution over another without flagging a conflict. In this case, the closest "middle ground" we have is actually the merge base of the candidate merge bases. (We could alternatively attempt a three way content merge using an empty file in place of the deleted file, but that seems to be more work than necessary.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                 |   32 +++++++++++++++++++++-----------
 t/t6036-recursive-corner-cases.sh |    4 ++--
 2 files changed, 23 insertions(+), 13 deletions(-)
Show changes to 2 files +23 −13

merge-recursive.c, t/t6036-recursive-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index da507a3..5a70a87 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1309,11 +1309,26 @@ error_return:
 
 static void handle_delete_modify(struct merge_options *o,
 				 const char *path,
-				 const char *new_path,
+				 unsigned char *o_sha, int o_mode,
 				 unsigned char *a_sha, int a_mode,
 				 unsigned char *b_sha, int b_mode)
 {
-	if (!a_sha) {
+	char *new_path = NULL;
+	int free_me = 0;
+	if (dir_in_way(path, !o->call_depth)) {
+		new_path = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
+		free_me = 1;
+	}
+
+	if (o->call_depth) {
+		/*
+		 * We cannot arbitrarily accept either a_sha or b_sha as
+		 * correct; since there is no true "middle point" between
+		 * them, simply reuse the base version for virtual merge base.
+		 */
+		remove_file_from_cache(path);
+		update_file(o, 0, o_sha, o_mode, new_path ? new_path : path);
+	} else if (!a_sha) {
 		output(o, 1, "CONFLICT (delete/modify): %s deleted in %s "
 		       "and modified in %s. Version %s of %s left in tree%s%s.",
 		       path, o->branch1,
@@ -1330,6 +1345,9 @@ static void handle_delete_modify(struct merge_options *o,
 		       NULL == new_path ? "" : new_path);
 		update_file(o, 0, a_sha, a_mode, new_path ? new_path : path);
 	}
+	if (free_me)
+		free(new_path);
+
 }
 
 static int merge_content(struct merge_options *o,
@@ -1482,17 +1500,9 @@ static int process_entry(struct merge_options *o,
 			remove_file(o, 1, path, !a_sha);
 		} else {
 			/* Modify/delete; deleted side may have put a directory in the way */
-			char *new_path = NULL;
-			int free_me = 0;
 			clean_merge = 0;
-			if (dir_in_way(path, !o->call_depth)) {
-				new_path = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
-				free_me = 1;
-			}
-			handle_delete_modify(o, path, new_path,
+			handle_delete_modify(o, path, o_sha, o_mode,
 					     a_sha, a_mode, b_sha, b_mode);
-			if (free_me)
-				free(new_path);
 		}
 	} else if ((!o_sha && a_sha && !b_sha) ||
 		   (!o_sha && !a_sha && b_sha)) {
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index 8d1d303..d27477b 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -290,7 +290,7 @@ test_expect_success 'setup criss-cross + modify/delete resolved differently' '
 	git tag E
 '
 
-test_expect_failure 'git detects conflict merging criss-cross+modify/delete' '
+test_expect_success 'git detects conflict merging criss-cross+modify/delete' '
 	git checkout D^0 &&
 
 	test_must_fail git merge -s recursive E^0 &&
@@ -302,7 +302,7 @@ test_expect_failure 'git detects conflict merging criss-cross+modify/delete' '
 	test $(git rev-parse :2:file) = $(git rev-parse B:file)
 '
 
-test_expect_failure 'git detects conflict merging criss-cross+modify/delete, reverse direction' '
+test_expect_success 'git detects conflict merging criss-cross+modify/delete, reverse direction' '
 	git reset --hard &&
 	git checkout E^0 &&
 
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 21, 2011, 18:43 UTC · re: Elijah Newren · lore

Re: [PATCH 37/48] merge-recursive: Fix modify/delete resolution in the recursive case

Elijah Newren <newren@gmail.com> writes:
Show 16 quoted lines
> When o->call_depth>0 and we have conflicts, we try to find "middle ground"
> when creating the virtual merge base.  In the case of content conflicts,
> this can be done by doing a three-way content merge and using the result.
> In all parts where the three-way content merge is clean, it is the correct
> middle ground, and in parts where it conflicts there is no middle ground
> but the conflict markers provide a good compromise since they are unlikely
> to accidentally match any further changes.
>
> In the case of a modify/delete conflict, we cannot do the same thing.
> Accepting either endpoint as the resolution for the virtual merge base
> runs the risk that when handling the non-recursive case we will silently
> accept one person's resolution over another without flagging a conflict.
> In this case, the closest "middle ground" we have is actually the merge
> base of the candidate merge bases.  (We could alternatively attempt a
> three way content merge using an empty file in place of the deleted file,
> but that seems to be more work than necessary.)
What did we use before this patch as the "middle ground"?

Doesn't the "middle ground" also need to contain information from both sides? At the content level, The half-merge result with conflict marker contains all the necessary information on what both sides agreed to do outside the marker area, while showing what each side wished to do inside, so as you describe, it is a good "middle ground".

Using the merge-base would mean that at the tree-structural level of this merge you ignore the wish of one side that wanted to delete the path, and then at the content level you also ignore the wish of both sides (the side that wanted to delete the path wanted to leave zero content, while the other side wanted to modify from the version in merge-base, all of which is ignored by the above argument).

When you want to make a criss-corss merge between E and F that looks like this:

      A---C---E---?
     / \ /       /
    O   .       /
     \ / \     /
      B---D---F

if there is no deletion, we run a content level merge between C and D by using X that is a potentially conflicting merge between A and B as a "virtual" ancestor.

      A---C---E---?
     / \ /       /
    O   X       /
     \ / \     /
      B---D---F

X would contain both what both A and B agreed to do on top of O, and what A and B wanted to do that they do not agree with. The differing opinion of A and B are recorded inside the conflict markers. The change that turns X into E contains C's opinion (i.e. it would likely agree to take what both A and B agreed to do, and it may agree with A and the result in E would resemble what A wanted to bring to the result despite B's objection). If C and D both agree how to resolve the conflict, then B's "objection" will cancel out from the three-way merge between E and F that uses X as the ancestor.

In a delete/modify situation (e.g. A modified while B deleted), there are three possibilities:

 * C and D both agree to delete the path;
 * C and D both agree to keep the path, with modifications that may or may
   not conflict (which can be handled by the usual content-level merge);
 * C decides to delete, while D decides to modify (or vice versa).

And in the last case, the outer merge ? needs to decide if it wants to keep or delete the path anyway, so a simplest solution is to punt the whole recursive business and make it the responsibility of the user to resolve it as a merge between E and F using O as the common ancestor. This patch does so in all three cases.

I however wonder if we can do better in the second case (I do not think the first case would come into the picture, as we would not see such a path when merging E and F as it would have been long gone from both branches). We wouldn't know which commits C and D are exactly, but we do have E and F. If A modified the path and B deleted it, and C and D both decided to keep the path and E and F both inherited that path, wouldn't it be fair to say that what both branches wanted to do is closer to what A did than what B did? In other words, instead of using O, wouldn't it give us a better result if we used A (the side that did not delete) as the common ancestor for the content level merge when both E and F has the path?

Show 16 quoted lines
> diff --git a/merge-recursive.c b/merge-recursive.c
> index da507a3..5a70a87 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -1309,11 +1309,26 @@ error_return:
>  
>  static void handle_delete_modify(struct merge_options *o,
>  				 const char *path,
> -				 const char *new_path,
> +				 unsigned char *o_sha, int o_mode,
>  				 unsigned char *a_sha, int a_mode,
>  				 unsigned char *b_sha, int b_mode)
>  {
> -	if (!a_sha) {
> +	char *new_path = NULL;
> +	int free_me = 0;
No need for free_me anymore, no?
Show 24 quoted lines
> +	if (dir_in_way(path, !o->call_depth)) {
> +		new_path = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
> +		free_me = 1;
> +	}
> +
> +	if (o->call_depth) {
> +		/*
> +		 * We cannot arbitrarily accept either a_sha or b_sha as
> +		 * correct; since there is no true "middle point" between
> +		 * them, simply reuse the base version for virtual merge base.
> +		 */
> +		remove_file_from_cache(path);
> +		update_file(o, 0, o_sha, o_mode, new_path ? new_path : path);
> +	} else if (!a_sha) {
>  		output(o, 1, "CONFLICT (delete/modify): %s deleted in %s "
>  		       "and modified in %s. Version %s of %s left in tree%s%s.",
>  		       path, o->branch1,
> @@ -1330,6 +1345,9 @@ static void handle_delete_modify(struct merge_options *o,
>  		       NULL == new_path ? "" : new_path);
>  		update_file(o, 0, a_sha, a_mode, new_path ? new_path : path);
>  	}
> +	if (free_me)
> +		free(new_path);
> +
... as new_path is non-NULL only when we allocated it.
Elijah Newren· Aug 8, 2011, 22:09 UTC · re: Junio C Hamano · lore

Re: [PATCH 37/48] merge-recursive: Fix modify/delete resolution in the recursive case

On Thu, Jul 21, 2011 at 12:43 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 20 quoted lines
> Elijah Newren <newren@gmail.com> writes:
>
>> When o->call_depth>0 and we have conflicts, we try to find "middle ground"
>> when creating the virtual merge base.  In the case of content conflicts,
>> this can be done by doing a three-way content merge and using the result.
>> In all parts where the three-way content merge is clean, it is the correct
>> middle ground, and in parts where it conflicts there is no middle ground
>> but the conflict markers provide a good compromise since they are unlikely
>> to accidentally match any further changes.
>>
>> In the case of a modify/delete conflict, we cannot do the same thing.
>> Accepting either endpoint as the resolution for the virtual merge base
>> runs the risk that when handling the non-recursive case we will silently
>> accept one person's resolution over another without flagging a conflict.
>> In this case, the closest "middle ground" we have is actually the merge
>> base of the candidate merge bases.  (We could alternatively attempt a
>> three way content merge using an empty file in place of the deleted file,
>> but that seems to be more work than necessary.)
>
> What did we use before this patch as the "middle ground"?

We used precisely what you seem to suggest at the end of your email. It's a trade-off, and I agree with you about the cases where my new choice is suboptimal, but I personally think the original choice has worse trade-offs.

Show 12 quoted lines
> Doesn't the "middle ground" also need to contain information from both
> sides?  At the content level, The half-merge result with conflict marker
> contains all the necessary information on what both sides agreed to do
> outside the marker area, while showing what each side wished to do inside,
> so as you describe, it is a good "middle ground".
>
> Using the merge-base would mean that at the tree-structural level of this
> merge you ignore the wish of one side that wanted to delete the path, and
> then at the content level you also ignore the wish of both sides (the side
> that wanted to delete the path wanted to leave zero content, while the
> other side wanted to modify from the version in merge-base, all of which
> is ignored by the above argument).

I think it is optimal if the "middle ground" can contain information from both sides, but I simply do not see how it is possible in either a modify/delete or rename/delete case. That is, at least not without some kind of significant change to the index data structures or allowing a "virtual merge base" to be an index with higher stage merge entries instead of just a tree (though that would introduce several new complications).

And yes, I am essentially ignoring the intents of both sides, and intentionally so. I thought I addressed that in the arguments above, but perhaps I could have been more clear and explicit about that. Your explanation below provides a great framework for clarifying why I made this change, so let me respond below using the framework you have beautifully illustrated and explained.

Show 38 quoted lines
> When you want to make a criss-corss merge between E and F that looks like
> this:
>
>      A---C---E---?
>     / \ /       /
>    O   .       /
>     \ / \     /
>      B---D---F
>
> if there is no deletion, we run a content level merge between C and D by
> using X that is a potentially conflicting merge between A and B as a
> "virtual" ancestor.
>
>      A---C---E---?
>     / \ /       /
>    O   X       /
>     \ / \     /
>      B---D---F
>
> X would contain both what both A and B agreed to do on top of O, and what
> A and B wanted to do that they do not agree with. The differing opinion of
> A and B are recorded inside the conflict markers.  The change that turns X
> into E contains C's opinion (i.e. it would likely agree to take what both
> A and B agreed to do, and it may agree with A and the result in E would
> resemble what A wanted to bring to the result despite B's objection). If C
> and D both agree how to resolve the conflict, then B's "objection" will
> cancel out from the three-way merge between E and F that uses X as the
> ancestor.
>
> In a delete/modify situation (e.g. A modified while B deleted), there are
> three possibilities:
>
>  * C and D both agree to delete the path;
>
>  * C and D both agree to keep the path, with modifications that may or may
>   not conflict (which can be handled by the usual content-level merge);
>
>  * C decides to delete, while D decides to modify (or vice versa).

This is a really nicely written explanation. I particularly like this list of three cases. Let me augment this list of situations (which I'll number (1)-(3)) with a list of four choices for virtual merge base X:

(J) Make X not have any content at the path (i.e. accept the deletion in B)
(K) Make X have the content from O at path (i.e. what my patch does)
(L) Make X have the content from A at path (which we did before)
(M) Make X have some other content at path

I'll get back to these 12 situations (3 cases each with how any of 4 choices could affect them) in a minute...

Show 17 quoted lines
> And in the last case, the outer merge ? needs to decide if it wants to
> keep or delete the path anyway, so a simplest solution is to punt the
> whole recursive business and make it the responsibility of the user to
> resolve it as a merge between E and F using O as the common ancestor.
> This patch does so in all three cases.
>
> I however wonder if we can do better in the second case (I do not think
> the first case would come into the picture, as we would not see such a
> path when merging E and F as it would have been long gone from both
> branches). We wouldn't know which commits C and D are exactly, but we do
> have E and F. If A modified the path and B deleted it, and C and D both
> decided to keep the path and E and F both inherited that path, wouldn't it
> be fair to say that what both branches wanted to do is closer to what A
> did than what B did? In other words, instead of using O, wouldn't it give
> us a better result if we used A (the side that did not delete) as the
> common ancestor for the content level merge when both E and F has the
> path?

Yes, absolutely. In the second case, using the content of the path from A for the virtual merge ancestor would be a slight improvement over using the content for path from O. But let's look at all three cases you specified and how different choices for the virtual merge base affect it. Here are my goals for any merge, in priority order:

1st goal: detect real conflicts and notify the user 2nd goal: avoid conflicts in cases where we should be able to reasonably confidently merge cleanly 3rd goal: make it easy for the user to figure out how to resolve problems

Obviously, we can't always meet these goals, particularly due to "semantic" conflicts that may be present. But they're useful to keep in mind. Here is how I broke down the three cases:

Case 1: C and D both agree to delete the path

In this case, regardless of the choice (J)-(M), the merge will be resolved correctly.

Case 2: C and D both agree to keep the path, with modifications that may or may not conflict (which can be handled by the usual content-level merge);

In this case, choice (J) would cause content conflicts whenever C & D had different content at path and would treat it as a two-way content merge. That makes choice (J) particularly problematic as it breaks the first goal. In order to get the merge right, we really need to do a three-way content merge, using content that can serve as a reasonable merge-base for the contents chosen at C & D. That makes choice (M) for X a poor one. Both choices (K) and (L) are reasonable, with (L) being a better choice (as you also point out above) to optimize the 2nd goal.

Case 3: C decides to delete, while D decides to modify (or vice versa)

It is worth noting here that the most likely choice for the content at path for D is exactly the content that is at A. git is designed for trees of slowly changing data after all.

This is a case where git should detect the modify/delete conflict and alert the user. In this case, choice (J) is particularly bad. Regardless of the content that D has for path, the merge conflict won't even be detected and git would silently resolve this by recording the content from D at path. That's a fail for the first goal.

Another choice that is likely to fail the first goal is (L). If the content at path D is the same as at A (the most likely case), then git would not detect the conflict and would simply silently delete the path.

In fact, any choice here for the merge base could cause the conflict to go silently undetected (whenever the content at D matched our choice for X), meaning that no choice for the merge base will be perfect. That would mean of the four choices, (M) would be best as it would be least likely to match the choice made at D, but (K) would be pretty reasonable.

So, as I view it: choice (J) is horrible in two of the three cases; choice (M) is really bad in the second case for goal 2; and choice (L) is likely to be bad in the third case for goal 1. Choice (K) is pretty reasonable in all three cases, and that choice is what this patch implements.

Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 38/48] merge-recursive: Introduce a merge_file convenience function

merge_file previously required diff_filespec arguments, but all callers only had sha1s and modes. Rename merge_file to merge_file_1 and introduce a new merge_file convenience function which takes the sha1s and modes and creates the temporary diff_filespec variables needed to call merge_file_1.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   74 +++++++++++++++++++++++++++-------------------------
 1 files changed, 38 insertions(+), 36 deletions(-)
Show changes to merge-recursive.c +38 −36
diff --git a/merge-recursive.c b/merge-recursive.c
index 5a70a87..3a364fa 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -831,12 +831,12 @@ static int merge_3way(struct merge_options *o,
 	return merge_status;
 }
 
-static struct merge_file_info merge_file(struct merge_options *o,
-					 const struct diff_filespec *one,
-					 const struct diff_filespec *a,
-					 const struct diff_filespec *b,
-					 const char *branch1,
-					 const char *branch2)
+static struct merge_file_info merge_file_1(struct merge_options *o,
+					   const struct diff_filespec *one,
+					   const struct diff_filespec *a,
+					   const struct diff_filespec *b,
+					   const char *branch1,
+					   const char *branch2)
 {
 	struct merge_file_info result;
 	result.merge = 0;
@@ -905,6 +905,26 @@ static struct merge_file_info merge_file(struct merge_options *o,
 	return result;
 }
 
+static struct merge_file_info merge_file(struct merge_options *o,
+					 const char *path,
+					 const unsigned char *o_sha, int o_mode,
+					 const unsigned char *a_sha, int a_mode,
+					 const unsigned char *b_sha, int b_mode,
+					 const char *branch1,
+					 const char *branch2)
+{
+	struct diff_filespec one, a, b;
+
+	one.path = a.path = b.path = (char*)path;
+	hashcpy(one.sha1, o_sha);
+	one.mode = o_mode;
+	hashcpy(a.sha1, a_sha);
+	a.mode = a_mode;
+	hashcpy(b.sha1, b_sha);
+	b.mode = b_mode;
+	return merge_file_1(o, &one, &a, &b, branch1, branch2);
+}
+
 static void conflict_rename_delete(struct merge_options *o,
 				   struct diff_filepair *pair,
 				   const char *rename_branch,
@@ -1002,16 +1022,10 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	/* Two files were renamed to the same thing. */
 	if (o->call_depth) {
 		struct merge_file_info mfi;
-		struct diff_filespec one, a, b;
-
-		one.path = a.path = b.path = path;
-		hashcpy(one.sha1, null_sha1);
-		one.mode = 0;
-		hashcpy(a.sha1, ren1->pair->two->sha1);
-		a.mode = ren1->pair->two->mode;
-		hashcpy(b.sha1, ren2->pair->two->sha1);
-		b.mode = ren2->pair->two->mode;
-		mfi = merge_file(o, &one, &a, &b, branch1, branch2);
+		mfi = merge_file(o, path, null_sha1, 0,
+				 ren1->pair->two->sha1, ren1->pair->two->mode,
+				 ren2->pair->two->sha1, ren2->pair->two->mode,
+				 branch1, branch2);
 		output(o, 1, "Adding merged %s", path);
 		update_file(o, 0, mfi.sha, mfi.mode, path);
 	} else {
@@ -1198,24 +1212,12 @@ static int process_renames(struct merge_options *o,
 				       ren1_dst, branch2);
 				if (o->call_depth) {
 					struct merge_file_info mfi;
-					struct diff_filespec one, a, b;
-
-					one.path = a.path = b.path =
-						(char *)ren1_dst;
-					hashcpy(one.sha1, null_sha1);
-					one.mode = 0;
-					hashcpy(a.sha1, ren1->pair->two->sha1);
-					a.mode = ren1->pair->two->mode;
-					hashcpy(b.sha1, dst_other.sha1);
-					b.mode = dst_other.mode;
-					mfi = merge_file(o, &one, &a, &b,
-							 branch1,
-							 branch2);
+					mfi = merge_file(o, ren1_dst, null_sha1, 0,
+							 ren1->pair->two->sha1, ren1->pair->two->mode,
+							 dst_other.sha1, dst_other.mode,
+							 branch1, branch2);
 					output(o, 1, "Adding merged %s", ren1_dst);
-					update_file(o, 0,
-						    mfi.sha,
-						    mfi.mode,
-						    ren1_dst);
+					update_file(o, 0, mfi.sha, mfi.mode, ren1_dst);
 					try_merge = 0;
 				} else {
 					char *new_path = unique_path(o, ren1_dst, branch2);
@@ -1388,13 +1390,13 @@ static int merge_content(struct merge_options *o,
 		if (dir_in_way(path, !o->call_depth))
 			df_conflict_remains = 1;
 	}
-	mfi = merge_file(o, &one, &a, &b,
-			 side1 ? side1:o->branch1, side2 ? side2:o->branch2);
+	mfi = merge_file_1(o, &one, &a, &b,
+			   side1 ? side1:o->branch1, side2 ? side2:o->branch2);
 	free(side1);
 	free(side2);
 
 	if (mfi.clean && !df_conflict_remains &&
-	    sha_eq(mfi.sha, a_sha) && mfi.mode == a.mode &&
+	    sha_eq(mfi.sha, a_sha) && mfi.mode == a_mode &&
 	    was_tracked(path)) {
 		output(o, 3, "Skipped %s (merged same as existing)", path);
 		add_cacheinfo(mfi.mode, mfi.sha, path,
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 39/48] merge-recursive: Fix rename/rename(1to2) resolution for virtual merge base

When renaming one file to two files, we really should be doing a content merge. Also, in the recursive case, undoing the renames and recording the merged file in the index with the source of the rename (while deleting both destinations) allows the renames to be re-detected in the non-recursive merge and will result in fewer spurious conflicts.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                 |   30 +++++++++++++-----------------
 t/t6036-recursive-corner-cases.sh |    2 +-
 2 files changed, 14 insertions(+), 18 deletions(-)
Show changes to 2 files +14 −18

merge-recursive.c, t/t6036-recursive-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 3a364fa..576e02d 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -971,17 +971,6 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 	       src, pair1->two->path, branch1,
 	       src, pair2->two->path, branch2,
 	       o->call_depth ? " (left unresolved)" : "");
-	if (o->call_depth) {
-		/*
-		 * FIXME: Why remove file from cache, and then
-		 * immediately readd it?  Why not just overwrite using
-		 * update_file only?  Also...this is buggy for
-		 * rename/add-source situations...
-		 */
-		remove_file_from_cache(src);
-		update_file(o, 0, pair1->one->sha1, pair1->one->mode, src);
-	}
-
 	if (dir_in_way(ren1_dst, !o->call_depth)) {
 		dst_name1 = del[delp++] = unique_path(o, ren1_dst, branch1);
 		output(o, 1, "%s is a directory in %s adding as %s instead",
@@ -993,14 +982,21 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		       ren2_dst, branch1, dst_name2);
 	}
 	if (o->call_depth) {
-		remove_file_from_cache(dst_name1);
-		remove_file_from_cache(dst_name2);
+		struct merge_file_info mfi;
+		mfi = merge_file(o, src,
+				 pair1->one->sha1, pair1->one->mode,
+				 pair1->two->sha1, pair1->two->mode,
+				 pair2->two->sha1, pair2->two->mode,
+				 branch1, branch2);
 		/*
-		 * Uncomment to leave the conflicting names in the resulting tree
-		 *
-		 * update_file(o, 0, pair1->two->sha1, pair1->two->mode, dst_name1);
-		 * update_file(o, 0, pair2->two->sha1, pair2->two->mode, dst_name2);
+		 * FIXME: For rename/add-source conflicts (if we could detect
+		 * such), this is wrong.  We should instead find a unique
+		 * pathname and then either rename the add-source file to that
+		 * unique path, or use that unique path instead of src here.
 		 */
+		update_file(o, 0, mfi.sha, mfi.mode, src);
+		remove_file_from_cache(ren1_dst);
+		remove_file_from_cache(ren2_dst);
 	} else {
 		update_stages(ren1_dst, NULL, pair1->two, NULL);
 		update_stages(ren2_dst, NULL, NULL, pair2->two);
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index d27477b..a1b609f 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -611,7 +611,7 @@ test_expect_success 'setup rename/rename(1to2)/modify followed by what looks lik
 	git tag E
 '
 
-test_expect_failure 'handle rename/rename(1to2)/modify followed by what looks like rename/rename(2to1)/modify' '
+test_expect_success 'handle rename/rename(1to2)/modify followed by what looks like rename/rename(2to1)/modify' '
 	git checkout D^0 &&
 
 	git merge -s recursive E^0 &&
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jul 25, 2011, 20:55 UTC · re: Elijah Newren · lore

Re: [PATCH 39/48] merge-recursive: Fix rename/rename(1to2) resolution for virtual merge base

Elijah Newren <newren@gmail.com> writes:
Show 5 quoted lines
> When renaming one file to two files, we really should be doing a content
> merge.  Also, in the recursive case, undoing the renames and recording the
> merged file in the index with the source of the rename (while deleting
> both destinations) allows the renames to be re-detected in the
> non-recursive merge and will result in fewer spurious conflicts.
In other words,...
Show 8 quoted lines
> +		 * FIXME: For rename/add-source conflicts (if we could detect
> +		 * such), this is wrong.  We should instead find a unique
> +		 * pathname and then either rename the add-source file to that
> +		 * unique path, or use that unique path instead of src here.
>  		 */
> +		update_file(o, 0, mfi.sha, mfi.mode, src);
> +		remove_file_from_cache(ren1_dst);
> +		remove_file_from_cache(ren2_dst);

... the use of "src" here is taking a "middle ground" and punting on resolving the conflicting opinions of both branches for the outer merge to resolve. I think that is a sensible thing to do.

The rename destinations of both branches are removed. What happens if ren1_dst (the path one branch wanted to rename src to) were added by the other branch (which wanted to rename src to ren2_dst), causing rename/add conflict between branches (i.e. not the one you worry about in the above FIXME which is about one branch renaming src to ren1_dst while adding an unrelated content to src)?

Elijah Newren· Aug 8, 2011, 22:58 UTC · re: Junio C Hamano · lore

Re: [PATCH 39/48] merge-recursive: Fix rename/rename(1to2) resolution for virtual merge base

On Mon, Jul 25, 2011 at 2:55 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 15 quoted lines
> Elijah Newren <newren@gmail.com> writes:
>> +             update_file(o, 0, mfi.sha, mfi.mode, src);
>> +             remove_file_from_cache(ren1_dst);
>> +             remove_file_from_cache(ren2_dst);
>
> ... the use of "src" here is taking a "middle ground" and punting on
> resolving the conflicting opinions of both branches for the outer merge to
> resolve. I think that is a sensible thing to do.
>
> The rename destinations of both branches are removed. What happens if
> ren1_dst (the path one branch wanted to rename src to) were added by the
> other branch (which wanted to rename src to ren2_dst), causing rename/add
> conflict between branches (i.e. not the one you worry about in the above
> FIXME which is about one branch renaming src to ren1_dst while adding an
> unrelated content to src)?

You found another bug; this time one that was already present in git rather than having been introduced by my series (the remove_file_from_cache() calls were already present, I just moved them around a little bit). I have a testcase that exposes this new bug now.

Good catch.  :-)
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 40/48] merge-recursive: Small cleanups for conflict_rename_rename_1to2

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   60 +++++++++++++++++++++++-----------------------------
 1 files changed, 27 insertions(+), 33 deletions(-)
Show changes to merge-recursive.c +27 −33
diff --git a/merge-recursive.c b/merge-recursive.c
index 576e02d..9c0daf5 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -951,58 +951,55 @@ static void conflict_rename_delete(struct merge_options *o,
 }
 
 static void conflict_rename_rename_1to2(struct merge_options *o,
-					struct diff_filepair *pair1,
-					const char *branch1,
-					struct diff_filepair *pair2,
-					const char *branch2)
+					struct rename_conflict_info *ci)
 {
 	/* One file was renamed in both branches, but to different names. */
+	struct diff_filespec *one = ci->pair1->one;
+	struct diff_filespec *a = ci->pair1->two;
+	struct diff_filespec *b = ci->pair2->two;
+	const char *dst_name_a = a->path;
+	const char *dst_name_b = b->path;
 	char *del[2];
 	int delp = 0;
-	const char *src      = pair1->one->path;
-	const char *ren1_dst = pair1->two->path;
-	const char *ren2_dst = pair2->two->path;
-	const char *dst_name1 = ren1_dst;
-	const char *dst_name2 = ren2_dst;
 
 	output(o, 1, "CONFLICT (rename/rename): "
 	       "Rename \"%s\"->\"%s\" in branch \"%s\" "
 	       "rename \"%s\"->\"%s\" in \"%s\"%s",
-	       src, pair1->two->path, branch1,
-	       src, pair2->two->path, branch2,
+	       one->path, a->path, ci->branch1,
+	       one->path, b->path, ci->branch2,
 	       o->call_depth ? " (left unresolved)" : "");
-	if (dir_in_way(ren1_dst, !o->call_depth)) {
-		dst_name1 = del[delp++] = unique_path(o, ren1_dst, branch1);
+	if (dir_in_way(a->path, !o->call_depth)) {
+		dst_name_a = del[delp++] = unique_path(o, a->path, ci->branch1);
 		output(o, 1, "%s is a directory in %s adding as %s instead",
-		       ren1_dst, branch2, dst_name1);
+		       a->path, ci->branch2, dst_name_a);
 	}
-	if (dir_in_way(ren2_dst, !o->call_depth)) {
-		dst_name2 = del[delp++] = unique_path(o, ren2_dst, branch2);
+	if (dir_in_way(b->path, !o->call_depth)) {
+		dst_name_b = del[delp++] = unique_path(o, b->path, ci->branch2);
 		output(o, 1, "%s is a directory in %s adding as %s instead",
-		       ren2_dst, branch1, dst_name2);
+		       b->path, ci->branch1, dst_name_b);
 	}
 	if (o->call_depth) {
 		struct merge_file_info mfi;
-		mfi = merge_file(o, src,
-				 pair1->one->sha1, pair1->one->mode,
-				 pair1->two->sha1, pair1->two->mode,
-				 pair2->two->sha1, pair2->two->mode,
-				 branch1, branch2);
+		mfi = merge_file(o, one->path,
+				 one->sha1, one->mode,
+				 a->sha1, a->mode,
+				 b->sha1, b->mode,
+				 ci->branch1, ci->branch2);
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
 		 * pathname and then either rename the add-source file to that
 		 * unique path, or use that unique path instead of src here.
 		 */
-		update_file(o, 0, mfi.sha, mfi.mode, src);
-		remove_file_from_cache(ren1_dst);
-		remove_file_from_cache(ren2_dst);
+		update_file(o, 0, mfi.sha, mfi.mode, one->path);
+		remove_file_from_cache(a->path);
+		remove_file_from_cache(b->path);
 	} else {
-		update_stages(ren1_dst, NULL, pair1->two, NULL);
-		update_stages(ren2_dst, NULL, NULL, pair2->two);
+		update_stages(a->path, NULL, a, NULL);
+		update_stages(b->path, NULL, NULL, b);
 
-		update_file(o, 0, pair1->two->sha1, pair1->two->mode, dst_name1);
-		update_file(o, 0, pair2->two->sha1, pair2->two->mode, dst_name2);
+		update_file(o, 0, a->sha1, a->mode, dst_name_a);
+		update_file(o, 0, b->sha1, b->mode, dst_name_b);
 	}
 	while (delp--)
 		free(del[delp]);
@@ -1476,10 +1473,7 @@ static int process_entry(struct merge_options *o,
 			break;
 		case RENAME_ONE_FILE_TO_TWO:
 			clean_merge = 0;
-			conflict_rename_rename_1to2(o, conflict_info->pair1,
-						    conflict_info->branch1,
-						    conflict_info->pair2,
-						    conflict_info->branch2);
+			conflict_rename_rename_1to2(o, conflict_info);
 			break;
 		default:
 			entry->processed = 0;
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 41/48] merge-recursive: Defer rename/rename(2to1) handling until process_entry

This puts the code for the different types of double rename conflicts closer together (fewer lines of other code separating the two paths) and increases similarity between how they are handled.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |  104 +++++++++++++++++++++++++++++++---------------------
 1 files changed, 62 insertions(+), 42 deletions(-)
Show changes to merge-recursive.c +62 −42
diff --git a/merge-recursive.c b/merge-recursive.c
index 9c0daf5..dbc1dd3 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -67,7 +67,8 @@ enum rename_type {
 	RENAME_NORMAL = 0,
 	RENAME_DELETE,
 	RENAME_ONE_FILE_TO_ONE,
-	RENAME_ONE_FILE_TO_TWO
+	RENAME_ONE_FILE_TO_TWO,
+	RENAME_TWO_FILES_TO_ONE
 };
 
 struct rename_conflict_info {
@@ -1006,32 +1007,40 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 }
 
 static void conflict_rename_rename_2to1(struct merge_options *o,
-					struct rename *ren1,
-					const char *branch1,
-					struct rename *ren2,
-					const char *branch2)
+					struct rename_conflict_info *ci)
 {
-	char *path = ren1->pair->two->path; /* same as ren2->pair->two->path */
-	/* Two files were renamed to the same thing. */
+	/* Two files, a & b, were renamed to the same thing, c. */
+	struct diff_filespec *a = ci->pair1->one;
+	struct diff_filespec *b = ci->pair2->one;
+	struct diff_filespec *c1 = ci->pair1->two;
+	struct diff_filespec *c2 = ci->pair2->two;
+	char *path = c1->path; /* == c2->path */
+
+	output(o, 1, "CONFLICT (rename/rename): "
+	       "Rename %s->%s in %s. "
+	       "Rename %s->%s in %s",
+	       a->path, c1->path, ci->branch1,
+	       b->path, c2->path, ci->branch2);
+
+	remove_file(o, 1, a->path, would_lose_untracked(a->path));
+	remove_file(o, 1, b->path, would_lose_untracked(b->path));
+
 	if (o->call_depth) {
 		struct merge_file_info mfi;
 		mfi = merge_file(o, path, null_sha1, 0,
-				 ren1->pair->two->sha1, ren1->pair->two->mode,
-				 ren2->pair->two->sha1, ren2->pair->two->mode,
-				 branch1, branch2);
+				 c1->sha1, c1->mode,
+				 c2->sha1, c2->mode,
+				 ci->branch1, ci->branch2);
 		output(o, 1, "Adding merged %s", path);
 		update_file(o, 0, mfi.sha, mfi.mode, path);
 	} else {
-		char *new_path1 = unique_path(o, path, branch1);
-		char *new_path2 = unique_path(o, path, branch2);
+		char *new_path1 = unique_path(o, path, ci->branch1);
+		char *new_path2 = unique_path(o, path, ci->branch2);
 		output(o, 1, "Renaming %s to %s and %s to %s instead",
-		       ren1->pair->one->path, new_path1,
-		       ren2->pair->one->path, new_path2);
+		       a->path, new_path1, b->path, new_path2);
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, ren1->pair->two->sha1, ren1->pair->two->mode,
-			    new_path1);
-		update_file(o, 0, ren2->pair->two->sha1, ren2->pair->two->mode,
-			    new_path2);
+		update_file(o, 0, c1->sha1, c1->mode, new_path1);
+		update_file(o, 0, c2->sha1, c2->mode, new_path2);
 		free(new_path2);
 		free(new_path1);
 	}
@@ -1062,6 +1071,7 @@ static int process_renames(struct merge_options *o,
 		struct rename *ren1 = NULL, *ren2 = NULL;
 		const char *branch1, *branch2;
 		const char *ren1_src, *ren1_dst;
+		struct string_list_item *lookup;
 
 		if (i >= a_renames->nr) {
 			ren2 = b_renames->items[j++].util;
@@ -1093,30 +1103,30 @@ static int process_renames(struct merge_options *o,
 			ren1 = tmp;
 		}
 
+		if (ren1->processed)
+			continue;
+		ren1->processed = 1;
 		ren1->dst_entry->processed = 1;
 		/* BUG: We should only mark src_entry as processed if we
 		 * are not dealing with a rename + add-source case.
 		 */
 		ren1->src_entry->processed = 1;
 
-		if (ren1->processed)
-			continue;
-		ren1->processed = 1;
-
 		ren1_src = ren1->pair->one->path;
 		ren1_dst = ren1->pair->two->path;
 
 		if (ren2) {
+			/* One file renamed on both sides */
 			const char *ren2_src = ren2->pair->one->path;
 			const char *ren2_dst = ren2->pair->two->path;
 			enum rename_type rename_type;
-			/* Renamed in 1 and renamed in 2 */
 			if (strcmp(ren1_src, ren2_src) != 0)
-				die("ren1.src != ren2.src");
+				die("ren1_src != ren2_src");
 			ren2->dst_entry->processed = 1;
 			ren2->processed = 1;
 			if (strcmp(ren1_dst, ren2_dst) != 0) {
 				rename_type = RENAME_ONE_FILE_TO_TWO;
+				clean_merge = 0;
 			} else {
 				rename_type = RENAME_ONE_FILE_TO_ONE;
 				/* BUG: We should only remove ren1_src in
@@ -1136,9 +1146,32 @@ static int process_renames(struct merge_options *o,
 						   branch2,
 						   ren1->dst_entry,
 						   ren2->dst_entry);
+		} else if ((lookup = string_list_lookup(renames2Dst, ren1_dst))) {
+			/* Two different files renamed to the same thing */
+			char *ren2_dst;
+			ren2 = lookup->util;
+			ren2_dst = ren2->pair->two->path;
+			if (strcmp(ren1_dst, ren2_dst) != 0)
+				die("ren1_dst != ren2_dst");
+
+			clean_merge = 0;
+			ren2->processed = 1;
+			/*
+			 * BUG: We should only mark src_entry as processed
+			 * if we are not dealing with a rename + add-source
+			 * case.
+			 */
+			ren2->src_entry->processed = 1;
+
+			setup_rename_conflict_info(RENAME_TWO_FILES_TO_ONE,
+						   ren1->pair,
+						   ren2->pair,
+						   branch1,
+						   branch2,
+						   ren1->dst_entry,
+						   ren2->dst_entry);
 		} else {
 			/* Renamed in 1, maybe changed in 2 */
-			struct string_list_item *item;
 			/* we only use sha1 and mode of these */
 			struct diff_filespec src_other, dst_other;
 			int try_merge;
@@ -1173,23 +1206,6 @@ static int process_renames(struct merge_options *o,
 							   branch2,
 							   ren1->dst_entry,
 							   NULL);
-			} else if ((item = string_list_lookup(renames2Dst, ren1_dst))) {
-				char *ren2_src, *ren2_dst;
-				ren2 = item->util;
-				ren2_src = ren2->pair->one->path;
-				ren2_dst = ren2->pair->two->path;
-
-				clean_merge = 0;
-				ren2->processed = 1;
-				remove_file(o, 1, ren2_src,
-					    renamed_stage == 3 || would_lose_untracked(ren1_src));
-
-				output(o, 1, "CONFLICT (rename/rename): "
-				       "Rename %s->%s in %s. "
-				       "Rename %s->%s in %s",
-				       ren1_src, ren1_dst, branch1,
-				       ren2_src, ren2_dst, branch2);
-				conflict_rename_rename_2to1(o, ren1, branch1, ren2, branch2);
 			} else if ((dst_other.mode == ren1->pair->two->mode) &&
 				   sha_eq(dst_other.sha1, ren1->pair->two->sha1)) {
 				/* Added file on the other side
@@ -1475,6 +1491,10 @@ static int process_entry(struct merge_options *o,
 			clean_merge = 0;
 			conflict_rename_rename_1to2(o, conflict_info);
 			break;
+		case RENAME_TWO_FILES_TO_ONE:
+			clean_merge = 0;
+			conflict_rename_rename_2to1(o, conflict_info);
+			break;
 		default:
 			entry->processed = 0;
 			break;
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 42/48] merge-recursive: Record more data needed for merging with dual renames

When two different files are renamed to one, we need to be able to do three-way merges for both of those files. To do that, we need to record the sha1sum of the (possibly modified) file on the unrenamed side. Modify setup_rename_conflict_info() to take this extra information and record it when the rename_type is RENAME_TWO_FILES_TO_ONE.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   42 +++++++++++++++++++++++++++++++++++++++---
 1 files changed, 39 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +39 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index dbc1dd3..fdf67a3 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -79,6 +79,8 @@ struct rename_conflict_info {
 	const char *branch2;
 	struct stage_data *dst_entry1;
 	struct stage_data *dst_entry2;
+	struct diff_filespec ren1_other;
+	struct diff_filespec ren2_other;
 };
 
 /*
@@ -100,7 +102,10 @@ static inline void setup_rename_conflict_info(enum rename_type rename_type,
 					      const char *branch1,
 					      const char *branch2,
 					      struct stage_data *dst_entry1,
-					      struct stage_data *dst_entry2)
+					      struct stage_data *dst_entry2,
+					      struct merge_options *o,
+					      struct stage_data *src_entry1,
+					      struct stage_data *src_entry2)
 {
 	struct rename_conflict_info *ci = xcalloc(1, sizeof(struct rename_conflict_info));
 	ci->rename_type = rename_type;
@@ -118,6 +123,24 @@ static inline void setup_rename_conflict_info(enum rename_type rename_type,
 		ci->pair2 = pair2;
 		dst_entry2->rename_conflict_info = ci;
 	}
+
+	if (rename_type == RENAME_TWO_FILES_TO_ONE) {
+		/*
+		 * For each rename, there could have been
+		 * modifications on the side of history where that
+		 * file was not renamed.
+		 */
+		int ostage1 = o->branch1 == branch1 ? 3 : 2;
+		int ostage2 = ostage1 ^ 1;
+
+		ci->ren1_other.path = pair1->one->path;
+		hashcpy(ci->ren1_other.sha1, src_entry1->stages[ostage1].sha);
+		ci->ren1_other.mode = src_entry1->stages[ostage1].mode;
+
+		ci->ren2_other.path = pair2->one->path;
+		hashcpy(ci->ren2_other.sha1, src_entry2->stages[ostage2].sha);
+		ci->ren2_other.mode = src_entry2->stages[ostage2].mode;
+	}
 }
 
 static int show(struct merge_options *o, int v)
@@ -1145,7 +1168,10 @@ static int process_renames(struct merge_options *o,
 						   branch1,
 						   branch2,
 						   ren1->dst_entry,
-						   ren2->dst_entry);
+						   ren2->dst_entry,
+						   o,
+						   NULL,
+						   NULL);
 		} else if ((lookup = string_list_lookup(renames2Dst, ren1_dst))) {
 			/* Two different files renamed to the same thing */
 			char *ren2_dst;
@@ -1169,7 +1195,11 @@ static int process_renames(struct merge_options *o,
 						   branch1,
 						   branch2,
 						   ren1->dst_entry,
-						   ren2->dst_entry);
+						   ren2->dst_entry,
+						   o,
+						   ren1->src_entry,
+						   ren2->src_entry);
+
 		} else {
 			/* Renamed in 1, maybe changed in 2 */
 			/* we only use sha1 and mode of these */
@@ -1205,6 +1235,9 @@ static int process_renames(struct merge_options *o,
 							   branch1,
 							   branch2,
 							   ren1->dst_entry,
+							   NULL,
+							   o,
+							   NULL,
 							   NULL);
 			} else if ((dst_other.mode == ren1->pair->two->mode) &&
 				   sha_eq(dst_other.sha1, ren1->pair->two->sha1)) {
@@ -1256,6 +1289,9 @@ static int process_renames(struct merge_options *o,
 							   branch1,
 							   NULL,
 							   ren1->dst_entry,
+							   NULL,
+							   o,
+							   NULL,
 							   NULL);
 			}
 		}
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 43/48] merge-recursive: Create function for merging with branchname:file markers

We want to be able to reuse the code to do a three-way file content merge and have the conflict markers use both branchname and filename. Split it out into a separate function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   51 ++++++++++++++++++++++++++++++++++++++-------------
 1 files changed, 38 insertions(+), 13 deletions(-)
Show changes to merge-recursive.c +38 −13
diff --git a/merge-recursive.c b/merge-recursive.c
index fdf67a3..b8fd686 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -929,6 +929,36 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 	return result;
 }
 
+static struct merge_file_info
+merge_file_special_markers(struct merge_options *o,
+			   const struct diff_filespec *one,
+			   const struct diff_filespec *a,
+			   const struct diff_filespec *b,
+			   const char *branch1,
+			   const char *filename1,
+			   const char *branch2,
+			   const char *filename2)
+{
+	char *side1 = NULL;
+	char *side2 = NULL;
+	struct merge_file_info mfi;
+
+	if (filename1) {
+		side1 = xmalloc(strlen(branch1) + strlen(filename1) + 2);
+		sprintf(side1, "%s:%s", branch1, filename1);
+	}
+	if (filename2) {
+		side2 = xmalloc(strlen(branch2) + strlen(filename2) + 2);
+		sprintf(side2, "%s:%s", branch2, filename2);
+	}
+
+	mfi = merge_file_1(o, one, a, b,
+			   side1 ? side1 : branch1, side2 ? side2 : branch2);
+	free(side1);
+	free(side2);
+	return mfi;
+}
+
 static struct merge_file_info merge_file(struct merge_options *o,
 					 const char *path,
 					 const unsigned char *o_sha, int o_mode,
@@ -1405,7 +1435,7 @@ static int merge_content(struct merge_options *o,
 			 struct rename_conflict_info *rename_conflict_info)
 {
 	const char *reason = "content";
-	char *side1 = NULL, *side2 = NULL;
+	const char *path1 = NULL, *path2 = NULL;
 	struct merge_file_info mfi;
 	struct diff_filespec one, a, b;
 	unsigned df_conflict_remains = 0;
@@ -1423,22 +1453,17 @@ static int merge_content(struct merge_options *o,
 	b.mode = b_mode;
 
 	if (rename_conflict_info) {
-		const char *path1 = rename_conflict_info->pair2 ?
-				    rename_conflict_info->pair2->one->path : path;
-		const char *path2 = rename_conflict_info->pair1 ?
-				    rename_conflict_info->pair1->one->path : path;
-		side1 = xmalloc(strlen(o->branch1) + strlen(path1) + 2);
-		side2 = xmalloc(strlen(o->branch2) + strlen(path2) + 2);
-		sprintf(side1, "%s:%s", o->branch1, path1);
-		sprintf(side2, "%s:%s", o->branch2, path2);
+		path1 = rename_conflict_info->pair2 ?
+			rename_conflict_info->pair2->one->path : path;
+		path2 = rename_conflict_info->pair1 ?
+			rename_conflict_info->pair1->one->path : path;
 
 		if (dir_in_way(path, !o->call_depth))
 			df_conflict_remains = 1;
 	}
-	mfi = merge_file_1(o, &one, &a, &b,
-			   side1 ? side1:o->branch1, side2 ? side2:o->branch2);
-	free(side1);
-	free(side2);
+	mfi = merge_file_special_markers(o, &one, &a, &b,
+					 o->branch1, path1,
+					 o->branch2, path2);
 
 	if (mfi.clean && !df_conflict_remains &&
 	    sha_eq(mfi.sha, a_sha) && mfi.mode == a_mode &&
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 44/48] merge-recursive: Consider modifications in rename/rename(2to1) conflicts

Our previous conflict resolution for renaming two different files to the same name ignored the fact that each of those files may have modifications from both sides of history to consider. We need to do a three-way merge for each of those files, and then handle the conflict of both sets of merged contents trying to be recorded with the same name.

It is important to note that this changes our strategy in the recursive case. After doing a three-way content merge of each of the files involved, we still are faced with the fact that we are trying to put both of the results (including conflict markers) into the same path. We could do another two-way merge, but I think that becomes confusing. Also, taking a hint from the modify/delete and rename/delete cases we handled earlier, a more useful "common ground" would be to keep the three-way content merge but record it with the original filename. The renames can still be detected, we just allow it to be done in the o->call_depth=0 case. This seems to result in simpler & easier to understand merge conflicts as well, as evidenced by some of the changes needed in our testsuite in t6036. (However, it should be noted that this change will cause problems those renames also occur along with a file being added whose name matches the source of the rename. Since git currently cannot detect rename/add-source situations, though, this codepath is not currently used for those cases anyway.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                    |   30 ++++++++++++++++++--------
 t/t6036-recursive-corner-cases.sh    |   38 +++++++++-------------------------
 t/t6039-merge-rename-corner-cases.sh |    2 +-
 3 files changed, 32 insertions(+), 38 deletions(-)
Show changes to 3 files +32 −38

merge-recursive.c, t/t6036-recursive-corner-cases.sh, t/t6039-merge-rename-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index b8fd686..ea538e9 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1068,6 +1068,8 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	struct diff_filespec *c1 = ci->pair1->two;
 	struct diff_filespec *c2 = ci->pair2->two;
 	char *path = c1->path; /* == c2->path */
+	struct merge_file_info mfi_c1;
+	struct merge_file_info mfi_c2;
 
 	output(o, 1, "CONFLICT (rename/rename): "
 	       "Rename %s->%s in %s. "
@@ -1078,22 +1080,32 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	remove_file(o, 1, a->path, would_lose_untracked(a->path));
 	remove_file(o, 1, b->path, would_lose_untracked(b->path));
 
+	mfi_c1 = merge_file_special_markers(o, a, c1, &ci->ren1_other,
+					    o->branch1, c1->path,
+					    o->branch2, ci->ren1_other.path);
+	mfi_c2 = merge_file_special_markers(o, b, &ci->ren2_other, c2,
+					    o->branch1, ci->ren2_other.path,
+					    o->branch2, c2->path);
+
 	if (o->call_depth) {
-		struct merge_file_info mfi;
-		mfi = merge_file(o, path, null_sha1, 0,
-				 c1->sha1, c1->mode,
-				 c2->sha1, c2->mode,
-				 ci->branch1, ci->branch2);
-		output(o, 1, "Adding merged %s", path);
-		update_file(o, 0, mfi.sha, mfi.mode, path);
+		/*
+		 * If mfi_c1.clean && mfi_c2.clean, then it might make
+		 * sense to do a two-way merge of those results.  But, I
+		 * think in all cases, it makes sense to have the virtual
+		 * merge base just undo the renames; they can be detected
+		 * again later for the non-recursive merge.
+		 */
+		remove_file(o, 0, path, 0);
+		update_file(o, 0, mfi_c1.sha, mfi_c1.mode, a->path);
+		update_file(o, 0, mfi_c2.sha, mfi_c2.mode, b->path);
 	} else {
 		char *new_path1 = unique_path(o, path, ci->branch1);
 		char *new_path2 = unique_path(o, path, ci->branch2);
 		output(o, 1, "Renaming %s to %s and %s to %s instead",
 		       a->path, new_path1, b->path, new_path2);
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, c1->sha1, c1->mode, new_path1);
-		update_file(o, 0, c2->sha1, c2->mode, new_path2);
+		update_file(o, 0, mfi_c1.sha, mfi_c1.mode, new_path1);
+		update_file(o, 0, mfi_c2.sha, mfi_c2.mode, new_path2);
 		free(new_path2);
 		free(new_path1);
 	}
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index a1b609f..5c3868a 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -51,23 +51,15 @@ test_expect_success 'merge simple rename+criss-cross with no modifications' '
 
 	test_must_fail git merge -s recursive R2^0 &&
 
-	test 5 = $(git ls-files -s | wc -l) &&
-	test 3 = $(git ls-files -u | wc -l) &&
-	test 0 = $(git ls-files -o | wc -l) &&
+	test 2 = $(git ls-files -s | wc -l) &&
+	test 2 = $(git ls-files -u | wc -l) &&
+	test 2 = $(git ls-files -o | wc -l) &&
 
-	test $(git rev-parse :0:one) = $(git rev-parse L2:one) &&
-	test $(git rev-parse :0:two) = $(git rev-parse R2:two) &&
 	test $(git rev-parse :2:three) = $(git rev-parse L2:three) &&
 	test $(git rev-parse :3:three) = $(git rev-parse R2:three) &&
 
-	cp one merged &&
-	>empty &&
-	test_must_fail git merge-file \
-		-L "Temporary merge branch 1" \
-		-L "" \
-		-L "Temporary merge branch 2" \
-		merged empty two &&
-	test $(git rev-parse :1:three) = $(git hash-object merged)
+	test $(git rev-parse L2:three) = $(git hash-object three~HEAD) &&
+	test $(git rev-parse R2:three) = $(git hash-object three~R2^0)
 '
 
 #
@@ -126,25 +118,15 @@ test_expect_success 'merge criss-cross + rename merges with basic modification'
 
 	test_must_fail git merge -s recursive R2^0 &&
 
-	test 5 = $(git ls-files -s | wc -l) &&
-	test 3 = $(git ls-files -u | wc -l) &&
-	test 0 = $(git ls-files -o | wc -l) &&
+	test 2 = $(git ls-files -s | wc -l) &&
+	test 2 = $(git ls-files -u | wc -l) &&
+	test 2 = $(git ls-files -o | wc -l) &&
 
-	test $(git rev-parse :0:one) = $(git rev-parse L2:one) &&
-	test $(git rev-parse :0:two) = $(git rev-parse R2:two) &&
 	test $(git rev-parse :2:three) = $(git rev-parse L2:three) &&
 	test $(git rev-parse :3:three) = $(git rev-parse R2:three) &&
 
-	head -n 10 two >merged &&
-	cp one merge-me &&
-	>empty &&
-	test_must_fail git merge-file \
-		-L "Temporary merge branch 1" \
-		-L "" \
-		-L "Temporary merge branch 2" \
-		merge-me empty merged &&
-
-	test $(git rev-parse :1:three) = $(git hash-object merge-me)
+	test $(git rev-parse L2:three) = $(git hash-object three~HEAD) &&
+	test $(git rev-parse R2:three) = $(git hash-object three~R2^0)
 '
 
 #
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index 4d2fd10..86e96d0 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -524,7 +524,7 @@ test_expect_success 'setup rename/rename (2to1) + modify/modify' '
 	git commit -m C
 '
 
-test_expect_failure 'handle rename/rename (2to1) conflict correctly' '
+test_expect_success 'handle rename/rename (2to1) conflict correctly' '
 	git checkout B^0 &&
 
 	test_must_fail git merge -s recursive C^0 >out &&
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 45/48] merge-recursive: Make modify/delete handling code reusable

modify/delete and rename/delete share a lot of similarities; we'd like all the criss-cross and D/F conflict handling specializations to be shared between the two.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c       |   89 +++++++++++++++++++++++++++-------------------
 t/t6022-merge-rename.sh |    4 +-
 2 files changed, 54 insertions(+), 39 deletions(-)
Show changes to 2 files +54 −39

merge-recursive.c, t/t6022-merge-rename.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index ea538e9..e38c5b0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -979,6 +979,50 @@ static struct merge_file_info merge_file(struct merge_options *o,
 	return merge_file_1(o, &one, &a, &b, branch1, branch2);
 }
 
+static void handle_change_delete(struct merge_options *o,
+				 const char *path,
+				 const unsigned char *o_sha, int o_mode,
+				 const unsigned char *a_sha, int a_mode,
+				 const unsigned char *b_sha, int b_mode,
+				 const char *change, const char *change_past)
+{
+	char *new_path = NULL;
+	int free_me = 0;
+	if (dir_in_way(path, !o->call_depth)) {
+		new_path = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
+		free_me = 1;
+	}
+
+	if (o->call_depth) {
+		/*
+		 * We cannot arbitrarily accept either a_sha or b_sha as
+		 * correct; since there is no true "middle point" between
+		 * them, simply reuse the base version for virtual merge base.
+		 */
+		remove_file_from_cache(path);
+		update_file(o, 0, o_sha, o_mode, new_path ? new_path : path);
+	} else if (!a_sha) {
+		output(o, 1, "CONFLICT (%s/delete): %s deleted in %s "
+		       "and %s in %s. Version %s of %s left in tree%s%s.",
+		       change, path, o->branch1,
+		       change_past, o->branch2, o->branch2, path,
+		       NULL == new_path ? "" : " at ",
+		       NULL == new_path ? "" : new_path);
+		update_file(o, 0, b_sha, b_mode, new_path ? new_path : path);
+	} else {
+		output(o, 1, "CONFLICT (%s/delete): %s deleted in %s "
+		       "and %s in %s. Version %s of %s left in tree%s%s.",
+		       change, path, o->branch2,
+		       change_past, o->branch1, o->branch1, path,
+		       NULL == new_path ? "" : " at ",
+		       NULL == new_path ? "" : new_path);
+		update_file(o, 0, a_sha, a_mode, new_path ? new_path : path);
+	}
+	if (free_me)
+		free(new_path);
+
+}
+
 static void conflict_rename_delete(struct merge_options *o,
 				   struct diff_filepair *pair,
 				   const char *rename_branch,
@@ -1396,47 +1440,18 @@ error_return:
 	return ret;
 }
 
-static void handle_delete_modify(struct merge_options *o,
+static void handle_modify_delete(struct merge_options *o,
 				 const char *path,
 				 unsigned char *o_sha, int o_mode,
 				 unsigned char *a_sha, int a_mode,
 				 unsigned char *b_sha, int b_mode)
 {
-	char *new_path = NULL;
-	int free_me = 0;
-	if (dir_in_way(path, !o->call_depth)) {
-		new_path = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
-		free_me = 1;
-	}
-
-	if (o->call_depth) {
-		/*
-		 * We cannot arbitrarily accept either a_sha or b_sha as
-		 * correct; since there is no true "middle point" between
-		 * them, simply reuse the base version for virtual merge base.
-		 */
-		remove_file_from_cache(path);
-		update_file(o, 0, o_sha, o_mode, new_path ? new_path : path);
-	} else if (!a_sha) {
-		output(o, 1, "CONFLICT (delete/modify): %s deleted in %s "
-		       "and modified in %s. Version %s of %s left in tree%s%s.",
-		       path, o->branch1,
-		       o->branch2, o->branch2, path,
-		       NULL == new_path ? "" : " at ",
-		       NULL == new_path ? "" : new_path);
-		update_file(o, 0, b_sha, b_mode, new_path ? new_path : path);
-	} else {
-		output(o, 1, "CONFLICT (delete/modify): %s deleted in %s "
-		       "and modified in %s. Version %s of %s left in tree%s%s.",
-		       path, o->branch2,
-		       o->branch1, o->branch1, path,
-		       NULL == new_path ? "" : " at ",
-		       NULL == new_path ? "" : new_path);
-		update_file(o, 0, a_sha, a_mode, new_path ? new_path : path);
-	}
-	if (free_me)
-		free(new_path);
-
+	handle_change_delete(o,
+			     path,
+			     o_sha, o_mode,
+			     a_sha, a_mode,
+			     b_sha, b_mode,
+			     "modify", "modified");
 }
 
 static int merge_content(struct merge_options *o,
@@ -1586,7 +1601,7 @@ static int process_entry(struct merge_options *o,
 		} else {
 			/* Modify/delete; deleted side may have put a directory in the way */
 			clean_merge = 0;
-			handle_delete_modify(o, path, o_sha, o_mode,
+			handle_modify_delete(o, path, o_sha, o_mode,
 					     a_sha, a_mode, b_sha, b_mode);
 		}
 	} else if ((!o_sha && a_sha && !b_sha) ||
diff --git a/t/t6022-merge-rename.sh b/t/t6022-merge-rename.sh
index cfce3d3..a1ed00b 100755
--- a/t/t6022-merge-rename.sh
+++ b/t/t6022-merge-rename.sh
@@ -303,7 +303,7 @@ test_expect_success 'Rename+D/F conflict; renamed file merges but dir in way' '
 	git checkout -q renamed-file-has-no-conflicts^0 &&
 	test_must_fail git merge --strategy=recursive dir-in-way >output &&
 
-	grep "CONFLICT (delete/modify): dir/file-in-the-way" output &&
+	grep "CONFLICT (modify/delete): dir/file-in-the-way" output &&
 	grep "Auto-merging dir" output &&
 	grep "Adding as dir~HEAD instead" output &&
 
@@ -325,7 +325,7 @@ test_expect_success 'Same as previous, but merged other way' '
 	test_must_fail git merge --strategy=recursive renamed-file-has-no-conflicts >output 2>errors &&
 
 	! grep "error: refusing to lose untracked file at" errors &&
-	grep "CONFLICT (delete/modify): dir/file-in-the-way" output &&
+	grep "CONFLICT (modify/delete): dir/file-in-the-way" output &&
 	grep "Auto-merging dir" output &&
 	grep "Adding as dir~renamed-file-has-no-conflicts instead" output &&
 
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 46/48] merge-recursive: Have conflict_rename_delete reuse modify/delete code

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c |   48 +++++++++++++++++++++++++++++++-----------------
 1 files changed, 31 insertions(+), 17 deletions(-)
Show changes to merge-recursive.c +30 −16
diff --git a/merge-recursive.c b/merge-recursive.c
index e38c5b0..93b7da5 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1028,24 +1028,38 @@ static void conflict_rename_delete(struct merge_options *o,
 				   const char *rename_branch,
 				   const char *other_branch)
 {
-	char *dest_name = pair->two->path;
-	int df_conflict = 0;
+	const struct diff_filespec *orig = pair->one;
+	const struct diff_filespec *dest = pair->two;
+	const char *path;
+	const unsigned char *a_sha = NULL;
+	const unsigned char *b_sha = NULL;
+	int a_mode = 0;
+	int b_mode = 0;
+
+	if (rename_branch == o->branch1) {
+		a_sha = dest->sha1;
+		a_mode = dest->mode;
+	} else {
+		b_sha = dest->sha1;
+		b_mode = dest->mode;
+	}
 
-	output(o, 1, "CONFLICT (rename/delete): Rename %s->%s in %s "
-	       "and deleted in %s",
-	       pair->one->path, pair->two->path, rename_branch,
-	       other_branch);
-	if (!o->call_depth)
-		update_stages(dest_name, NULL,
-			      rename_branch == o->branch1 ? pair->two : NULL,
-			      rename_branch == o->branch1 ? NULL : pair->two);
-	if (dir_in_way(dest_name, !o->call_depth)) {
-		dest_name = unique_path(o, dest_name, rename_branch);
-		df_conflict = 1;
+	if (o->call_depth) {
+		remove_file_from_cache(dest->path);
+		path = orig->path;
+	} else {
+		path = dest->path;
+		update_stages(dest->path, NULL,
+			      rename_branch == o->branch1 ? dest : NULL,
+			      rename_branch == o->branch1 ? NULL : dest);
 	}
-	update_file(o, 0, pair->two->sha1, pair->two->mode, dest_name);
-	if (df_conflict)
-		free(dest_name);
+
+	handle_change_delete(o,
+			     path,
+			     orig->sha1, orig->mode,
+			     a_sha, a_mode,
+			     b_sha, b_mode,
+			     "rename", "renamed");
 }
 
 static void conflict_rename_rename_1to2(struct merge_options *o,
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 47/48] merge-recursive: add handling for rename/rename/add-dest/add-dest

Each side of the rename in rename/rename(1to2) could potentially also be involved in a rename/add conflict. Ensure stages for such conflicts are also recorded.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                    |   21 +++++++++++++++++++--
 t/t6039-merge-rename-corner-cases.sh |    2 +-
 2 files changed, 20 insertions(+), 3 deletions(-)
Show changes to 2 files +20 −3

merge-recursive.c, t/t6039-merge-rename-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 93b7da5..0cfc215 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1062,6 +1062,19 @@ static void conflict_rename_delete(struct merge_options *o,
 			     "rename", "renamed");
 }
 
+static struct diff_filespec* filespec_from_entry(struct diff_filespec *target,
+						 struct stage_data *entry,
+						 int stage)
+{
+	unsigned char *sha = entry->stages[stage].sha;
+	unsigned mode = entry->stages[stage].mode;
+	if (mode == 0 || is_null_sha1(sha))
+		return NULL;
+	hashcpy(target->sha1, sha);
+	target->mode = mode;
+	return target;
+}
+
 static void conflict_rename_rename_1to2(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
@@ -1107,8 +1120,12 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		remove_file_from_cache(a->path);
 		remove_file_from_cache(b->path);
 	} else {
-		update_stages(a->path, NULL, a, NULL);
-		update_stages(b->path, NULL, NULL, b);
+		struct diff_filespec other;
+		update_stages(a->path, NULL,
+			      a, filespec_from_entry(&other, ci->dst_entry1, 3));
+
+		update_stages(b->path, NULL,
+			      filespec_from_entry(&other, ci->dst_entry2, 2), b);
 
 		update_file(o, 0, a->sha1, a->mode, dst_name_a);
 		update_file(o, 0, b->sha1, b->mode, dst_name_b);
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index 86e96d0..d2d93e7 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -437,7 +437,7 @@ test_expect_success 'setup rename/rename(1to2)/add-dest conflict' '
 	git commit -m two
 '
 
-test_expect_failure 'rename/rename/add-dest merge still knows about conflicting file versions' '
+test_expect_success 'rename/rename/add-dest merge still knows about conflicting file versions' '
 	git checkout C^0 &&
 	test_must_fail git merge -s recursive B^0 &&
 
-- 
1.7.6.rc0.62.g2d69f
Elijah Newren· Jun 8, 2011, 07:31 UTC · re: Elijah Newren · lore

[PATCH 48/48] merge-recursive: Fix working copy handling for rename/rename/add/add

If either side of a rename/rename(1to2) conflict is itself also involved in a rename/add-dest conflict, then we need to make sure both the rename and the added file appear in the working copy.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                    |   73 ++++++++++++++++++++++------------
 t/t6039-merge-rename-corner-cases.sh |   11 +++++-
 2 files changed, 58 insertions(+), 26 deletions(-)
Show changes to 2 files +58 −26

merge-recursive.c, t/t6039-merge-rename-corner-cases.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 0cfc215..612a8cb 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1075,6 +1075,52 @@ static struct diff_filespec* filespec_from_entry(struct diff_filespec *target,
 	return target;
 }
 
+static void handle_file(struct merge_options *o,
+			struct diff_filespec *rename,
+			int stage,
+			struct rename_conflict_info *ci)
+{
+	char *dst_name = rename->path;
+	struct stage_data *dst_entry;
+	const char *cur_branch, *other_branch;
+	struct diff_filespec other;
+	struct diff_filespec *add;
+
+	if (stage == 2) {
+		dst_entry = ci->dst_entry1;
+		cur_branch = ci->branch1;
+		other_branch = ci->branch2;
+	} else {
+		dst_entry = ci->dst_entry2;
+		cur_branch = ci->branch2;
+		other_branch = ci->branch1;
+	}
+
+	add = filespec_from_entry(&other, dst_entry, stage ^ 1);
+	if (stage == 2)
+		update_stages(rename->path, NULL, rename, add);
+	else
+		update_stages(rename->path, NULL, add, rename);
+
+	if (add) {
+		char *add_name = unique_path(o, rename->path, other_branch);
+		update_file(o, 0, add->sha1, add->mode, add_name);
+
+		remove_file(o, 0, rename->path, 0);
+		dst_name = unique_path(o, rename->path, cur_branch);
+	} else {
+		if (dir_in_way(rename->path, !o->call_depth)) {
+			dst_name = unique_path(o, rename->path, cur_branch);
+			output(o, 1, "%s is a directory in %s adding as %s instead",
+			       rename->path, other_branch, dst_name);
+		}
+	}
+	update_file(o, 0, rename->sha1, rename->mode, dst_name);
+
+	if (dst_name != rename->path)
+		free(dst_name);
+}
+
 static void conflict_rename_rename_1to2(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
@@ -1082,10 +1128,6 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 	struct diff_filespec *one = ci->pair1->one;
 	struct diff_filespec *a = ci->pair1->two;
 	struct diff_filespec *b = ci->pair2->two;
-	const char *dst_name_a = a->path;
-	const char *dst_name_b = b->path;
-	char *del[2];
-	int delp = 0;
 
 	output(o, 1, "CONFLICT (rename/rename): "
 	       "Rename \"%s\"->\"%s\" in branch \"%s\" "
@@ -1093,16 +1135,6 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 	       one->path, a->path, ci->branch1,
 	       one->path, b->path, ci->branch2,
 	       o->call_depth ? " (left unresolved)" : "");
-	if (dir_in_way(a->path, !o->call_depth)) {
-		dst_name_a = del[delp++] = unique_path(o, a->path, ci->branch1);
-		output(o, 1, "%s is a directory in %s adding as %s instead",
-		       a->path, ci->branch2, dst_name_a);
-	}
-	if (dir_in_way(b->path, !o->call_depth)) {
-		dst_name_b = del[delp++] = unique_path(o, b->path, ci->branch2);
-		output(o, 1, "%s is a directory in %s adding as %s instead",
-		       b->path, ci->branch1, dst_name_b);
-	}
 	if (o->call_depth) {
 		struct merge_file_info mfi;
 		mfi = merge_file(o, one->path,
@@ -1120,18 +1152,9 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		remove_file_from_cache(a->path);
 		remove_file_from_cache(b->path);
 	} else {
-		struct diff_filespec other;
-		update_stages(a->path, NULL,
-			      a, filespec_from_entry(&other, ci->dst_entry1, 3));
-
-		update_stages(b->path, NULL,
-			      filespec_from_entry(&other, ci->dst_entry2, 2), b);
-
-		update_file(o, 0, a->sha1, a->mode, dst_name_a);
-		update_file(o, 0, b->sha1, b->mode, dst_name_b);
+		handle_file(o, a, 2, ci);
+		handle_file(o, b, 3, ci);
 	}
-	while (delp--)
-		free(del[delp]);
 }
 
 static void conflict_rename_rename_2to1(struct merge_options *o,
diff --git a/t/t6039-merge-rename-corner-cases.sh b/t/t6039-merge-rename-corner-cases.sh
index d2d93e7..9db1663 100755
--- a/t/t6039-merge-rename-corner-cases.sh
+++ b/t/t6039-merge-rename-corner-cases.sh
@@ -444,11 +444,20 @@ test_expect_success 'rename/rename/add-dest merge still knows about conflicting
 	test 5 -eq $(git ls-files -s | wc -l) &&
 	test 2 -eq $(git ls-files -u b | wc -l) &&
 	test 2 -eq $(git ls-files -u c | wc -l) &&
+	test 4 -eq $(git ls-files -o | wc -l) &&
 
 	test $(git rev-parse :2:b) = $(git rev-parse C:b) &&
 	test $(git rev-parse :3:b) = $(git rev-parse B:b) &&
 	test $(git rev-parse :2:c) = $(git rev-parse C:c) &&
-	test $(git rev-parse :3:c) = $(git rev-parse B:c)
+	test $(git rev-parse :3:c) = $(git rev-parse B:c) &&
+
+	test $(git hash-object c~HEAD) = $(git rev-parse C:c) &&
+	test $(git hash-object c~B\^0) = $(git rev-parse B:c) &&
+	test $(git hash-object b~HEAD) = $(git rev-parse C:b) &&
+	test $(git hash-object b~B\^0) = $(git rev-parse B:b) &&
+
+	test ! -f b &&
+	test ! -f c
 '
 
 test_expect_success 'setup simple rename/rename (1to2) conflict' '
-- 
1.7.6.rc0.62.g2d69f
Junio C Hamano· Jun 11, 2011, 18:12 UTC · re: Elijah Newren · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

Elijah Newren <newren@gmail.com> writes:
> You can note that I did not fix all the testcases I added.  I consider
> some cases either unfixable or not worth fixing.

Thanks for working on these. A quick comment (after reading a handful of tests in the early part of the series) I have on rename cases is that the test might be aiming too high to be too strict in "undetected rename" cases. Consider:

 - We start from commit A that has path file1 where commit B and C fork
   from;
 - One side, commit B, renames file1 to file2 and edits file2;
 - The other side, commit C, modifies file1.

When merging B and C, we will ideally want to see file1 removed and content level merge done between C's edit to file1 and B's edit to file2 left in file2. The content level merge may or may not conflict, and if it cleanly merges, then that is Ok.

But if B's edit is too extensive, we may see what B did as "delete file1, add file2", and we may report conflict at file1 (delete/modify) while resolving file2 cleanly (one side added). When notifying the user with delete/modify conflict at file1, we should make him aware that among the paths added by the merge (i.e. file2) there might be a corresponding rewrite of it on the other side, so that such a case can be manually merged. As long as that is done, a test of such a case should consider both clean merge (rename noticed) and a conflict at the tree structure level (delete/modify with new) valid expected result, I think.

To put it another way, merge-recursive should expect that one side of history may rewrite a path it renames beyond recognition, and when punting, loudly ask the user for help, with enough information to help the user to help it (e.g. perhaps say "file1 has delete/modify conflict; the deleted side has added file2---you the user might want to inspect it and see if it is a rename with extensive rewrite and resolve it accordingly if that is the case"). We might want to dig deeper at that point by checking the similarity again between the two blobs ourselves at that stage before punting.

If we change what C did to "removes file1", instead of "modifies file1", the story changes.

When merging B and C, that would be rename/delete conflict (B renamed, C deleted). However, if we see that B's history as "delete file1 and add file2", it would merge cleanly with C that deletes file1. The end result would be an addition of file2. This tastes bad, as it will end up with a clean merge that the user may not even notice.

Ideally we would want the merge to somehow warn the user to see if the added "file2" still is wanted, and the worst part of this problem is that this cannot be mechanically inferred. We could warn against every merge that has a history that adds new paths and removes some other paths, but then the warning becomes meaningless.

The changes done by B to the other parts of the tree (that wasn't involved in the conflict) still want the updated content in file2 to work correctly, and the changes done by C will not want the original content in file1 (now in file2). The latter may or may not care about the presense of what was added to "file2" by B. If C's work was to remove file1 that was nothing but dead code (and other parts of C's work removed all the callsite of code in original "file1" to make them dead), while B's work was to add a code with purpose similar to the original code "file1" had (and other parts of B's work calls that new code), then the right merge result of "file2" might be to keep the line removal of dead code made by C while keeping the code added by B there. So (I am thinking aloud here without thinking things through) a possible approach to this issue might involve considering a removal of a path and modifying the contents of a path to make it empty similarly.

Show 7 quoted lines
> However, there is one large class of problems that I think is fixable,
> I'm just not sure whether it is worth fixing.  git's rename detection
> optimization of only considering files that exist on one side of the
> diff but not the other causes issues with merges (undetected conflicts,
> spurious conflicts, or merged cleanly but wrongly due to deleting files
> that should be present).  To fix these cases, we would need some way of
> including rewritten files as potential rename candidates,...

This may be part of the larger "when punting, loudly ask the user for help, or we may want to go deeper". If we can come up with a way to avoid "undetected conflicts" ("merged cleanly but wrongly" is saying the same thing as "undetected conflicts"), maybe we can attempt a cheap way first, and then only when we know there is something complicated going on, we can redo the diff with -B/-M options on. Spurious conflicts can be handled the same way, as we should be able to come up with a clean merge once we dig deeper (that's the definition of "spurious", right?).

Junio C Hamano· Aug 4, 2011, 00:20 UTC · re: Elijah Newren · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

Unfortunately I seem to have found a regression that manifests in the real life.

When this series is merged to 'next', it mismerges a trivial renamed path. A sample commit to reproduce is 2d11f21c3, which is a merge between 28b9264dd6 and 5b42477b59. Check out the former and try to merge in the latter.

The merge base of these two commits is 2cfe8a68ccb (no criss-cross).

One branch (current one, 28b9264dd6) does not have that path, but it does have t/t4037-whitespace-status.sh:

    $ git ls-tree 28b9264dd6 t/t4037-whitespace-status.sh
    100755 blob 3c728a3ebf9ce52e5c24c81525d5cb749cfb2957 t/t4037-whitespace-status.sh
What happened to that path on the current branch is this:
    $ git log -m -M --raw --pretty=short 2cfe8a68ccb..28b9264dd6 -- \
      t/t40{37,40}-whitespace-status.sh
    commit af7b41c923677ff9291bab56ec7069922e37453b
    Author: Jeff King <peff@peff.net>
        diff_tree: disable QUICK optimization with diff filter
    :100755 100755 abc4934... 3c728a3... M  t/t4037-whitespace-status.sh

In other words, we had t4037, and we have t4037 but we updated its contents from abc4934 to 3c728a3.

Running "log -m -M --raw --pretty=short 2cfe8a68ccb..5b42477b59" shows that the merged branch (5b42477b59) has renamed t4037 to t4040 at 0e098b6d:

    commit 0e098b6d79fbcab763874f2b6fde5aa82144d150
    Author: Johannes Sixt <j6t@kdbg.org>
        Make test case number unique
    :100755 100755 a30b03b... a30b03b... R100 t/t4037-whitespace-status.sh    t/t4040-wh..

This commit is on a side-branch that has not been merged at the merge base 2cfe8a68ccb; its contents a30b03b is older than what 2cfe8a68ccb has, and was updated by 2cfe8a68ccb to abc4934.

Another merge after this commit updated it at 7d0cf357:
    commit 7d0cf357a31cc8a442342696788d776265482ce9 (from 98b256bd...)
    Merge: 98b256b 2cfe8a6
    Author: Junio C Hamano <gitster@pobox.com>
        Merge branch 'jc/maint-diff-q-filter'
    :100755 100755 a30b03b... abc4934... M  t/t4040-whitespace-status.sh
This abc4934 is the same contents as the merge base had at t4037.

So this is an "our side kept t4037 and updated its contents from abc4934 to 3c728a3, while the other side renamed t4037 to t4040 and kept its contents as abc4934". We should merge this as structural change "our side left it untouched, their side renamed, so take the rename without conflict" and content level change "our side updated but their side left it untouched, so take our modification without conflict".

In other words, in the result, we should have 3c728a3 at t4040, and that indeed is what we got in the recorded history:

    $ git ls-tree 2d11f21c3 -- t/t4040-whitespace-status.sh
    100755 blob 3c728a3ebf9ce52e5c24c81525d5cb749cfb2957 t/t4040-whitespace-status.sh

However, the merge retried with this series cleanly resolves the path, but with wrong contents:

    $ git checkout 28b9264dd6
    $ git merge 5b42477b59
    $ git ls-files -s t/t4040-whitespace-status.sh
    100755 abc49348b196cf0fec232b6f2399356e4fe324d5 0 t/t4040-whitespace-status.sh
Correct answer should be:
    $ git ls-files -s t/t4040-whitespace-status.sh
    100755 3c728a3ebf9ce52e5c24c81525d5cb749cfb2957 0 t/t4040-whitespace-status.sh

I am rewinding today's integration of 'next' to unmerge this topic now. :-(...

Junio C Hamano· Aug 4, 2011, 01:48 UTC · re: Junio C Hamano · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

A very simple reproduction recipe.

-- >8 -- #!/bin/sh

mkdir en && cd en || exit
git init

echo 1 >one git add one git commit -m 'origin'

git checkout -b side git mv one two git commit -m 'side renames one to two'

git checkout master echo 2 >one git add one git commit -m 'master updates one'

git checkout HEAD^0 git merge side -- 8< --

Tonight's "pu" fails like this:
$ git merge side
error: addinfo_cache failed for path 'two'
Merge made by the 'recursive' strategy.
 one |    1 -
 two |    1 +
 2 files changed, 1 insertions(+), 1 deletions(-)
 delete mode 100644 one
 create mode 100644 two
Elijah Newren· Aug 4, 2011, 02:12 UTC · re: Junio C Hamano · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

On Wed, Aug 3, 2011 at 7:48 PM, Junio C Hamano <gitster@pobox.com> wrote:
> A very simple reproduction recipe.

Thanks, I'll take a look this weekend. I'm a little surprised that you mentioned merging this to next already; I was assuming you wanted me to address the various things you pointed out in your reviews first (as well as the comments from Johannes). I've been swamped at work, particularly after trying to catch up on the pile of work accumulated during my vacation, but should have time again this weekend...finally.

Elijah
Elijah Newren· Aug 4, 2011, 17:26 UTC · re: Junio C Hamano · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

On Wed, Aug 3, 2011 at 6:20 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Unfortunately I seem to have found a regression that manifests in the real
> life.
>
> When this series is merged to 'next', it mismerges a trivial renamed path.

I know the problem, and can think of a couple potential solutions but I'm not sure which one of those (if any) should be used. I may need advice from an unpack_trees expert.

This breakage was introduced in d487486 (merge-recursive: When we detect we can skip an update, actually skip it 2011-06-08), and the reason for the breakage is that was_tracked(path) is lying to us. Let me explain...

If you run with GIT_MERGE_VERBOSITY=3, you'll note that you get the message
  "Skipped two (merged same as existing)"
which is NOT the path it should be taking.  The reason for this is
that the call to was_tracked('two') just before that is returning true
for us when the correct answer is false.  Explaining why we get the
wrong answer requires understanding a couple things.
Using your simple reproduction recipe as an example to explain this,
the sha1sums and paths for your merge case are:
  Base commit: d00491f... one
  HEAD commit: 0cfbf08... one
  side commit: d00491f... two
In other words, 'one' is modified in HEAD, and unchanged other than
being renamed (to 'two') on side.

Now, was_tracked() simply checks whether there is a cache entry in the current index with either stage 0 or stage 2. Clearly, I was expecting 'two' to only appear with stage 3, but that's not what happened.

When unpacking these trees, we got three cache entries: one for path 'one' at stage 1, one for path 'one' at stage 2, and one for path 'two' at stage 0. Yes, stage 0 and not stage 3. The reason for this is that this is case 2ALT from Documentation/technical/trivial-merge.txt:

case ancest head remote result ---------------------------------------- ... 2ALT (empty)+ *empty* remote remote

This means the threeway_merge code considers it a clean merge and calls merged_entry on it, which clears the CE_STAGEMASK bits from the cache entry flags.

Since the cache entry for path 'two' has the CE_STAGEMASK bits cleared, when we call was_tracked on that path, we get back the answer true. 'two' was not tracked in HEAD, though, so this is the wrong answer and the source of the bug.

Now, the question is...how do we fix the was_tracked function? It would be nice to make use of the original index we had before unpacking, but that is overwritten at the end of unpack_trees. Should we save it somewhere? Or should we save off all the tracked filenames before unpacking into a linked list and then have was_tracked() use that instead of looking it up in the current cache? (I'm slightly worried that may be slow given the number of lookups that exist).

Thoughts?
Elijah
Junio C Hamano· Aug 4, 2011, 19:03 UTC · re: Elijah Newren · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

Elijah Newren <newren@gmail.com> writes:
> ...  It
> would be nice to make use of the original index we had before
> unpacking, but that is overwritten at the end of unpack_trees.

I somehow thought that we can give separate src and dst index to the unpack_tree() machinery these days. Aren't you using it?

Elijah Newren· Aug 4, 2011, 19:16 UTC · re: Junio C Hamano · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

On Thu, Aug 4, 2011 at 1:03 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 8 quoted lines
> Elijah Newren <newren@gmail.com> writes:
>
>> ...  It
>> would be nice to make use of the original index we had before
>> unpacking, but that is overwritten at the end of unpack_trees.
>
> I somehow thought that we can give separate src and dst index
> to the unpack_tree() machinery these days. Aren't you using it?
Ah, yes, it appears to be possible.  I have not touched that part of
merge-recursive, so it still reads:
	opts.fn = threeway_merge;
	opts.src_index = &the_index;
	opts.dst_index = &the_index;
	setup_unpack_trees_porcelain(&opts, "merge");
I'll take that route to fix this.
Thanks.
Junio C Hamano· Aug 6, 2011, 05:22 UTC · re: Elijah Newren · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

Elijah Newren <newren@gmail.com> writes:
Show 11 quoted lines
> On Thu, Aug 4, 2011 at 1:03 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Elijah Newren <newren@gmail.com> writes:
>>
>>> ... It
>>> would be nice to make use of the original index we had before
>>> unpacking, but that is overwritten at the end of unpack_trees.
>>
>> I somehow thought that we can give separate src and dst index
>> to the unpack_tree() machinery these days. Aren't you using it?
>
> Ah, yes, it appears to be possible.

But why do you care about the original index (i.e. the starting state of our side) in the first place? If your algorithm depends on the original index, wouldn't that mean you would screw up the same merge if the user merged branches in the opposite direction?

I think the fact you have a path "two" at stage 0, combined with the two diffs you ran for rename detection between the common ancestor and two branches, should be enough to decide if one branch added the path or moved it from elsewhere (in which case the common ancestor would not have that path), or it kept the path at the original place (with or without content change), no?

Elijah Newren· Aug 6, 2011, 20:31 UTC · re: Junio C Hamano · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

On Fri, Aug 5, 2011 at 11:22 PM, Junio C Hamano <gitster@pobox.com> wrote:
> But why do you care about the original index (i.e. the starting state of
> our side) in the first place? If your algorithm depends on the original
> index, wouldn't that mean you would screw up the same merge if the user
> merged branches in the opposite direction?

No, it does not mess up the same merge in the opposite direction. The code in question (with some local modifications) is

       if (mfi.clean && !df_conflict_remains &&
           sha_eq(mfi.sha, a_sha) && mfi.mode == a_mode) {
               output(o, 3, "Skipped %s (merged same as existing)", path);
               /*
                * The content merge resulted in the same file contents we
                * already had.  We can return early if those file contents
                * are recorded at the correct path (which may not be true
                * if the merge involves a rename).
                */
               if (was_tracked(&o->original_index, path)) {
                       add_cacheinfo(mfi.mode, mfi.sha, path,
                                     0 /*stage*/, 1 /*refresh*/, 0 /*options*/)
                       return mfi.clean;
               }
       } else
                output(o, 2, "Auto-merging %s", path);
And the merge is:
  BASE: original-path: v1
  HEAD: original-path: v2
  SIDE: renamed-path: v1

So, HEAD did have the correct contents that we wanted, as checked in the first if. However, we cannot bail early because those contents were recorded at the wrong path. Repeating the merge in the other direction would simply show that we don't have the right contents (a_sha != mfi.sha). Only if we have both the right contents and have it at the right path should we exit early.

If we use "&the_index" instead of "&o->original_index", which is what my previous series essentially did, then we get the wrong answer about whether the contents are recorded at the necessary path, i.e. was_tracked() lies to us.

Now, your gut feel and question here does turn out to be important...in a different case. You'll note that I'm passing an index to was_tracked(); that's because there is one case (the call to was_tracked() from would_lose_untracked()) where it is important that we use the index as modified by unpack_trees rather than the original index and not doing so causes a bug depending on the direction things are merged. I added a big comment in the code about that one.

Show 6 quoted lines
> I think the fact you have a path "two" at stage 0, combined with the two
> diffs you ran for rename detection between the common ancestor and two
> branches, should be enough to decide if one branch added the path or moved
> it from elsewhere (in which case the common ancestor would not have that
> path), or it kept the path at the original place (with or without content
> change), no?

You are correct. I just need to reorder the patch series somewhat (other patches added the passing of the relevant diff_filepair information to merge_content, which is needed to do these checks you suggest), and then I can change the was_tracked() call to instead compare pathnames.

Junio C Hamano· Jun 12, 2011, 06:18 UTC · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

Elijah Newren <newren@gmail.com> writes:
> It does sound potentially expensive, though, and might mean a lot
> more work in merge-recursive to handle that extra information.  Is that a
> path we want to take at some point?

Probably you can start with backend specific option (e.g. -Xbreak=yes) to experiment. We made recursive the default not because it deals with renames (in a broken way) but primarily because it handles criss-cross better; at some point we might also want to add another backend specific option (e.g. -Xrename=off) to allow the users to keep the "recursive" aspect of the strategy while declining a more expensive (and brittle) rename handling to take effect.

My gut feeling is that -Xbreak=yes, once the code does work well enough, would have to become the default. It would make the default mode of merge possibly quite expensive but it is Ok as long as we give projects with simple/clean history an easy way to use either "recursive -Xrename=off" or even "resolve" to avoid cost that is unnecessary to handle their needs.

Junio C Hamano· Jun 12, 2011, 06:28 UTC · re: Junio C Hamano · lore

Re: [PATCH 00/48] Handling more corner cases in merge-recursive.c

Junio C Hamano <gitster@pobox.com> writes:
Show 19 quoted lines
> Elijah Newren <newren@gmail.com> writes:
>
>> It does sound potentially expensive, though, and might mean a lot
>> more work in merge-recursive to handle that extra information.  Is that a
>> path we want to take at some point?
>
> Probably you can start with backend specific option (e.g. -Xbreak=yes) to
> experiment. We made recursive the default not because it deals with
> renames (in a broken way) but primarily because it handles criss-cross
> better; at some point we might also want to add another backend specific
> option (e.g. -Xrename=off) to allow the users to keep the "recursive"
> aspect of the strategy while declining a more expensive (and brittle)
> rename handling to take effect.
>
> My gut feeling is that -Xbreak=yes, once the code does work well enough,
> would have to become the default. It would make the default mode of merge
> possibly quite expensive but it is Ok as long as we give projects with
> simple/clean history an easy way to use either "recursive -Xrename=off" or
> even "resolve" to avoid cost that is unnecessary to handle their needs.

I forgot to mention that there is jk/maint-merge-rename-create effort by Peff already queued in 'pu' to use "break", which of course has unfortunate conflicts with your series.

← back to recent threads