threads / patch / 19282

patchAdd a reminder test case for a merge with F/D transition

Subject: [PATCH] Add a reminder test case for a merge with F/D transition

## tl;dr

8 messages between May 11, 2009 and May 20, 2009. Diffs are folded; open one to read it.

replies: 7people: 3as markdown or json

Alex Riesen· May 11, 2009, 09:42 UTC · lore

The problem is that if a file was replaced with a directory containing another file with the same content and mode, an attempt to merge it with a branch descended from a commit before this F->D transition will cause merge-recursive to break. It breaks even if there were no conflicting changes on that other branch.

Originally reported by Anders Melchiorsen.
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
2009/5/11 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
>
> Maybe you can turn this into a patch adding a test (with
> test_expect_failure to mark it as a bug)?  This would make debugging a lot
> easier, as a non-installed Git could be tested.
Here.
 t/t6020-merge-df.sh |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)
Show changes to t/t6020-merge-df.sh +23 −0
diff --git a/t/t6020-merge-df.sh b/t/t6020-merge-df.sh
index a19d49d..b62b52a 100755
--- a/t/t6020-merge-df.sh
+++ b/t/t6020-merge-df.sh
@@ -22,4 +22,27 @@ git commit -m "File: dir"'

 test_expect_code 1 'Merge with d/f conflicts' 'git merge "merge msg" B master'

+test_expect_failure 'F/D conflict' '
+	git reset --hard &&
+	git checkout master &&
+	rm .git/index &&
+
+	mkdir before &&
+	echo FILE >before/one &&
+	echo FILE >after &&
+	git add . &&
+	git commit -mfirst &&
+
+	rm -f after &&
+	git mv before after &&
+	git commit -mmove &&
+
+	git checkout -b para HEAD^ &&
+	echo COMPLETELY ANOTHER FILE >another &&
+	git add . &&
+	git commit -mpara &&
+
+	git merge master
+'
+
 test_done

-- 
1.6.3.28.ga852b


From 05433a6f896c40e4a039815d86613d7e351ef0da Mon Sep 17 00:00:00 2001
From: Alex Riesen <raa.lkml@gmail.com>
Date: Mon, 11 May 2009 11:31:42 +0200
Subject: [PATCH] Add a reminder test case for a merge with F/D transition

The problem is that if a file was replaced with a directory containing
another file with the same content and mode, an attempt to merge it
with a branch descended from a commit before this F->D transition will
cause merge-recursive to break. It breaks even if there were no
conflicting changes on that other branch.

Originally reported by Anders Melchiorsen.

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
 t/t6020-merge-df.sh |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/t/t6020-merge-df.sh b/t/t6020-merge-df.sh
index a19d49d..b62b52a 100755
--- a/t/t6020-merge-df.sh
+++ b/t/t6020-merge-df.sh
@@ -22,4 +22,27 @@ git commit -m "File: dir"'
 
 test_expect_code 1 'Merge with d/f conflicts' 'git merge "merge msg" B master'
 
+test_expect_failure 'F/D conflict' '
+	git reset --hard &&
+	git checkout master &&
+	rm .git/index &&
+
+	mkdir before &&
+	echo FILE >before/one &&
+	echo FILE >after &&
+	git add . &&
+	git commit -mfirst &&
+
+	rm -f after &&
+	git mv before after &&
+	git commit -mmove &&
+
+	git checkout -b para HEAD^ &&
+	echo COMPLETELY ANOTHER FILE >another &&
+	git add . &&
+	git commit -mpara &&
+
+	git merge master
+'
+
 test_done
-- 
1.6.3.28.ga852b
Alex Riesen· May 11, 2009, 19:25 UTC · re: Alex Riesen · lore

[PATCH] Fix for a merge where a branch has an F->D transition

Some path names which transitioned from file to a directory were not updated in the final part of the merge (loop around unmerged entries in merge_trees), because the branch in process_renames which filtered out updates for the files with the same content ("merged same as existing") has left the rename entry in processed state. In this case, the processing cannot be finished at the process_renames phase (because the old file still blocks creation of directory where new files should appear), and must be postponed until the update_entry phase.

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
Alex Riesen, Mon, May 11, 2009 11:42:17 +0200:
Show 17 quoted lines
> The problem is that if a file was replaced with a directory containing
> another file with the same content and mode, an attempt to merge it
> with a branch descended from a commit before this F->D transition will
> cause merge-recursive to break. It breaks even if there were no
> conflicting changes on that other branch.
> 
> 2009/5/11 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
> >
> > Maybe you can turn this into a patch adding a test (with
> > test_expect_failure to mark it as a bug)?  This would make debugging a lot
> > easier, as a non-installed Git could be tested.
> 
> Here.
> 
>  t/t6020-merge-df.sh |   23 +++++++++++++++++++++++
>  1 files changed, 23 insertions(+), 0 deletions(-)
>

Frankly, I'm not really sure. The solution came largely ... empirical way. IOW, I tried more or less random things which looked like they should fix the problem. So a review is very much appreciated. Please.

 merge-recursive.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +3 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index a3721ef..3c5420b 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -980,14 +980,15 @@ static int process_renames(struct merge_options *o,
 
 				if (mfi.clean &&
 				    sha_eq(mfi.sha, ren1->pair->two->sha1) &&
-				    mfi.mode == ren1->pair->two->mode)
+				    mfi.mode == ren1->pair->two->mode) {
 					/*
 					 * This messaged is part of
 					 * t6022 test. If you change
 					 * it update the test too.
 					 */
 					output(o, 3, "Skipped %s (merged same as existing)", ren1_dst);
-				else {
+					ren1->dst_entry->processed = 0;
+				} else {
 					if (mfi.merge || !mfi.clean)
 						output(o, 1, "Renaming %s => %s", ren1_src, ren1_dst);
 					if (mfi.merge)

-- 
1.6.3.28.ga852b
Junio C Hamano· May 13, 2009, 06:19 UTC · re: Alex Riesen · lore

Re: [PATCH] Fix for a merge where a branch has an F->D transition

Alex Riesen <raa.lkml@gmail.com> writes:
> Frankly, I'm not really sure. The solution came largely ... empirical
> way. IOW, I tried more or less random things which looked like they
> should fix the problem. So a review is very much appreciated. Please.

I've always thought that D/F conflict logic in merge-recursive is placed at the wrong processing phase. IIRC, it enumerates potential D/F conflicting paths before even attempting to process renames, and it is easy to miss a path that was previously file going away as the result of a clean merge (in which case it is ok to have a directory there as the result of a merge for other paths). This breakage could be a small example of it.

Regardless, I think your patch is a reasonable fix to go to the maintenance track. Thanks for looking into it.

Show 27 quoted lines
>  merge-recursive.c |    5 +++--
>  1 files changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index a3721ef..3c5420b 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -980,14 +980,15 @@ static int process_renames(struct merge_options *o,
>  
>  				if (mfi.clean &&
>  				    sha_eq(mfi.sha, ren1->pair->two->sha1) &&
> -				    mfi.mode == ren1->pair->two->mode)
> +				    mfi.mode == ren1->pair->two->mode) {
>  					/*
>  					 * This messaged is part of
>  					 * t6022 test. If you change
>  					 * it update the test too.
>  					 */
>  					output(o, 3, "Skipped %s (merged same as existing)", ren1_dst);
> -				else {
> +					ren1->dst_entry->processed = 0;
> +				} else {
>  					if (mfi.merge || !mfi.clean)
>  						output(o, 1, "Renaming %s => %s", ren1_src, ren1_dst);
>  					if (mfi.merge)
> -- 
> 1.6.3.28.ga852b
Alex Riesen· May 13, 2009, 06:38 UTC · re: Junio C Hamano · lore

Re: [PATCH] Fix for a merge where a branch has an F->D transition

2009/5/13 Junio C Hamano <gitster@pobox.com>:
Show 15 quoted lines
> Alex Riesen <raa.lkml@gmail.com> writes:
>> Frankly, I'm not really sure. The solution came largely ... empirical
>> way. IOW, I tried more or less random things which looked like they
>> should fix the problem. So a review is very much appreciated. Please.
>
> I've always thought that D/F conflict logic in merge-recursive is placed
> at the wrong processing phase.  IIRC, it enumerates potential D/F
> conflicting paths before even attempting to process renames, and it is
> easy to miss a path that was previously file going away as the result of a
> clean merge (in which case it is ok to have a directory there as the
> result of a merge for other paths).  This breakage could be a small
> example of it.
>
> Regardless, I think your patch is a reasonable fix to go to the
> maintenance track.  Thanks for looking into it.

I'm afraid the fix is not that simple: the "if" branch where the change is placed supposed to prevent updating files in the working tree which already have the same content as the merge's output. My change may force them to be updated regardless. I think... Johannes, you know this area the best, could you please have a look?

Johannes Schindelin· May 13, 2009, 09:59 UTC · re: Alex Riesen · lore

Re: [PATCH] Fix for a merge where a branch has an F->D transition

Hi,
On Wed, 13 May 2009, Alex Riesen wrote:
Show 23 quoted lines
> 2009/5/13 Junio C Hamano <gitster@pobox.com>:
> > Alex Riesen <raa.lkml@gmail.com> writes:
> >> Frankly, I'm not really sure. The solution came largely ... empirical
> >> way. IOW, I tried more or less random things which looked like they
> >> should fix the problem. So a review is very much appreciated. Please.
> >
> > I've always thought that D/F conflict logic in merge-recursive is placed
> > at the wrong processing phase.  IIRC, it enumerates potential D/F
> > conflicting paths before even attempting to process renames, and it is
> > easy to miss a path that was previously file going away as the result of a
> > clean merge (in which case it is ok to have a directory there as the
> > result of a merge for other paths).  This breakage could be a small
> > example of it.
> >
> > Regardless, I think your patch is a reasonable fix to go to the
> > maintenance track.  Thanks for looking into it.
> 
> I'm afraid the fix is not that simple: the "if" branch where the change
> is placed supposed to prevent updating files in the working tree
> which already have the same content as the merge's output.
> My change may force them to be updated regardless. I think...
> Johannes, you know this area the best, could you please have
> a look?
Sorry, no time at the moment...

Ciao, Dscho

Alex Riesen· May 13, 2009, 11:33 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Fix for a merge where a branch has an F->D transition

2009/5/13 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
Show 27 quoted lines
> On Wed, 13 May 2009, Alex Riesen wrote:
>> 2009/5/13 Junio C Hamano <gitster@pobox.com>:
>> > Alex Riesen <raa.lkml@gmail.com> writes:
>> >> Frankly, I'm not really sure. The solution came largely ... empirical
>> >> way. IOW, I tried more or less random things which looked like they
>> >> should fix the problem. So a review is very much appreciated. Please.
>> >
>> > I've always thought that D/F conflict logic in merge-recursive is placed
>> > at the wrong processing phase.  IIRC, it enumerates potential D/F
>> > conflicting paths before even attempting to process renames, and it is
>> > easy to miss a path that was previously file going away as the result of a
>> > clean merge (in which case it is ok to have a directory there as the
>> > result of a merge for other paths).  This breakage could be a small
>> > example of it.
>> >
>> > Regardless, I think your patch is a reasonable fix to go to the
>> > maintenance track.  Thanks for looking into it.
>>
>> I'm afraid the fix is not that simple: the "if" branch where the change
>> is placed supposed to prevent updating files in the working tree
>> which already have the same content as the merge's output.
>> My change may force them to be updated regardless. I think...
>> Johannes, you know this area the best, could you please have
>> a look?
>
> Sorry, no time at the moment...
>

Maybe the patch could live on master or next for while? Many people track master, so a performance degradation, if it happens, will be noticed. And with master (or even better - next) we have an auditory which is at least capable to complain.

Johannes Schindelin· May 20, 2009, 10:34 UTC · re: Alex Riesen · lore

Re: [PATCH] Fix for a merge where a branch has an F->D transition

Hi,
On Mon, 11 May 2009, Alex Riesen wrote:
Show 8 quoted lines
> Some path names which transitioned from file to a directory were not
> updated in the final part of the merge (loop around unmerged entries in
> merge_trees), because the branch in process_renames which filtered out
> updates for the files with the same content ("merged same as existing")
> has left the rename entry in processed state. In this case, the
> processing cannot be finished at the process_renames phase (because
> the old file still blocks creation of directory where new files should
> appear), and must be postponed until the update_entry phase.

I know that as a German, I am supposed to like long sentences and crowded paragraphs. Maybe I am not that German after all.

> Frankly, I'm not really sure. The solution came largely ... empirical 
> way. IOW, I tried more or less random things which looked like they 
> should fix the problem.

This does not give me the cozy feeling I need to review a patch. After all, I do not want to do all the work that should be done by the patch author, but I just want to put in my knowledge to verify that the patch is correct.

But as you asked me explicitely...
Show 19 quoted lines
> diff --git a/merge-recursive.c b/merge-recursive.c
> index a3721ef..3c5420b 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -980,14 +980,15 @@ static int process_renames(struct merge_options *o,
>  
>  				if (mfi.clean &&
>  				    sha_eq(mfi.sha, ren1->pair->two->sha1) &&
> -				    mfi.mode == ren1->pair->two->mode)
> +				    mfi.mode == ren1->pair->two->mode) {
>  					/*
>  					 * This messaged is part of
>  					 * t6022 test. If you change
>  					 * it update the test too.
>  					 */
>  					output(o, 3, "Skipped %s (merged same as existing)", ren1_dst);
> -				else {
> +					ren1->dst_entry->processed = 0;
> +				} else {

So basically, you say that a dst_entry has not been processed, when it _has_ been? That cannot be correct...

Ciao, Dscho

Alex Riesen· May 20, 2009, 22:09 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Fix for a merge where a branch has an F->D transition

2009/5/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
Show 14 quoted lines
> On Mon, 11 May 2009, Alex Riesen wrote:
>
>> Some path names which transitioned from file to a directory were not
>> updated in the final part of the merge (loop around unmerged entries in
>> merge_trees), because the branch in process_renames which filtered out
>> updates for the files with the same content ("merged same as existing")
>> has left the rename entry in processed state. In this case, the
>> processing cannot be finished at the process_renames phase (because
>> the old file still blocks creation of directory where new files should
>> appear), and must be postponed until the update_entry phase.
>
> I know that as a German, I am supposed to like long sentences and crowded
> paragraphs.  Maybe I am not that German after all.
>
Will be shortened.
Show 23 quoted lines
>> diff --git a/merge-recursive.c b/merge-recursive.c
>> index a3721ef..3c5420b 100644
>> --- a/merge-recursive.c
>> +++ b/merge-recursive.c
>> @@ -980,14 +980,15 @@ static int process_renames(struct merge_options *o,
>>
>>                               if (mfi.clean &&
>>                                   sha_eq(mfi.sha, ren1->pair->two->sha1) &&
>> -                                 mfi.mode == ren1->pair->two->mode)
>> +                                 mfi.mode == ren1->pair->two->mode) {
>>                                       /*
>>                                        * This messaged is part of
>>                                        * t6022 test. If you change
>>                                        * it update the test too.
>>                                        */
>>                                       output(o, 3, "Skipped %s (merged same as existing)", ren1_dst);
>> -                             else {
>> +                                     ren1->dst_entry->processed = 0;
>> +                             } else {
>
> So basically, you say that a dst_entry has not been processed, when it
> _has_ been?  That cannot be correct...
>

My feeling exactly. I'm afraid I just zeroed the effect of the branch which should avoid rewriting of files with known same content. It's just that the code has grown a little confusing... I'm still thinking about how to avoid unneeded rewrites of the files with same content under same name.

← back to recent threads