threads / discuss / 56540

Question about git merge -s ort command‏‏

Subject: Question about git merge -s ort command‏‏

## tl;dr

4 messages between Sep 19, 2021 and Oct 1, 2021.

replies: 3people: 2as markdown or json

Ariel Man· Sep 19, 2021, 21:39 UTC · lore

Hi, following the below thread, led me to do some tests: https://stackoverflow.com/a/69114710/11705021

One of the test can be seen here, in the last example in my question: https://stackoverflow.com/questions/69150777/git-mv-doesnt-work-as-expected-wrong-files-location

I wanted to get confirmation for my case: Does `git merge -s ort` command, doesn't know to detect new folder location when merging it? It did worked for folders and files renaming, and new files under existing folders (as describe here: https://stackoverflow.com/a/52227201/6309)

Ariel
Elijah Newren· Sep 30, 2021, 05:27 UTC · re: Ariel Man · lore

Re: Question about git merge -s ort command‏‏

On Sun, Sep 19, 2021 at 8:12 PM Ariel Man <arielman22@gmail.com> wrote:
Show 13 quoted lines
>
> Hi,
> following the below thread, led me to do some tests:
> https://stackoverflow.com/a/69114710/11705021
>
> One of the test can be seen here, in the last example in my question:
> https://stackoverflow.com/questions/69150777/git-mv-doesnt-work-as-expected-wrong-files-location
>
> I wanted to get confirmation for my case:
> Does `git merge -s ort` command, doesn't know to detect new folder
> location when merging it? It did worked for folders and files
> renaming, and new files under existing folders (as describe here:
> https://stackoverflow.com/a/52227201/6309)

I mean, if you have a testcase and want to see how it works, you can just try it. merge-ort has been released and is part of git already.

But I'll take a stab...

Your testcase with new folders sounds like testcase 12f from t6423 in our testsuite (from commit 902c521a35 ("t6423: more involved directory rename test", 2020-10-15)), a case that works with the `ort` merge strategy but not with `recursive`. So, the `ort` strategy is likely to be better for your usecase than `recursive`, but it's hard for me to definitively state that it'll solve all your usecase(s) since you didn't fully describe them. This is especially important since you appear to have discovered a segfault (in the first post); I was not aware of any circumstances when the old merge machinery would do so. I'd be interested to know if the same issue repeats with the `ort` strategy, and, if so, whether you can get a stack trace, or find an example that reproduces the problem, or provide us with access to your repository. Also, in your second link, you allude to problems in a case that does not include new folders, in situations where I'd expect `recursive` and `ort` to behave the same. But it's hard to know the expected behavior since you didn't provide that many details about the setup -- e.g. you didn't cover the contents of any of the files (so it's hard to know if files are more similar to ones with different names than what you expected to be matched up), it's unclear if there are other files involved (e.g. if you have multiple copies of some library and the rename detection is detecting a different copy than you expected, or if there are lots of additional files besides those you are listing in your example and they are being renamed differently, etc.), whether you have lots of large files also being renamed (perhaps the VDI files you allude to somewhere?), etc., etc. If you can provide access to a repository and the commands you reported and what you expected vs. what you saw, I'd try to find some time to take a look.

I hope that helps, though I'm not sure I've got enough details to
answer all your questions.  I guess if I were to summarize, I'd say:
  * the `ort` strategy would be better than `recursive` for you, for
one particular type of issue, but probably behave the same on other
areas
  * you seem to have useful testcases that might be different than
ones we have in our testsuite or that I know about
  * you haven't provided enough information for me to investigate or
answer any further, though.

You have gotten me really curious about this segfaulting testcase -- whether it's just in the recursive backend or also afflicts merge-ort, and how to duplicate it.

Ariel Man· Sep 30, 2021, 06:57 UTC · re: Elijah Newren · lore

Re: Question about git merge -s ort command‏‏

Hi, Thanks for the detailed answer, but just to clarify, my main question is regarding ORT strategy, and not recursive. Mainly in this case, where ORT didn't behave as I expected: https://stackoverflow.com/questions/69150777/git-mv-doesnt-work-as-expected-wrong-files-location#:~:text=test3.py%0A%E2%94%82%20%20%20%E2%94%94%E2%94%80%E2%94%80%20dir3-,UPDATE,-I%27m%20adding%20one

As I wrote, this specific case is where new folder and files were created after the consolidation (and not new files under the existing folder). As you can see, when only new files were added to the existing folder, the git ORT recognized it and moved the folder python1 with its files test1.py and test4.py to the right place (dir2). But when new folder was created on the target (new_test with the file test5.py), when I ran the ORT command, the GIT didn't recognized it and created again the folder python1 on the old path (dir1-->python1-->new_test-->test5.py). I believe you can reproduce it yourself on your repository and see the behaviour. Ariel

‫בתאריך יום ה׳, 30 בספט׳ 2021 ב-8:27 מאת ‪Elijah Newren‬‏ <‪newren@gmail.com‬‏>:‬

Show 63 quoted lines
>
> On Sun, Sep 19, 2021 at 8:12 PM Ariel Man <arielman22@gmail.com> wrote:
> >
> > Hi,
> > following the below thread, led me to do some tests:
> > https://stackoverflow.com/a/69114710/11705021
> >
> > One of the test can be seen here, in the last example in my question:
> > https://stackoverflow.com/questions/69150777/git-mv-doesnt-work-as-expected-wrong-files-location
> >
> > I wanted to get confirmation for my case:
> > Does `git merge -s ort` command, doesn't know to detect new folder
> > location when merging it? It did worked for folders and files
> > renaming, and new files under existing folders (as describe here:
> > https://stackoverflow.com/a/52227201/6309)
>
> I mean, if you have a testcase and want to see how it works, you can
> just try it.  merge-ort has been released and is part of git already.
>
> But I'll take a stab...
>
> Your testcase with new folders sounds like testcase 12f from t6423 in
> our testsuite (from commit 902c521a35 ("t6423: more involved directory
> rename test", 2020-10-15)), a case that works with the `ort` merge
> strategy but not with `recursive`.  So, the `ort` strategy is likely
> to be better for your usecase than `recursive`, but it's hard for me
> to definitively state that it'll solve all your usecase(s) since you
> didn't fully describe them.  This is especially important since you
> appear to have discovered a segfault (in the first post); I was not
> aware of any circumstances when the old merge machinery would do so.
> I'd be interested to know if the same issue repeats with the `ort`
> strategy, and, if so, whether you can get a stack trace, or find an
> example that reproduces the problem, or provide us with access to your
> repository.  Also, in your second link, you allude to problems in a
> case that does not include new folders, in situations where I'd expect
> `recursive` and `ort` to behave the same.  But it's hard to know the
> expected behavior since you didn't provide that many details about the
> setup -- e.g. you didn't cover the contents of any of the files (so
> it's hard to know if files are more similar to ones with different
> names than what you expected to be matched up), it's unclear if there
> are other files involved (e.g. if you have multiple copies of some
> library and the rename detection is detecting a different copy than
> you expected, or if there are lots of additional files besides those
> you are listing in your example and they are being renamed
> differently, etc.), whether you have lots of large files also being
> renamed (perhaps the VDI files you allude to somewhere?), etc., etc.
> If you can provide access to a repository and the commands you
> reported and what you expected vs. what you saw, I'd try to find some
> time to take a look.
>
> I hope that helps, though I'm not sure I've got enough details to
> answer all your questions.  I guess if I were to summarize, I'd say:
>   * the `ort` strategy would be better than `recursive` for you, for
> one particular type of issue, but probably behave the same on other
> areas
>   * you seem to have useful testcases that might be different than
> ones we have in our testsuite or that I know about
>   * you haven't provided enough information for me to investigate or
> answer any further, though.
>
> You have gotten me really curious about this segfaulting testcase --
> whether it's just in the recursive backend or also afflicts merge-ort,
> and how to duplicate it.
Elijah Newren· Oct 1, 2021, 08:04 UTC · re: Ariel Man · lore

Re: Question about git merge -s ort command‏‏

Hi,
Please do not top-post on this mailing list.
On Wed, Sep 29, 2021 at 11:58 PM Ariel Man <arielman22@gmail.com> wrote:
>
> Hi,
> Thanks for the detailed answer, but just to clarify, my main question
> is regarding ORT strategy, and not recursive.

Since you simply linked to three posts in your original email, and the first showed a segfault, this wasn't at all clear.

Show 12 quoted lines
> Mainly in this case, where ORT didn't behave as I expected:
> https://stackoverflow.com/questions/69150777/git-mv-doesnt-work-as-expected-wrong-files-location#:~:text=test3.py%0A%E2%94%82%20%20%20%E2%94%94%E2%94%80%E2%94%80%20dir3-,UPDATE,-I%27m%20adding%20one
>
> As I wrote, this specific case is where new folder and files were
> created after the consolidation (and not new files under the existing
> folder). As you can see, when only new files were added to the
> existing folder, the git ORT recognized it and moved the folder
> python1 with its files test1.py and test4.py to the right place
> (dir2). But when new folder was created on the target (new_test with
> the file test5.py), when I ran the ORT command, the GIT didn't
> recognized it and created again the folder python1 on the old path
> (dir1-->python1-->new_test-->test5.py).

Yes, I read through that and pointed out your examples weren't fully specified. You haven't stated what else exists in all these directories, what kind of contents files had, if there were big binaries, etc.

> I believe you can reproduce it yourself on your repository and see the
> behaviour.

Nope, I can't. I went through the trouble of setting up the testcases as close as I could from your diagrams. When using the recursive strategy, I do see that test4.py and test5.py are left behind in dir1, but when using the ort strategy, it behaves exactly as you said you wanted (with *both* test4.py and test5.py appearing somewhere under dir2 rather than dir1).

Here's the output from my script showing all the files that exist in each of the relevant commits:

=== Original === dir1/otherfile dir1/python1/test1.py dir2/otherfile dir2/python2/test2.py dir3/otherfile dir3/python3/test3.py

=== Consolidated === dir1/otherfile dir2/otherfile dir2/python1/test1.py dir2/python2/test2.py dir2/python3/test3.py dir3/otherfile

=== New files === dir1/otherfile dir1/python1/new_test/test5.py dir1/python1/test1.py dir1/python1/test4.py dir2/otherfile dir2/python2/test2.py dir3/otherfile dir3/python3/test3.py

=== Merged result === dir1/otherfile dir2/otherfile dir2/python1/new_test/test5.py dir2/python1/test1.py dir2/python1/test4.py dir2/python2/test2.py dir2/python3/test3.py dir3/otherfile

and here's the script I used to run this testcase and generate the above output:
#!/bin/bash

git init -b main nukeme cd nukeme

echo '''
│   └── dir1
│   │   └── python1
│   │   │   └── test1.py
│   └── dir2
│   │   └── python2
│   │   │   └── test2.py
│   └── dir3
│   │   └── python3
│   │   │   └── test3.py
''' >/dev/null
for i in $(seq 1 3); do
    mkdir -p dir$i/python$i
    echo $i >dir$i/python$i/test$i.py
    >dir$i/otherfile
done

git add . git commit -qm "Initial"

git branch consolidated git branch newfiles

git checkout consolidated echo ''' │ └── dir1 │ └── dir2 │ │ └── python1 │ │ │ └── test1.py │ │ └── python2 │ │ │ └── test2.py │ │ └── python3 │ │ │ └── test3.py │ └── dir3 ''' >/dev/null git mv dir1/python1 dir2/ git mv dir3/python3 dir2/ git commit -qm consolidated

git checkout newfiles echo 4 >dir1/python1/test4.py git add dir1/python1/test4.py mkdir dir1/python1/new_test echo 5 >dir1/python1/new_test/test5.py git add dir1/python1/new_test/test5.py git commit -qm newfiles

git checkout -qb merged consolidated git merge --no-edit -s ort newfiles

echo echo "=== Original ===" git ls-tree -r --name-only main echo echo "=== Consolidated ===" git ls-tree -r --name-only consolidated echo echo "=== New files ===" git ls-tree -r --name-only newfiles echo echo "=== Merged result ===" git ls-tree -r --name-only merged

← back to recent threads