threads / discuss / 62471

Extremely long subtree split duration

Subject: Extremely long subtree split duration

## tl;dr

6 messages between Nov 8, 2024 and Nov 29, 2024.

replies: 5people: 2as markdown or json

Peter Kassak· Nov 8, 2024, 21:40 UTC · lore

Windows has always been a bit slower with subtree split, but now we encountered an extreme difference.

The test is on the opensource repository of bitnami charts:

git clone https://github.com/bitnami/charts.git cd charts git subtree split -P bitnami/minio -b split

On live Ubuntu 24.04 booted from usb stick, it took about one and a half minutes.

On Windows 11 installed on internal nvme, it took almost 4 hours.

Yes, it sounds crazy, but it really took three hours and 53 minutes on the same hardware.

Main difference (except OS) is git version. On Ubuntu is latest installed via apt 2.43.0, on Windows is 2.47.0.windows.2.

I have tried several different OS in docker, virtualbox, different HW, different git versions..

Long story short, it looks like since version 2.44.0-rc0, probably after merging 'zf/subtree-split-fix', subtree split is much much slower.

And practically unusable on Windows.

Can this be fixed?

Thanks, Peter "Cache" Kassak

Kristoffer Haugsbakk· Nov 8, 2024, 22:07 UTC · re: Peter Kassak · lore

Re: Extremely long subtree split duration

On Fri, Nov 8, 2024, at 22:40, Peter Kassak wrote:

> Main difference (except OS) is git version. On Ubuntu is latest
> installed via apt 2.43.0, on Windows is 2.47.0.windows.2.

I tried a bit on Ubuntu 22.04, Git 2.47.0. I gave it 14 minutes and then it was apparently about 10000/32714 done.

-- 
Kristoffer Haugsbakk

Kristoffer Haugsbakk· Nov 9, 2024, 14:27 UTC · re: Kristoffer Haugsbakk · lore

Re: Extremely long subtree split duration

On Fri, Nov 8, 2024, at 23:07, Kristoffer Haugsbakk wrote:

Show 9 quoted lines

> On Fri, Nov 8, 2024, at 22:40, Peter Kassak wrote:
>> Main difference (except OS) is git version. On Ubuntu is latest
>> installed via apt 2.43.0, on Windows is 2.47.0.windows.2.
>
> I tried a bit on Ubuntu 22.04, Git 2.47.0.  I gave it 14 minutes and then
> it was apparently about 10000/32714 done.
>
> --
> Kristoffer Haugsbakk

I tried again and ran the whole command. It took 3 hours and 43 minutes. My `/bin/sh` is dash(1).

I then checked out the first parent of the commit that you pointed to: bcf524023e (Merge branch 'zf/subtree-split-fix', 2024-02-02). In other words I checked out bbc8c05670 (Merge branch 'jc/ls-files-doc-update', 2024-02-02).

A *split* on that took 3m49s.

-- 
Kristoffer Haugsbakk

Peter Kassak· Nov 9, 2024, 21:10 UTC · re: Kristoffer Haugsbakk · lore

Re: Extremely long subtree split duration

Hello,

I didn't compile my own version from sources, I wrote that it was probably after 'zf/subtree-split-fix', because it was related to subtree split.

And I managed to find out by trying that the change happened after version 2.43.0 and was included in version 2.44.0-rc0, so i assumed that.

Go on and try same split with 2.43.0.

Peter "Cache" Kassak.

On Sat, Nov 9, 2024 at 3:28 PM Kristoffer Haugsbakk <kristofferhaugsbakk@fastmail.com> wrote:

Show 24 quoted lines

>
> On Fri, Nov 8, 2024, at 23:07, Kristoffer Haugsbakk wrote:
> > On Fri, Nov 8, 2024, at 22:40, Peter Kassak wrote:
> >> Main difference (except OS) is git version. On Ubuntu is latest
> >> installed via apt 2.43.0, on Windows is 2.47.0.windows.2.
> >
> > I tried a bit on Ubuntu 22.04, Git 2.47.0.  I gave it 14 minutes and then
> > it was apparently about 10000/32714 done.
> >
> > --
> > Kristoffer Haugsbakk
>
> I tried again and ran the whole command.  It took 3 hours and 43
> minutes.  My `/bin/sh` is dash(1).
>
> I then checked out the first parent of the commit that you pointed to:
> bcf524023e (Merge branch 'zf/subtree-split-fix', 2024-02-02).  In other
> words I checked out bbc8c05670 (Merge branch 'jc/ls-files-doc-update',
> 2024-02-02).
>
> A *split* on that took 3m49s.
>
> --
> Kristoffer Haugsbakk

Kristoffer Haugsbakk· Nov 29, 2024, 19:44 UTC · re: Peter Kassak · lore

Re: Extremely long subtree split duration

Hi again

On Sat, Nov 9, 2024, at 22:10, Peter Kassak wrote:

Show 13 quoted lines

> Hello,
>
> I didn't compile my own version from sources, I wrote that it was
> probably after 'zf/subtree-split-fix', because it was related to
> subtree split.
>
> And I managed to find out by trying that the change happened after
> version 2.43.0 and was included in version 2.44.0-rc0, so i assumed
> that.
>
> Go on and try same split with 2.43.0.
>
> Peter "Cache" Kassak.

Back when I read this I thought I checked these. So I didn’t do anything more.

I just came back to this since I found this project idea about rewriting this contrib/ script in C (make it a builtin):

https://github.com/gitgitgadget/git/issues/1410

I’ll link this thread on that issue.

-- 
Kristoffer Haugsbakk

Kristoffer Haugsbakk· Nov 29, 2024, 19:50 UTC · re: Kristoffer Haugsbakk · lore

Re: Extremely long subtree split duration

I don’t use this command and I am an outsider to the code (to all of the code in this project). But the commit that introduced this apparent performance regression does a lot of `git log -1` calls. Very apparent when I ran it since I got a lot of stderr output about missing note refs (since I use them in another project).

For what it’s worth.