Re: [PATCH] Speedup recursive by flushing index only once for all entries

From: Alex Riesen <raa.lkml@gmail.com>
Date: Jan 5, 2007, 11:22 UTC
Message-ID: <81b0412b0701050322u67131900xea969b2da9981a94@mail.gmail.com>
In-Reply-To: <7v8xgileza.fsf@assigned-by-dhcp.cox.net>

On 1/4/07, Junio C Hamano <junkio@cox.net> wrote:

Show 9 quoted lines

> >> However, I was wondering if the index has to be written at all.
> >> I expect the written index (except the last one, of course) to have no
> >> user...
> >
> > Good question...
>
> That's most likely because you played safe, and started from the
> Python version whose only way to manipulate the index and write
> out a tree was to actually write the index out.

Yes, it was because of that. Just didn't thought about when to update index at all (never really had a time).

> So let's step back a bit.

Excellent analysis, thanks! I suspect heavily it will work as is. Now, if only someone could find time to code it up...

Show 7 quoted lines

> By the way, Alex, you seem to heavily work on Cygwin, and I am
> interested in your experience with Shawn's sliding mmap() on
> real projects, as I suspect Cygwin would be more heavily
> impacted with any mmap() related changes.  You already said
> performance is "bearable".  Does that mean it was better and
> got worse but still bearable, or it was unusably slow but now it
> is bearably usable?

It is usably slow: ~30 sec for a commit (I stopped using normal commit, using update-index and simplified index commit now), around minute for the recursive merges (if they are simple), ~10 sec for a hot-cache hard reset. Avoiding gitk whenever possible.

Compared to my only linux system here:

2-3 times slower in diff-tree for 44K files, around 9k differences (0.2 sec against 0.6 sec, it quickly adds up if you do it often, like when merging (it's slower for really big merges, constrained by CPU and memory), commiting, gitk).

The windows machine is a corporate Lenovo 2.66MHz/1Gb,SATA laptop. The linux machine is 1.2MHz/384Mb, IDE noname notebook.

I somehow adapted myself to it though (reading mails, drinking coffee).