Re: [PATCH] Implement limited context matching in git-apply.

From: Eric W. Biederman <ebiederm@xmission.com>
Date: Apr 13, 2006, 12:02 UTC
Message-ID: <m1mzep65uc.fsf@ebiederm.dsl.xmission.com>
In-Reply-To: <Pine.LNX.4.64.0604111100510.10745@g5.osdl.org>

Linus Torvalds <torvalds@osdl.org> writes:

Show 25 quoted lines

> On Mon, 10 Apr 2006, Eric W. Biederman wrote:
>> 
>> So at a quick inspection it looks to me like:
>> About .059s to perform to check for missing files.
>> About .019s to write the new tree.
>> About .155s in start up overhead, read_cache, and sanity checks.
>> 
>> So at a first glance it looks like librification to
>> allow the redundant work to be skipped, is where
>> the big speed win on my machine would be.
>
> That sounded wrong to me, so I did a stupid patch to datestamp the 
> different phases of git-write-tree, and here's what it says for me:
>
>      0.000479 setup_git_directory
>      0.008333 read_cache
>      0.000813 ce_stage check
>      0.001838 tree validity check
>      0.037233 write_tree itself
>
> 	real    0m0.051s
> 	user    0m0.044s
> 	sys     0m0.008s
>
> all times are in seconds.

Ok. This is interesting and probably reveals what is different about my setup. For you user+sys = real. For me there was a significant gap. So it looks like for some reason I was not succeeding in keeping .git/index hot in the page cache.

When you are I/O bound it does make sense for read_cache to be the dominate time. I just need to track what is up with my machine that makes me I/O bound. Having too little ram is an obvious candidate but it is too simple. Currently out of 512M I only have 21M in the page cache which sounds really low. Something for me to look at.

Show 6 quoted lines

> Which would imply pretty major surgery - you'd have to add the tree entry 
> information to the index file, and make sure they got invalidated properly 
> (all the way to the root) whenever adding/deleting/updating a path in the 
> index file.
>
> Quite frankly, I don't think it's really worth it.

For the current size of the kernel tree I agree.

It is a potential scaling limitation and if someone starts tracking really big tress with git it may be worth revisiting.

Show 5 quoted lines

> Yes, it would speed up applying of huge patch-sets, but it's not like 
> we're really slow at that even now, and I suspect you'd be better off 
> trying to either live with it, or trying to see if you could change your 
> workflow. There clearly _are_ tools that are better at handling pure 
> patches, with quilt being the obvious example.

Probably. For my workflow not having to switch tool chains is the biggest win. Which is part of what the -C is about.

Show 8 quoted lines

> I routinely apply 100-200 patches in a go, and that's fast enough to not 
> even be an issue. Yes, I have reasonably fast hardware, but we're likely 
> talking thousands of patches in a series for it to be _really_ painful 
> even on pretty basic developer hardware. Even a slow machine should do a 
> few hundred patches in a couple of minutes.
>
> Maybe enough time to get a cup of coffee, but no more than it would take 
> to compile the project.

Agreed. I did the analysis so I could understand what was going on. If the analysis revealed low hanging fruit I would have plucked it.

Eric