Re: q: faster way to integrate/merge lots of topic branches?

From: Pierre Habouzit <madcoder@artemis.madism.org>
Date: Jul 23, 2008, 20:27 UTC
Message-ID: <20080723202722.GA18160@artemis.madism.org>
In-Reply-To: <20080723190920.GG20614@artemis.madism.org>

On Wed, Jul 23, 2008 at 07:09:20PM +0000, Pierre Habouzit wrote:

Show 21 quoted lines

> On Wed, Jul 23, 2008 at 05:59:01PM +0000, Linus Torvalds wrote:
> > In fact, the two top entries in a profile look roughly like:
> > 
> > 	102161   70.2727  libz.so.1.2.3            libz.so.1.2.3            (no symbols)
> > 	7685      5.2862  git                      git                      find_pack_entry_one
> > 	...
> > 
> > ie 70% of the time is just purely unpacking the data, and another 5% is 
> > just finding it. We could perhaps improve on it, but not a whole lot.
> 
>   Well there is an easy way though, that could reduce that: using
> adaptative compression. I proposed a patch once upon a time, that set
> the compression strengh to 0 for "small" objects with a configurable
> cut-off. If you do that, most trees, commits messages and so on aren't
> compressed, and it will reduce (with IIRC a 5-liner) this time quite
> dramatically.
> 
>   I could maybe resurect it to see if for people that do the kind of
> things Ingo does it helps. By setting the cut-off at 1k, I had packs
> being less than 1% bigger IIRC. I'll try to find it again and run your
> tests with it to see how much it helps.

  Unsurprisingly with a 1024o cutoff, the numbers are (first run is
forced cold-cache with /proc/.../drop_caches, second is the best run of 5):

default git:

    3.10user 0.16system 0:08.10elapsed 40%CPU (0avgtext+0avgdata 0maxresident)k
    116152inputs+0outputs (671major+35286minor)pagefaults 0swaps

    2.01user 0.11system 0:02.12elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
    0inputs+0outputs (0major+35958minor)pagefaults 0swaps

With a 1024k cutoff:

    1.16user 0.13system 0:08.29elapsed 15%CPU (0avgtext+0avgdata 0maxresident)k
    154208inputs+0outputs (947major+39777minor)pagefaults 0swaps

    0.76user 0.06system 0:00.82elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
    0inputs+0outputs (0major+40724minor)pagefaults 0swaps

According to [0], a 1k cutoff meant something like a 10% larger pack. 512o meant an almost identical pack in size, but with reduced performance improvements.

  [0] http://thread.gmane.org/gmane.comp.version-control.git/70019/focus=70250

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org