Re: [PATCH 8/9] sha1-file: use loose object cache for quick existence check

From: Geert Jansen <gerardu@amazon.com>
Date: Nov 12, 2018, 22:44 UTC
Message-ID: <20181112224445.GA32727@amazon.com>
In-Reply-To: <20181112162150.GB7612@sigill.intra.peff.net>

On Mon, Nov 12, 2018 at 11:21:51AM -0500, Jeff King wrote:

Show 19 quoted lines

> No, but they don't even really need to be actual objects. So I suspect
> something like:
> 
>   git init
>   for i in $(seq 256); do
>     i=$(printf %02x $i)
>     mkdir -p .git/objects/$i
>     for j in $(seq --format=%038g 1000); do
>       echo foo >.git/objects/$i/$j
>     done
>   done
>   git index-pack -v --stdin </path/to/git.git/objects/pack/XYZ.pack
> 
> might work (for various values of 1000). The shell loop would probably
> be faster as perl, too. :)
> 
> Make sure you clear the object directory between runs, though (otherwise
> the subsequent index-pack's really do find collisions and spend time
> accessing the objects).

Below are my results. They are not as comprehensive as Ævar's tests. Similary I kept the loose objects between tests and removed the packs instead. And I also used the "echo 3 | sudo tee /proc/sys/vm/drop_caches" trick :)

This is with git.git:

                   origin/master    jk/loose-object-cache

256*100 objects 520s 13.5s (-97%) 256*1000 objects 826s 59s (-93%)

I've started a 256*10K setup but that's still creating the 2.5M loose objects. I'll post the results when it's done. I would expect that jk/loose-object-cache is still marginally faster than origin/master based on a simple linear extrapolation.