Re: caching commit patch-ids for fast git-cherry

From: Geoffrey Irving <irving@naml.us>
Date: May 29, 2008, 17:34 UTC
Message-ID: <7f9d599f0805291034l6c655ccbk219dd74964c65737@mail.gmail.com>
In-Reply-To: <alpine.DEB.1.00.0805291809340.13507@racer.site.net>

On Thu, May 29, 2008 at 10:13 AM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 19 quoted lines

> Hi,
>
> On Thu, 29 May 2008, Geoffrey Irving wrote:
>
>> I'm planning to use cherry picking to manage long term syncing between
>> cvs/perforce and git repositories.  This means I'll have scripts running
>> git-cherry between branches with hundreds of uncommon commits, and I
>> want git-cherry to be much, much, faster.
>>
>> It looks like I can do this by caching commit->patch-id pairs from
>> commit_patch_id() in patch-ids.c to a file, say
>> $GIT_DIR/commit-patch-id-cache.  The file would be binary and append
>> only, and could be blown away if .  Any suggestions / concerns before
>> I write this?  Is there any reusable efficient map code for storing
>> the commit->patch-id map, or should I just mirror the blocked storage
>> + binary search used for struct patch_ids?
>
> I would store the stuff sorted, so that the lookup is fast, generation
> less so.

The motivation for append-only was robustness, not speed, but I don't think either concern is very significant.

> For inspiration, you might want to look at the "notes" branch in my
> personal fork:
>
> http://repo.or.cz/w/git/dscho.git?a=shortlog;h=refs/heads/notes

Cool. I'd rather copy just that code entirely rather than use it for inspiration, since it does exactly what I need. It would be silly to have two blocks of code implementing "persistent map from 20 byte hash to 20 byte hash".

I'll start by just copying the entire nodes-index implementation (with a few name substitutions), and we (or I) can refactor it later if both end up in the same respository.

Geoffrey