Re: GSoC proposal: port pack bitmap support to libgit2.

From: Yuxuan Shui <yshuiv7@gmail.com>
Date: Mar 20, 2014, 13:38 UTC
Message-ID: <CAGqt0zwp2a6wrzLY0KOmH1zd0Bj5wmwvyBjy0j1JDjm3X4cSQg@mail.gmail.com>
In-Reply-To: <20140314043404.GD31906@sigill.intra.peff.net>

Hi,

Sorry for this late reply, I was busy for past few days.

On Fri, Mar 14, 2014 at 12:34 PM, Jeff King <peff@peff.net> wrote:

Show 23 quoted lines

> On Wed, Mar 12, 2014 at 04:19:23PM +0800, Yuxuan Shui wrote:
>
>> I'm Yuxuan Shui, a undergraduate student from China. I'm applying for
>> GSoC 2014, and here is my proposal:
>>
>> I found this idea on the ideas page, and did some research about it.
>> The pack bitmap patchset add a new .bitmap file for every pack file
>> which contains the reachability information of selected commits. This
>> information is used to speed up git fetching and cloning, and produce
>> a very convincing results.
>>
>> The goal of my project is to port the pack bitmap implementation in
>> core git to libgit2, so users of libgit2 could benefit from this
>> optimization as well.
>>
>> Please let me know if my proposal makes sense, thanks.
>
> You'd want to flesh it out a bit more to show how you're thinking about
> tackling the problem:
>
>   - What are the areas of libgit2 that you will need to touch? Be
>     specific. What's the current state of the packing code? What
>     files and functions will you need to touch?

Firstly I will need to implement bitmap creation in libgit2's git_packbuilder_* functions (probably also git_odb_write_pack), so libgit2 could support bitmap creation. Then I will need to change git_revwalk_* functions to make them use bitmap. Since the operations that can benefit from bitmap is, if my understanding is correct, all using the git_revwalk_* functions, having bitmap support in revwalk functions should be enough.

Files I need to touch probably are: revwalk.c pack-objects.c If I need to change the API of packbuilder or revwalk functions I will have to change the callers as well: push.c fetch.c and transport/smart_protocol.c

I haven't read all the code to put together a list of functions I need to change, but I think the list will be long.

>
>   - What are the challenges you expect to encounter in porting the code?

The architecture differences between git and libgit2 will probably be a challenge.

>
>   - Can you give a detailed schedule of the summer's work? What will you
>     work on in each week? What milestones do you expect to hit, and
>     when?

I don't really have a plan, but I'll try to provide a rough schedule.

I'll read the code and try to understand the code, to the point where I can start to add new code. This will probably take a week. For next three or four weeks I should be implementing bitmap creation in packbuilder. Then for the rest of time I will be optimizing revwalk using bitmap.

>
> -Peff

--

Regards Yuxuan Shui