Re: [PATCH 11/19] pack-objects: use bitmaps when packing objects

From: Shawn Pearce <spearce@spearce.org>
Date: Oct 25, 2013, 14:14 UTC
Message-ID: <CAJo=hJuDx=3AOoz2oEORVOzeBYBwvOWO_ye8D5d8PcDc3Zm+Ew@mail.gmail.com>
In-Reply-To: <20131024180419.GK24180@sigill.intra.peff.net>

On Thu, Oct 24, 2013 at 6:04 PM, Jeff King <peff@peff.net> wrote:

Show 7 quoted lines

> For bitmaps to be used, the following must be true:
>
>   1. We must be packing to stdout (as a normal `pack-objects` from
>      `upload-pack` would do).
>
>   2. There must be a .bitmap index containing at least one of the
>      "have" objects that the client is asking for.

The client must explicitly "have" a commit that has a bitmap? In JGit we allow the client to have anything, and walk backwards using traditional graph traversal until a bitmap is found.

Show 19 quoted lines

> @@ -704,6 +759,18 @@ static void write_pack_file(void)
>                 offset = write_pack_header(f, nr_remaining);
>                 if (!offset)
>                         die_errno("unable to write pack header");
> +
> +               if (reuse_packfile) {
> +                       off_t packfile_size;
> +                       assert(pack_to_stdout);
> +
> +                       packfile_size = write_reused_pack(f);
> +                       if (!packfile_size)
> +                               die_errno("failed to re-use existing pack");
> +
> +                       offset += packfile_size;
> +               }
> +
>                 nr_written = 0;
>                 for (; i < to_pack.nr_objects; i++) {
>                         struct object_entry *e = write_order[i];

Can reuse_packfile be true at the same time as to_pack.nr_objects > 0?

In JGit we write the to_pack list first, then the reuse pack. Our rationale was the to_pack list is recent objects that are newer and would appear first in a traditional traversal, so they should go at the front of the stream. This does mean if they delta compress against an object in that reuse_packfile slice they have to use REF_DELTA instead of OFS_DELTA.

Is this series running on github.com/torvalds/linux? Last Saturday I ran a live demo clone comparing github.com/torvalds/linux to a JGit bitmap clone and some guy heckled me because GitHub was only a few seconds slower. :-)