Re: thin packs ending up fat
On Thu, 12 Jan 2012, Junio C Hamano wrote:
Show 28 quoted lines
> From: Jeff King <peff@peff.net>
> Subject: [PATCH] thin-pack: try harder to create delta against preferred base
>
> When creating a pack using objects that reside in existing packs, we try
> to avoid recomputing futile delta between an object (trg) and a candidate
> for its base object (src) if they are stored in the same packfile, and trg
> is not recorded as a delta already. This heuristics makes sense because it
> is likely that we tried to express trg as a delta based on src but it did
> not produce a good delta when we created the existing pack.
>
> As the pack heuristics prefer producing delta to remove data, and Linus's
> law dictates that the size of a file grows over time, we tend to record
> the newest version of the file as inflated, and older ones as delta
> against it.
>
> When creating a thin-pack to transfer recent history, it is likely that we
> will try to send an object that is recorded in full, as it is newer. But
> the heuristics to avoid recomputing futile delta effectively forbids us
> from attempting to express such an object as a delta based on another
> object. Sending an object in full is often more expensive than sending a
> suboptimal delta based on other objects, and it is even more so if we
> could use an object we know the receiving end already has (i.e. referred
> base object) as the delta base.
>
> Tweak the recomputation avoidance logic, so that we do not punt on
> computing delta against a preferred base object.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Show 31 quoted lines
> ---
>
> builtin/pack-objects.c | 9 +++++++--
> 1 files changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index c6e2d87..8bfe3a6 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -1248,11 +1248,16 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
> return -1;
>
> /*
> - * We do not bother to try a delta that we discarded
> - * on an earlier try, but only when reusing delta data.
> + * We do not bother to try a delta that we discarded on an
> + * earlier try, but only when reusing delta data. Note that
> + * src_entry that is marked as the preferred_base should always
> + * be considered, as even if we produce a suboptimal delta against
> + * it, we will still save the transfer cost, as we already know
> + * the other side has it and we won't send src_entry at all.
> */
> if (reuse_delta && trg_entry->in_pack &&
> trg_entry->in_pack == src_entry->in_pack &&
> + !src_entry->preferred_base &&
> trg_entry->in_pack_type != OBJ_REF_DELTA &&
> trg_entry->in_pack_type != OBJ_OFS_DELTA)
> return 0;
> --
> 1.7.9.rc0.53.gc8bc2
>
Nicolas Pitre <nico@fluxnic.net>