Re: [PATCHv3 3/9] pack-objects: Allow --max-pack-size to be used together with --stdout

From: Shawn Pearce <spearce@spearce.org>
Date: May 15, 2011, 22:06 UTC
Message-ID: <BANLkTingc375Kzm-TBS4aQ_912pg7G03cQ@mail.gmail.com>
In-Reply-To: <1305495440-30836-4-git-send-email-johan@herland.net>

On Sun, May 15, 2011 at 14:37, Johan Herland <johan@herland.net> wrote:

Show 8 quoted lines

> Currently we refuse combining --max-pack-size with --stdout since there's
> no way to make multiple packs when the pack is written to stdout. However,
> we want to be able to limit the maximum size of the pack created by
> --stdout (and abort pack-objects if we are unable to meet that limit).
>
> Therefore, when used together with --stdout, we reinterpret --max-pack-size
> to indicate the maximum pack size which - if exceeded - will cause
> pack-objects to abort with an error message.

...

>                if (pack_to_stdout) {
> +                       if (nr_written != nr_remaining)
> +                               die("unable to make pack within the pack size"
> +                                   " limit (%lu bytes)", pack_size_limit);

I think this is too late. We have already output a bunch of data, up to the size limit at this point. If the size limit is non-trivial (e.g. 5 MB) we have already sent most of that to the remote side, and its already written some of that out to disk.

I'd like this to be a soft limit derived from the reused object sizes. When planning the pack by looking at where we will reuse an object from, sum those sizes. If the sum of these sizes would break this limit, then we abort before even writing the pack header out.

-- 
Shawn.