Re: [PATCH 2/8] pack-objects: produce a stable pack when --skip is given
- From
Duy Nguyen <pclouds@gmail.com>
- Date
- Feb 5, 2016, 23:25 UTC
- Message-ID
- <CACsJy8CkQPX4zqW39gpODSkU+habM7TGTJ85MU-S0UuNPk9SCw@mail.gmail.com>
- In-Reply-To
- <xmqq1t8rdmky.fsf@gitster.mtv.corp.google.com>
On Sat, Feb 6, 2016 at 1:43 AM, Junio C Hamano <gitster@pobox.com> wrote:
Show 25 quoted lines
> Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
>
>> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
>> index 417c830..c58a9cb 100644
>> --- a/builtin/pack-objects.c
>> +++ b/builtin/pack-objects.c
>> @@ -2709,6 +2709,11 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
>> if (get_oid_hex(skip_hash_hex, &skip_hash))
>> die(_("%s is not SHA-1"), skip_hash_hex);
>> }
>> +
>> + /*
>> + * Parallel delta search can't produce stable packs.
>> + */
>> + delta_search_threads = 1;
>> }
>>
>> argv_array_push(&rp, "pack-objects");
>
> A multi-threaded packing is _one_ source of regenerating the same
> pack for the same set of objects, but we shouldn't be tying our
> hands by promising it will forever be the _only_ source of it by
> doing things like this. We may want to dynamically tweak the
> packing behaviour depending on the load of the minute and such for
> example.You noticed that tying the behavior only happens when the user asks for it, right? I don't expect people to do resumable fetch/clone by default. There are tradeoffs to make and they decide it, we offer options. So, it does not really tie our hands in the normal case.
Show 11 quoted lines
> I think a more sensible approach for "resuming" is to attack cloning > first. Take a reasonable baseline snapshot periodically (depending > on the activity level of the project, the interval may span between > 12 hours to 2 weeks and you would want to make it configurable) to > create a bundle, teach "clone" to check the bundle first and perform > a resumable and bulk transfer for the stable part of the history > (e.g. up to the latest tag or a branch that does not rewind, the set > of refs to use as the stable anchors you would want to make > configurable), and then fill the gap between that baseline snapshot > and up-to-date state by doing another round of "git fetch" and then > you'd have solved the most serious problem already.
_most_. On a troubled connection, fetch can fail as well, especially when the repo is not uptodate. What about shallow clone? I don't think you want to prepare snapshots for all depths (or some "popular" depths). Cloning full then cutting it shallow locally might work, but we're wasting bandwidth and disk space.
> This is an indication that the approach this series takes is taking > us in a wrong direction.
The way I see it, the two approaches complement each other. Snapshots, like pack bitmaps, helps tremendously, but it has corner cases that can be covered by this.
-- Duy