Re: object/pack size x5 larger than a fresh clone?

From: Hin-Tak Leung <hintak.leung@gmail.com>
Date: Jul 27, 2010, 21:15 UTC
Message-ID: <AANLkTikDGzc5d=1w43eJFyO4JdcHEqYYaEm=1P-LpqCj@mail.gmail.com>
In-Reply-To: <20100727170317.GC25268@spearce.org>

On 7/27/10, Shawn O. Pearce <spearce@spearce.org> wrote:

Show 27 quoted lines

> Junio C Hamano <gitster@pobox.com> wrote:
>> Hin-Tak Leung <hintak.leung@gmail.com> writes:
>>
>> > So I guess these *.idx without a corresponding *.pack are safe to
>> > delete? But git gc or one of the other house keeping commands should
>> > get rid of them though, I think.
>>
>> I agree.  I think the dumb transports like http:// grab *.idx files
>> without downloading corresponding *.pack files when they encounter an
>> object that is not found loose in the originating repository to see which
>> packfile to fetch, but after they are done (or when they are interrupted,
>> for that matter), these *.idx files may not be getting garbage-collected.
>>
>> And they should be, perhaps with or without some grace period (I don't
>> know which offhand---I didn't think this through).
>
> We should GC these, but only after a grace period.
>
> Long ago when I used dumb http it really helped to have the *.idx
> files cached.  If the upstream only did an incremental repack holding
> onto the *.idx files locally meant I didn't need to redownload
> them in order to rule-out those packs as onces interesting for the
> current fetch.
>
> Maybe we just prune those during git fetch if they don't have a
> local *.pack and they don't match a pack listed by the remote's
> objects/info/packs file?

Okay, so they are left-overs from using http:// for fetching but serves a useful purpose for a limited period. The usual gc --prune defaults to 2 weeks, is that good enough? Or should there be a longer grace period?

I only switch over to git:// these last few days after it failing to fetch for two weeks and it looks like only I have this problem and around the web, failure-to-fetch seems to indicate an http proxy problem.

FWIW, my left-over files seems to co-incide with the 2-3 week snapshot release schedule of wine. I don't know if any of you is familiar with wine, but only one person (AJ) has commit rights and he reviews all patches and does a periodic push, usually just before or after a snapshot release but occasionally more frequent. My left-overs seems to co-incide with the first fetch after such a push. My most recent "permanent" fetch failure is due to the long-awaited wine 1.2 release, I think.

Thanks for all the insights, and I hope a future git release will prune some of these left-over files after a period.

Hin-Tak