Re: fetching packs and storing them as packs

From: Eran Tromer <git2eran@tromer.org>
Date: Oct 27, 2006, 04:03 UTC
Message-ID: <4541850B.8060608@tromer.org>
In-Reply-To: <20061027030054.GB28407@spearce.org>

Hi,

On 2006-10-27 05:00, Shawn Pearce wrote:

Show 8 quoted lines

>> Change git-repack to follow references under $GIT_DIR/tmp/refs/ too.
>> To receive or fetch a pack:
>> 1. Add references to the new heads in
>>    `mktemp $GIT_DIR/tmp/refs/XXXXXX`.
>> 2. Put the new .pack under $GIT_DIR/objects/pack/.
>> 3. Put the new .idx under $GIT_DIR/objects/pack/.
>> 4. Update the relevant heads under $GIT_DIR/refs/.
>> 5. Delete the references from step 1.

> That was actually my (and also Sean's) solution.  Except I would
> put the temporary refs as "$GIT_DIR/refs/ref_XXXXXX" as this is
> less code to change and its consistent with how temporary loose
> objects are created.

If you do that, other programs (e.g., anyone who uses rev-list --all) may try to walk those heads or consider them available before the pack is really there. The point about $GIT_DIR/tmp/refs is that only programs meddling with physical packs (git-fetch, git-receive-pack, git-repack) will know about it.

Show 10 quoted lines

> What happens when the incoming pack (steps #2 and #3) takes 15
> minutes to upload (slow ADSL modem, lots of objects) and the
> background repack process sees those temporary refs and starts
> trying to include those objects?  It can't walk the DAG that those
> refs point at because the objects aren't in the current repository.
> 
>>From what I know of that code the pack-objects process will fail to
> find the object pointed at by the ref, rescan the packs directory,
> find no new packs, look for the object again, and abort over the
> "corruption".

Good point. Then I guess we'll need to change git-repack to ignore missing objects if they're referenced from $GIT_DIR/tmp/refs but not from $GIT_DIR/refs. Ugly, but shouldn't be too hard.