Re: [PATCH 00/10] RFC Partial Clone and Fetch

From: Junio C Hamano <gitster@pobox.com>
Date: May 8, 2017, 00:15 UTC
Message-ID: <xmqqwp9scifu.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <20170503182725.GC28740@aiede.svl.corp.google.com>

Jonathan Nieder <jrnieder@gmail.com> writes:

> - there shouldn't be any need for the blobs to even be mentioned in
>   the .pack stored locally.  The .idx file maps from sha1 to offset
>   within the packfile --- a special offset could mean "this is a
>   missing blob".

Clever.

Show 6 quoted lines

> - However, the list of missing blobs can be inferred from the existing
>   pack format, without a change to the pack format used in git
>   protocol.  As part of constructing the idx, "git index-pack"
>   inflates every object in the pack file sent by the server.  This
>   means we know what blobs they reference, so we can easily produce a
>   list for the idx file without changing the pack file format.

A minor wrinkle to keep in mind if you were to go this route is that you'd need a way to tell the reason why a blob that is referenced by a tree in the pack stream is not in the same pack stream.

If the resulting repository on the receiving side has that blob after the transfer, it is likely that the reason why the blob does not appear in the pack is because the want/have/ack exchange told the sending side that the receiving side has a commit that contains the blob. But when the blob does not exist in the receiving side after the transfer, we cannot tell between two possible cases. The server may have actively wanted to omit it (i.e. the case we are interested in in this discussion thread). Or the receiving end said that it has a commit that contains the blob, but due to preexisting corruption, the receiving repository was missing the blob in reality.